ON THE MAXIMUM ENTROPY APPROACH TO SPECTRAL ANALYSIS

Abstract

"Motion is more apt to be lost than got, and is always upon the decay", remarked Newton. But there were little attempts to view the various irreversible processes in relation to each other. This had to wait till the mid 19th century. There developed a marked tendency among the 19th century scientists to attribute any apparent randomness in natural phenomenon to a lack of sufficient knowledge about these.phenomena rather than to any real chance element in nature. And there remains at the present time a strongly entrenched view to the effect that Entropy is a subjective concept precisely because it is taken as a measure of," missing information", information which we might use but don't due to systems being incompletely satisfied. The increase in Entropy comes when a known distribution goes over into an unknown distribution. Gain in Entropy always means loss of information, and nothing more. It is a subjective concept, but we can express it in its least subjective form, as follows. If, on a page we read the description of a system, together with certain data which help to specify the system, the Entropy of system is determined by these specifications. If any of the essential data are erased, the Entropy becomes greater, if any essential data are added the Entropy becomes less. Our motivation for Maximum Entropy principle comes from purely logical considerations. If we do not have complete information about a distribution, I/u? best course is to be maximally unbiased and, accordingly, choose the most random possible distribution. Choice of any other distribution would mean use of additional information• not given to us and consequently, it contributes an illogical choice.