Engineering Analysis/Expectation and Entropy

Expectation The expectation operator of a random variable is defined as: This operator is very useful, and we can use it to derive the moments of the random variable. Moments A moment is a value that contains some information about the random variable. The n-moment of a random variable is defined as: Mean The mean value, or the "average value" of a random variable is defined as the first moment of the random variable: We will use the Greek letter μ to denote the mean of a random variable. Central Moments A central moment is similar to a moment, but it is also dependent on the mean of the random variable: The first central moment is always zero. Variance The variance of a random variable is defined as the second central moment: The square-root of the variance, σ, is known as the standard-deviation of the random variable Mean and Variance the mean and variance of a random variable can be related by: This is an important function, and we will use it later. Entropy the entropy of a random variable is defined as: