Variational Inference

We can think of a function as a mapping that takes the value of a variable as the input and returns the value of the function as the output. The derivative of the function then describes how the output value varies as we make infinitesimal changes to the input value. Similarly, we can define a functional as a mapping that takes a function as the input and that returns the value of the functional as the output. An example would be the entropy H[p], which takes a probability distribution p(x) as the input and returns the quantity $$H[p]=\int_a^b p(x)logp(x),dx \tag1$$