log sum of exponential

The so-called “log sum of exponentials” is a functional form commonly encountered in dynamic discrete choice models in economics. It arises when the choice-specific error terms have a type 1 extreme value distribution–a very common assumption. In these problems there is typically a vector v of length J which represents the non-stochastic payoff associated with each of J choices. In addition to vj, the payoff associated with choice j has a random component εj. If these components are independent and identically distributed across j and follow the type 1 extreme value distribution, then the expected payoff from choosing optimally is 𝔼[max{v1+ε1,…,vJ+εJ}]=ln[exp(v1)+…+exp(vJ)]. We need to numerically evaluate the expression on the right hand side for many different vectors v.

Special care is required to calculate this expression in compiled languages such as Fortran or C to avoid numerical problems. The function needs to work for v with very large and very small components. A large vj can cause overflow due to the exponentiation. Similarly, when the vj are large (in absolute value) and negative, the exponential terms vanish. Taking the logarithm of a very small number can result in underflow.

A simple transformation can avoid both of these problems. Consider the case where v=(a,b). Note that we can write exp(a)+exp(b)=[exp(a−c)+exp(b−c)]exp(c) for any c. Furthermore, we have ln[(exp(a−c)+exp(b−c))exp(c)]=ln[exp(a−c)+exp(b−c)]+c. We can choose c in a way that reduces the possibility of overflow. Underflow is also possible when taking the logarithm of a number close to zero, since ln(x)→−∞ as x→0. Thus, we also need to account for large negative elements of v.