Fibonacci Numbers, Caching and Closures

After a bit of hiatus, I am long overdue to get some code up on this blog. To give myself some direction, this is the start of an informal series that will attempt to shed some light on the functional programming ideas that have been sneaking into the C# world. I've got a lot that I want to write about so stay tuned...

The Problem

As an exercise (and because I'm a fan of cliché), I've been toying with Fibonacci numbers lately. I've been playing simply with the classic recursive Fibonacci equation. Please note that there are much faster algorithms to use to calculate Fibonacci numbers that don't use recursion. But, those tend to lack the simple elegance of the classic. My goal here is not to teach recursion (Fibonacci numbers aren't very good for that purpose anyway) but to explore some other possibilities.

In case you're not familiar with them, the basic idea is that any number after the first two starting numbers is the sum of the previous two numbers in the sequence. Huh? OK, maybe the idea is better represented by the following equation (stolen shamelessly from Wikipedia):

So, if n is 0, the answer is 0. If n is 1, the answer is 1. If n is 2, the answer is the Fibonacci of 1 plus the Fibonacci of 0, or 1. And so on. In sequence, Fibonacci numbers get big really quickly:

     n: 0   1   2   3   4   5   6   7   8   9   10  11  12  13  14  15  16  17
result: 0   1   1   2   3   5   8   13  21  34  55  89  144 233 377 610 987 1597

Here is the equation in C# code:

int Fibonacci(int n)
{
  if (n < 2)
    return n;
  else
    return Fibonacci(n - 1) + Fibonacci(n - 2);
}

That's pretty simple and works exactly as it should. But, there's one serious problem: it's extraordinarily slow. The problem with the Fibonacci equation is that the Fibonacci of every preceding number must be calculated (often several times) in order to calculate the Fibonacci of n. The following diagram illustrates the problem. It shows the calculations that are necessary to calculate the Fibonacci of 4.

As you can see, the number of calculations increases exponentially. To determine how serious the problem is, let's write a simple application to measure the performance of our Fibonacci function:

static void Main(string[] args)
{
  for (int i = 0; i <= 45; i++)
  {
    Stopwatch sw = Stopwatch.StartNew();
    int fib = Fibonacci(i);
    sw.Stop();

    Console.WriteLine("Fibonacci({0}) = {1:###,###,###,##0} [{2:0.###} seconds]", 
      i, fib, ((float)sw.ElapsedMilliseconds / 1000));
  }
}

The results start to get interesting around the 25th iteration:

At the risk of belaboring this, let's try a different approach. With a small adjustment to our Fibonacci function, we can measure the number of steps that it takes instead of measuring time. After this change our application looks something like this:

static long g_Steps;

static int Fibonacci(int n)
{
  g_Steps++;

  if (n < 2)
    return n;
  else
    return Fibonacci(n - 1) + Fibonacci(n - 2);
}

static void Main(string[] args)
{
  for (int i = 0; i <= 45; i++)
  {
    g_Steps = 0;
    int fib = Fibonacci(i);

    Console.WriteLine("Fibonacci({0}) = {1:###,###,###,##0} [{2:###,###,###,##0} steps]", i, fib, g_Steps);
  }
}

Here are the results starting at the 25th iteration:

The number of steps taken to calculate a Fibonacci number is staggering! But, if you look carefully, you'll notice a pattern. It turns out that the number of steps grows in a Fibonacci sequence. That is, the number of steps needed to calculate a Fibonacci number is the sum of the number of steps that it took to calculate the previous two Fibonacci numbers. This same pattern also appears in our performance test measuring time (though not as precisely). If you're familiar with Big O Notation, the performance can be represented as O(fibonacci(n)). If that looks like Greek to you, just know that the performance gets much worse as the value of n increases.

The Solution

If you're still with me (and still remember the title of this article), you probably already know what the solution is: caching. Instead of calculating a Fibonacci number every time it's needed, we should store the result of the calculation so that it can be retrieved later. This can be done easily by employing a System.Collections.Generic.Dictionary:

static Dictionary<int, int> g_Results = new Dictionary<int, int>();

static int Fibonacci(int n)
{
  if (n < 2)
    return n;
  else
  {
    if (g_Results.ContainsKey(n))
      return g_Results[n];

    int result = Fibonacci(n - 1) + Fibonacci(n - 2);
    g_Results.Add(n, result);
    return result;
  }
}

If you run the performance tests on this, you will get some amazing results:

1. The test that measures time no longer returns meaningful values because each calculation takes less than a millisecond.
2. The test that measures the number of necessary steps returns 3 for each calculation (after 0 and 1).

For all of my Big O buddies out there, our algorithm is now O(1). However, it comes at a cost:

1. Our algorithm now takes up memory that won't be freed until the AppDomain is unloaded. The memory usage is O(n).
2. Using a Dictionary, while simple, is probably not the best choice as it takes up more memory than the number of items it contains.
3. We've added data that might potentially be shared if Fibonacci is called by multiple threads. That could cause unreliable results if the Dictionary gets corrupted by multi-threaded access.

There are several possible solutions to these issues but the solution that I propose is to use a closure.

Closures

A closure is a function that is bound to the environment in which it is declared. If that doesn't make sense, read on. In C# 2.0, we can implement closures using anonymous methods. Consider this code carefully:

delegate void Action();

static void ClosureExample()
{
  int x = 0;
  Action a = delegate { Console.WriteLine(x); };

  x = 1;
  a();
}

In this code, "a" is a delegate of type Action that is assigned to an anonymous method which simply prints the value of the local variable "x" to the console. The interesting bit is that "x" is actually declared outside of the anonymous method. To make this possible, "a" represents a closure that is bound to the local variable "x" because "x" is part of the environment (the parenting method body) in which "a" is declared.

Please note that I did not say that "a" is "bound to the value of 'x'". I said that "a" is "bound to the variable of 'x'". If "a" were bound to the value of "x", this code would print 0 to the console because that's the value assigned to "x" when "a" is declared. However, because it is bound to the variable "x", 1 will be output to the console because "x" is reassigned before "a" is called. This binding is persisted even if "x" goes out of scope:

delegate void Action();

static Action GetAction()
{
  int x = 0;
  Action a = delegate { Console.WriteLine(x); };

  x = 1;

  return a;
}

static void CallAction()
{
  Action a = GetAction();
  a();
}

The above code prints 1 to the console instead of zero even though "x" is out of scope when "a" is called.

I intend to write a future article about the C# compiler magic that makes closures possible but, for the moment, we will remain blissfully ignorant of the internals. In addition, if this information seems trivial or unimportant, please realize that closures are very important to the functional programming constructs coming in C# 3.0. In fact, many practices of functional programming (e.g. currying) are made possible by closures.

Closure on Fibonacci Numbers

Getting back to our Fibonacci function, we can use a closure to solve the three problems that I mentioned earlier. Here is a working implementation:

delegate int FibonacciCalculator(int n);

int Fibonacci(int n)
{
  int[] results = new int[n + 1];

  FibonacciCalculator calculator = null;
  calculator = delegate(int x)
  {
    if (x < 2)
      return x;
    else
    {
      if (results[x] == 0)
        results[x] = calculator(x - 1) + calculator(x - 2);

      return results[x];
    }
  };

  return calculator(n);
}

Make sure that you read this method carefully to take everything in:

1. An array is used instead of a Dictionary to store calculated results. This results in a smaller memory footprint and gains a little speed.
2. The "calculator" delegate is declared and set to null before assigning it to an anonymous method. This is necessary because the anonymous method calls itself recursively. The C# compiler requires "calculator" to be definitely assigned before it can be called inside the anonymous method body. Setting "calculator" to null is a little bit of a hack to get around this limitation. Eric Lippert has excellent blog post about this compiler behavior here.
3. I had to declare a custom delegate type ("FibonacciCalculator") to make this work. In C# 3.0, this isn't necessary because there are several generic Func<> delegates available in the base class library. I would use Func<int, int> in C# 3.0 instead of declaring my own delegate.
4. I was careful to only refer to "x" inside of the anonymous method. If I had accidentally referred to "n" at any point, the algorithm would have been broken.
5. This method is now thread-safe! All state is contained in local variables so it can safely be called by multiple threads.

There are a couple of other minor performance optimizations that can be done to speed things up even further but I left them out for clarity. Here is my final method with optimizations added in:

delegate int FibonacciCalculator(int n);

int Fibonacci(int n)
{
  if (n == 0)
    return 0;
  if (n < 3)
    return 1;

  // The array can be of size n - 2 because we don't need slots for the n < 3 case.
  int[] results = new int[n - 2];

  FibonacciCalculator calculator = null;
  calculator = delegate(int x)
  {
    if (x == 0)
      return 0;
    else if (x < 3)
      return 1;
    else
    {
      int index = x - 3;
      int result = results[index];
      if (result == 0)
      {
        result = calculator(x - 1) + calculator(x - 2);
        results[index] = result;
      }

      return result;
    }
  };

  return calculator(n);
}

 

from:http://diditwith.net/2007/02/08/FibonacciNumbersCachingAndClosures.aspx