LLM4CO 写作素材

When further constrained to operate on the crucial part of the algorithm with a program skeleton, the LLM provides suggestions that marginally improve over existing ones in the population, which ultimately results in discovering new knowledge on open problems when combined with the evolutionary algorithm. Another crucial component of the effectiveness of FunSearch is that it operates in the space of programs: rather than directly searching for constructions (which is typically an enormous list of numbers), FunSearch searches for programs generating those constructions. Because most problems we care about are structured(highly non-random), we believe that solutions are described more concisely with a computer program, compared to other representations. In a loose sense, FunSearch attempts to find solutions that have low Kolmogorov complexity48–50 (which is the length of the shortest computer program that produces a given object as output), whereas traditional search procedures have a very different inductive bias. We believe that such Kolmogorov-compressed inductive bias is key to FunSearch scaling up to the large instances in our use cases.

FunSearch 之所以能取得良好效果，另一个关键因素在于它在程序空间中进行搜索：它并非直接寻找构造结果（通常是一长串庞大的数值列表），而是搜索能够生成这些构造的程序。因为我们所关注的大多数问题都具有结构性（高度非随机），因此我们认为，与其他表示形式相比，计算机程序能更简洁地描述解。从广义而言，FunSearch 致力于寻找柯尔莫哥洛夫复杂度较低的解⁴⁸⁻⁵⁰（即能生成特定目标对象的最短计算机程序的长度）；而传统搜索过程则具有截然不同的归纳偏置。

Mathematical discoveries from program search with large language models