Graph of Thoughts: Enhancing Large Language Models with Flexible Data Handling

Artificial Intelligence (AI) has witnessed an increase in the utilization of Large Language Models (LLMs) based on the Transformer architecture’s decoder-only design. The popularity of models like GPT, PaLM, and LLaMA has grown significantly. Prompt engineering is a successful and efficient technique that leverages LLMs to address various problems by incorporating task-specific instructions in the input text. By properly crafting these instructions, the LLM can generate relevant text and complete tasks effectively using its autoregressive token-based approach.

The Chain-of-Thought (CoT) method builds upon prompt engineering by providing intermediate steps or thoughts in addition to the task description in the input prompt. This addition considerably enhances the LLM’s problem-solving capabilities without requiring model updates. To compare the capabilities of LLMs with existing paradigms like Chain-of-Thought and Tree of Thoughts (ToT), a recent framework called Graph of Thoughts (GoT) has been introduced.

GoT represents data as an arbitrary graph, enabling LLMs to handle and generate data in a more flexible manner. Each piece of information, or LLM thought, is represented as a vertex in the graph, while the connections and dependencies between them are represented as edges. This allows for the combination of different LLM thoughts to produce more powerful and effective results. By allowing these thoughts to be interconnected inside the graph, complex thought networks can be captured, in contrast to linear paradigms that limit thinking. This opens up possibilities for combining diverse ideas into a cohesive answer, simplifying intricate thought networks, and enhancing ideas through feedback loops.

GoT’s effectiveness is demonstrated by its superior performance compared to existing methods across multiple tasks. In a sorting test, GoT improves sorting quality by 62% while reducing computing expenses by over 31%. This showcases GoT’s ability to balance task accuracy with resource efficiency. Another notable advantage of GoT is its extensibility. The framework can support creative prompting schemes and easily adapt to new idea transformations. This agility is crucial in a rapidly evolving landscape of LLM research and applications.

By establishing the GoT framework, this work significantly advances the alignment of LLM reasoning with human thinking processes and brain systems. Both human and brain thought processes involve complex networks where thoughts interact, branch out, and influence each other. GoT bridges the gap between traditional linear techniques and these sophisticated, network-like mental processes, thereby improving LLMs’ skills and their ability to tackle challenging problems.

Check out the Paper and Github. All credit for this research goes to the researchers on this project. Also, don’t forget to join our community on ML SubReddit, Facebook, Discord, and subscribe to our Email Newsletter for the latest updates on AI research, projects, and more.