Reasoning

Tree-of-Thoughts (ToT)

Tree-of-Thoughts (ToT) is a problem-solving framework in which a language model generates several candidate next steps, called thoughts, evaluates how promising each one is, and explores the best branches while backtracking from dead ends instead of committing to a single line of reasoning. It suits puzzles and planning problems where one early wrong choice ruins everything that follows.

Best for
Puzzles, planning and search problems with dead ends
Difficulty
Advanced
Updated

What is Tree-of-Thoughts (ToT)?

Chain-of-thought is a single path: if step two is a bad choice, steps three to ten are wasted. ToT treats reasoning as a search. At each step the model proposes alternatives, an evaluator (often the same model with a different prompt) rates them, and a search strategy — breadth-first or depth-first — decides which to expand.

Done properly, ToT is a small program around the model, not a single prompt. The single-prompt version below is a useful approximation for everyday problems: it forces the model to consider alternatives and prune explicitly, but it cannot truly backtrack its own hidden state the way a coded search can.

When to use it — and when not to

Use it when

  • Puzzles and constraint problems: scheduling, seating plans, word and number games.
  • Planning where early choices constrain later ones: an itinerary, a project plan, an architecture decision.
  • Creative tasks that benefit from comparing several plans before writing, such as story structure.
  • Any problem where intermediate states can be judged as promising or hopeless.

Skip it when

  • Straightforward questions — the cost multiplies with every branch and level.
  • Tasks with no way to evaluate partial progress; the search has nothing to prune on.
  • Many reasoning-model use cases: models that already explore internally often match a hand-built tree at far lower effort. Try them first.

Copyable template

Single-prompt Tree-of-Thoughts 
Problem: [PROBLEM]
Constraints: [LIST EVERY CONSTRAINT]

Do not commit to your first approach. Work as a search:
1. Propose [3] genuinely different first steps.
2. Rate each one: SURE / LIKELY / IMPOSSIBLE to lead to a valid solution, with one line of reasoning.
3. Drop anything IMPOSSIBLE. Expand the best remaining branch with [3] next steps and rate those the same way.
4. If a branch reaches a dead end, write BACKTRACK, return to the best unexplored branch and continue.
5. Stop when a branch reaches a complete solution, then check it against every constraint.

Output: a brief outline of the branches you explored, then the final solution and the constraint check.

For a coded version: one prompt proposes thoughts, a second rates states (sure / likely / impossible works well), and your code runs breadth-first search keeping the best few states per level.

Replace everything in [SQUARE BRACKETS] before you run it.

Worked example: The Game of 24

Use each of the numbers 4, 9, 10 and 13 exactly once with + − × ÷ to make 24 — the benchmark task from the original paper.

Prompt 
Problem: use 4, 9, 10, 13 exactly once each, with + - * /, to make 24.
(Run the single-prompt Tree-of-Thoughts above.)
Illustrative outputWritten to show the pattern, not a recorded model run
Branch A: 13 - 9 = 4  → left 4, 4, 10   LIKELY
Branch B: 10 - 4 = 6  → left 6, 9, 13   LIKELY
Branch C: 13 + 10 = 23 → left 4, 9, 23  IMPOSSIBLE (no way to reach 24 from 23 with 4 and 9)
Expand A: 4 + 4 = 8 → 8, 10   IMPOSSIBLE · 4 * 4 = 16 → 16, 10   IMPOSSIBLE · 10 - 4 = 6 → 4, 6   SURE (4 * 6 = 24)
Solution: (13 - 9) * (10 - 4) = 4 * 6 = 24
Check: 4, 9, 10, 13 each used once ✓  result 24 ✓

Why it works: A single chain that starts with 13 + 10 is stuck at once. Rating partial states lets the search abandon that branch and spend effort where a solution is possible.

Common mistakes

  1. Using it on easy problems. Branching multiplies tokens. If chain-of-thought already works, stop there.
  2. A weak evaluator. The search is only as good as the scoring. Make the rating criteria concrete and tied to the constraints.
  3. Branches that are the same idea reworded. Three phrasings of one approach is not a tree. Demand genuinely different first steps.
  4. Believing the single prompt is the full method. It is an approximation. For hard, repeated problems, implement the search in code.
  5. Unbounded depth or breadth. Set how many branches and levels to explore, or the cost explodes.

Origin & sources

Introduced by Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas Griffiths, Yuan Cao and Karthik Narasimhan (Princeton and Google DeepMind) in 2023 and published at NeurIPS 2023. On the Game of 24, GPT-4 with chain-of-thought prompting solved 4% of problems, while Tree-of-Thoughts solved 74%. They also tested creative writing and mini crosswords.

Try it on The Prompt Index

Decision Lab Compare options against evidence when the “tree” is a real decision. AI Labs Test the same search prompt across different models.

Frequently asked questions

What is the difference between chain-of-thought and tree-of-thoughts?

Chain-of-thought follows one line of reasoning from start to finish. Tree-of-thoughts generates several possible next steps at each point, evaluates them and explores the most promising, backtracking from dead ends.

Can tree-of-thoughts be done in a single prompt?

An approximation can: ask the model to propose alternatives, rate them and prune explicitly. The full method uses code to run a real search with separate generation and evaluation calls.

Do reasoning models make tree-of-thoughts obsolete?

For many problems they reduce the need for it, because they already explore and revise internally. Explicit tree search is still useful when you want control over branching, evaluation criteria and cost.