Free online consultation available. Please feel free to contact us.

Chain-of-Thought Prompting

  • Glossary

"Chain-of-Thought Prompting"

This glossary explains various keywords that will help you understand the mindset necessary for data utilization and successful DX.
This time, we will introduce "Chain-of-Thought Prompting," a technique for utilizing conversational AI using large-scale language models such as ChatGPT.

What is Chain-of-Thought Prompting?

Chain-of-Thought prompting is a prompt creation technique that, when using conversational AI such as ChatGPT, allows you to obtain a more desired output by providing a thought process as a guide when giving input (prompts).
Chain-of-Thought is sometimes abbreviated as CoT. Also, output can sometimes be improved by letting ChatGPT think for itself without providing a thought process as input, and this technique is sometimes called Chain-of-Thought.

What kind of technique is it? (Example of use)

ChatGPT can do a lot of different things, for example it can do some (small) calculations.

You have five apples. If you eat two apples, how many are left?

You start with five apples, so after eating two you have three apples left.

It works as if it understands what is being said, calculates it, and returns the answer. It's a frightening behavior, but when it comes to difficult problems, ChatGPT sometimes makes calculation errors.

From the Google paper that inspired the name:

Chain-of-Thought prompting is a technique originally proposed in a Google paper.

⇒ [2201.11903] Chain-of-Thought Prompting Elicits Reasoning in Large Language Models 

Here is an example of a calculation failure from the paper. The performance of conversational AI is improving every day, so at this point it may often give correct answers.

The cafeteria had 23 apples. If they used 20 to make lunch and bought 6 more, how many apples do they have?
There are 23 apples in the cafeteria. If you use 20 to make lunch and then buy 6 more, how many apples will you have left?

The answer is 27.
(The answer is 27)

It may seem odd that a computer would make a calculation mistake, but ChatGPT is built on a large-scale language model called GPT, which has not been trained for the task of computation.

​⇒Large Language Model (LLM) | Glossary

Even though it has not been trained, "when I made it, for some reason it was able to do calculations," but I don't know why it can do calculations, and it may or may not be able to do calculations depending on the problem it is given. In this way, ChatGPT can be used or not used for various tasks, not just calculations.

With assistance it becomes solvable

Researchers at Google have discovered and reported in a paper that problems that cannot be solved like this can sometimes be solved by providing some assistance with prompts (input).

Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 tennis balls. How many tennis balls does he have now?
Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can contains 3 tennis balls. How many tennis balls does he now have?

A: Roger started with 5 balls. 2 cans of 3 tennis balls each is 6 tennis balls. 5 + 6 = 11. The answer is 11.
Roger started with 5 tennis balls. He bought two cans of 3 tennis balls each, so he got 6 extra tennis balls. 5 + 6 = 11. The answer is 11.

Q: The cafeteria had 23 apples. If they used 20 to make lunch and bought 6 more, how many apples do they have?
There were 23 apples in the cafeteria. They used 20 for lunch and then bought 6. How many apples do they have now?

A: The cafeteria had 23 apples originally. They used 20 to make lunch. So they had 23 - 20 = 3. They bought 6 more apples, so they have 3 + 6 = 9. The answer is 9.
(Initially, there were 23 apples in the cafeteria. They used 20 for lunch, leaving 23 - 20 = 3 apples. Then they buy 6 more apples, so now they have 3 + 6 = 9 apples. The answer is 9.)

First, the answer is correct (the answer is 9), and second, there's a lot more typing.

Chain-of-Thought prompting input explanation

Before asking a question, a complete question and answer set is added as an example. This uses Few-Shot Prompting, a technique that teaches by showing examples of the answers you want to see (please see the separate article).

⇒ Few-Shot Prompting | Glossary

So, what is the example taught by Few-Shot Prompting? The example is also a calculation problem, one that ChatGPT can solve on its own. However, when asked to answer it itself, it will only answer "The answer is 11." The following answer is given as an example, with the underlined part added by a human:

A: Roger started with 5 balls. 2 cans of 3 tennis balls each is 6 tennis balls. 5 + 6 = 11. The answer is 11.
Roger started with 5 tennis balls. He bought two cans of 3 tennis balls each, so he got 6 extra tennis balls. 5 + 6 = 11. The answer is 11.​

ChatGPT will use this as an example answer and will respond to other questions in a similar format.

A: The cafeteria had 23 apples originally. They used 20 to make lunch. So they had 23 - 20 = 3. They bought 6 more apples, so they have 3 + 6 = 9. The answer is 9.
(Initially, there were 23 applesin the cafeteria. They used 20 for lunch, leaving 23 - 20 = 3apples. Then theybuy 6 more apples, so now they have 3 + 6 = 9 apples. The answer is 9.)

Without any assistance, he would simply answer "The answer is 27," which was incorrect, but when he began to organize the situation, show the corresponding formula, and indicate the calculation result, he began to give the correct answer.

What does Chain-of-Thought prompting do?

This means that before your answer, you provide a written transcript of the thought process that led you to the correct answer. ChatGPT will then use this as an example and try to provide an answer in a similar format.

As a result, it will not generate an answer right away. This is because the example instructs it to produce a different output (thought process) beforehand. Therefore, it first generates output in the instructed thought process format. In this example, it organizes the given conditions in order, presents the corresponding calculation formula, and then generates the answer. In other words, it can be said to be a "technique to control ChatGPT 's thought process."

Even if the answer is wrong, doing this can help you understand why it was wrong. Furthermore, for tasks like calculations where there is no correct answer, such as writing a persuasive explanation, it is possible to specify the logical structure you want the answer to be.

Chain-of-Thought (Zero-shot CoT) by adding just one phrase

This technique may seem effective, but it may seem like a bit of a time-consuming one. However, there is a very simple way to improve the accuracy of answers by having ChatGPT consider the thought process. Here is an example of using the technique from the Google paper mentioned earlier, which uses "mistake examples."

The cafeteria had 23 apples. If they used 20 to make lunch and bought 6 more, how many apples do they have?
There were 23 apples in the cafeteria. If they used 20 for lunch and then bought 6 more apples, how many apples do they now have?
Let's think step by step.
Let's think about it step by step.

Sure, let's break down the problem step by step.
Ok, let's break the problem down piece by piece.

  1. The cafeteria had 23 apples
  2. They used 20 apples to make lunch, so they have 23 - 20 = 3 apples left.
  3. They bought 6 more apples.
  4. Therefore, the total number of apples they have now is 3 + 6 = 9 apples.

So, the cafeteria has 9 apples now.

Here too, ChatGPT correctly answers that there are 9 apples. Moreover, even though we did not specifically instruct it to think in stages, it was able to think in stages on its own.

The only change I made to the input was to add one more phrase.

Let's think step by step.
Let's think about it step by step.

By simply adding this one phrase, ChatGPT was able to break down the problem into stages and think about it in order to answer the question. It is known that if you specify various other thinking styles in addition to this phrase, the output will change accordingly.

Because this technique achieves the effect aimed at by Chain-of-Thought even without a human providing an example, it is sometimes called "Zero-shot CoT (Zero-shot Chain-of-Though)." Alternatively, I think this technique is sometimes called Chain-of-Thought prompting.

It's very easy to use, just add one specific phrase to the end of the input, and yet it has the potential to significantly improve the output, so I think it's a technique worth remembering.

Keywords related to Generative AI/ChatGPT (for further understanding)

Glossary Column List

Alphanumeric characters and symbols

A row

Ka row

Sa row

Ta row

Na row

Ha row

Ma row

Ya row

Ra row

Wa row

»Data Utilization Column List

Recommended Content

Related Content

Return to column list