There are many strategies one can follow to design efficient prompts. The combination of a model and use cases requires different strategies to achieve the best results.
These are my notes on the most basic and used techniques to design prompts. Although there are many more, and new papers are published constantly.
Zero-shot Prompting
Zero-shot prompting means that the prompt used to interact with the model won't contain examples or demonstrations. Instead, it will directly instruct the model to perform a task.
Example
Prompt:
Classify the text into neutral, negative or positive.
Text: I think the vacation is okay.
Sentiment:Output:
NeutralNotes
Finetuning a model leads to better zero-shot performance.
https://arxiv.org/pdf/2109.01652
Few-shot Prompting
Few-shot prompting is similar to zero-shot prompting, but it includes a few examples to help the model understand the task better.
Additionally, few-shot can be used as a technique to enable in-context learning.
Example
Prompt:
A "whatpu" is a small, furry animal native to Tanzania. An example of a sentence that uses the word whatpu is:
We were traveling in Africa and we saw these very cute whatpus.
To do a "farduddle" means to jump up and down really fast. An example of a sentence that uses the word farduddle is:Output:
When we won the game, we all started to farduddle in celebration.Notes
When dealing with complex reasoning tasks, few-shot prompting falls short. In such cases, techniques like chain-of-thought prompting can be more effective.
Chain-of-Thought (CoT) Prompting
Introduced in this paper, chain-of-thought prompting is a technique that allows the model to reason over multiple steps.
This technique can be combined with few-shot prompting to achieve better results.
Example
Prompt:
The odd numbers in this group add up to an even number: 4, 8, 9, 15, 12, 2, 1.
A: Adding all the odd numbers (9, 15, 1) gives 25. The answer is False.
The odd numbers in this group add up to an even number: 17, 10, 19, 4, 8, 12, 24.
A: Adding all the odd numbers (17, 19) gives 36. The answer is True.
The odd numbers in this group add up to an even number: 16, 11, 14, 4, 8, 13, 24.
A: Adding all the odd numbers (11, 13) gives 24. The answer is True.
The odd numbers in this group add up to an even number: 17, 9, 10, 12, 13, 4, 2.
A: Adding all the odd numbers (17, 9, 13) gives 39. The answer is False.
The odd numbers in this group add up to an even number: 15, 32, 5, 13, 82, 7, 1.
A:Output:
Adding all the odd numbers (15, 5, 13, 7, 1) gives 41. The answer is False.Notes
- CoT prompting can be used with zero-shot by instructing the model to reason over multiple steps.
- Automatic Chain-of-Thought prompting is an area of active research.
Self-Consistency Prompting
One of the most advanced techniques today, self-consistency prompting is a technique iams to replace the naive greedy decoding used in CoT prompting with a more sophisticated approach. The idea is to samples multiple, diverse reasoning paths through few-shot CoT prompting and uses the generations to select the most consistent answer. This technique can be used to improve airthmetic and commonses reasoning tasks.
Notes
The paper on self-consistency prompting is a must-read. https://arxiv.org/abs/2203.11171
Prompt Chaining
Prompt chaining is a technique that involves using the output of one prompt as the input to the next prompt.
A very useful technique for multiple-step tasks, complex instructions, verifying outputs, and parallel processing.
Notes
The Claude documentation has a great description about prompt chaining. https://docs.anthropic.com/en/docs/chain-prompts
Retrival Augmented Generation (RAG)
RAG takes an input prompt and retrieves relevant information from a knowledge base to generate the output. This technique is particularly useful for tasks that require external knowledge without the need for finetuning.