Prompt engineering is a technique used in natural language processing (NLP) to improve the performance of large language models (LLMs) by providing them with more context and information about the task in hand. It involves creating prompts—short pieces of text—to provide additional information or guidance to the model to produce more accurate and relevant results.
Prompt engineering takes your Llama applications beyond basic interactions to create sophisticated, production-ready systems. While you can improve model performance through fine-tuning, distillation, or upgrading to larger or newer models, optimizing your prompts often provides the fastest path to better results—achieving the performance improvements you need without additional model training or infrastructure costs.
Creating effective prompts is an important part of prompt engineering. Here are some tips for creating prompts that will help improve the performance of your language model:
Detailed, explicit instructions produce better results than open-ended prompts. Giving explicit instructions is like placing rules and restrictions on how the model responds to your prompt.
You can steer the model towards responding in a specific writing style:
You can request specific formats using prompts:
Restrictions tell the model what not to do:
The example below illustrates how explicit instructions give more specific results by limiting the responses to recently created sources.
More likely to cite sources from 2017:
Gives more specific advances and only cites sources from 2020:
A shot is an example or demonstration of what type of prompt and response you expect from a large language model. This term originates from training computer vision models on photographs, where one shot was one example or instance that the model used to classify an image.
Modern LLMs like Llama are capable of following instructions and producing responses without having previously seen an example of a task. Prompting without examples is called "zero-shot prompting".
In the following example, the model is able to analyse the sentiment of simple text without needing examples of previous analyses:
Adding specific examples of your desired output generally results in a more accurate, consistent output when compared with zero-shot prompting. This technique is called "few-shot prompting".
In this example, the generated response follows our desired format that offers a more nuanced sentiment classifier that gives a positive, neutral, and negative response confidence percentage.
Creating prompts based on the role or perspective of the person or entity being addressed can be useful for generating more relevant and engaging responses from the model.
Pros:
Cons:
Chain-of-thought prompting provides the language model with a series of prompts or questions to help guide its thinking and generate a more coherent and relevant response. This technique can be useful for generating more thoughtful and well-reasoned responses from language models.
Pros:
Cons:
LLMs are probabilistic; even with chain-of-thought prompting, a single generation might produce incorrect results. Self-consistency introduces enhanced accuracy by selecting the most frequent answer from multiple generations, at the cost of higher compute.
Running the above several times and taking the most commonly returned value for the answer would make use of the self-consistency approach.
Modern models are generally able to produce common facts out-of-the-box, using just the model weights.
But unless a model was trained or fine-tuned on domain-specific or time-specific data, it is unlikely to produce accurate facts outside of its training knowledge.
Retrieval-augmented generation (RAG) describes the practice of including information in the prompt that has been retrieved from an external database. It's an effective way to incorporate facts into your LLM application and is more affordable than fine-tuning, which might also negatively impact the foundational model's capabilities.
The information source in a RAG system could be as simple as a lookup table or as sophisticated as a vector database containing all of your company's knowledge:
A common challenge in LLM applications is ensuring a model generates a suitable response without extraneous tokens (e.g. "Sure! Here's more information on…").
By combining a role, rules and restrictions, explicit instructions, and an example, the model can be prompted to generate the desired response.
Prompt:
Response:
json
LLMs, by nature, aren't great at performing calculations. While LLMs are bad at arithmetic, they're great for code generation. Program-Aided Language makes use of a model's code-generation skills by instructing the model to write code to solve calculation tasks.
Prompt:
Response:
python
Executing the returned code provides the correct result.
Even modern LLMs can produce hallucinations—confidently stated information that isn't supported by the source material. Meta’s Responsible Use Guide is a great resource to understand how best to prompt and address input/output risks of the language model. Refer to pages (14-17).
A well-crafted prompt can help to reduce hallucination in language models, by providing them with clear and accurate information and context.
Below are some examples of how a language model might hallucinate and some strategies for fixing the issue:
A language model is asked to generate a response to a question about a topic it has not been trained on. The language model may hallucinate information or make up facts that are not accurate or supported by evidence.
A language model is asked to generate a response to a question that requires a specific perspective or point of view. The language model may hallucinate information or make up facts that are not consistent with the desired perspective or point of view.
A language model is asked to generate a response to a question that requires a specific tone or style. The language model may hallucinate information or make up facts that are not consistent with the desired tone or style.