5. Prompt Engineering

Basic prompt: "What's 123 × 456?"
CoT prompt: "Let's solve 123 × 456 step by step:
1. First, let's break down 456: 400 + 50 + 6
2. Now multiply 123 by each part:
   - 123 × 400 = 49,200
   - 123 × 50 = 6,150
   - 123 × 6 = 738
3. Finally, add these results:
   49,200 + 6,150 + 738 = 56,088"

Question: "What color is the sky?"
Top-P = 0.5: "blue" (sticks to most common answer)
Top-P = 0.9: "azure", "cerulean", "sapphire" (considers more options)

## A financial analyst needs to extract specific data points from quarterly reports. Which prompt structure would be most effective? > Hint: Consider which prompt elements would help the model understand the exact format needed. 1. [ ] A simple instruction: "Extract financial data from this quarterly report." > This is too vague. Without specifying which data points to extract or how to format the output, the model would likely provide inconsistent or incomplete results. 1. [ ] A detailed explanation of why the data is needed > While context can be helpful, simply explaining the purpose without specifying the format or examples wouldn't effectively guide the model to extract the specific data points needed. 1. [x] Clear instructions with format specifications and examples > Correct! An effective prompt would include: 1) Specific instructions about which data points to extract, 2) The exact format for the output (e.g., JSON with specific fields), and 3) One or two examples showing the expected input-output pattern. This structure guides the model to consistently extract the required information in the desired format. 1. [ ] Chain-of-thought reasoning that walks through the report section by section > While chain-of-thought can be useful for complex reasoning, for data extraction tasks, clear formatting instructions and examples are more important than walking through the reasoning process. ## You're building a customer service chatbot that needs to remember details from earlier in the conversation. What's the most effective approach to manage this? > Hint: Consider the inherent limitations of LLMs regarding memory. 1. [ ] Instruct the model to "remember everything the customer says" > This won't work because LLMs don't have persistent memory between requests. Simply instructing the model to remember doesn't create an actual memory mechanism. 1. [ ] Increase the temperature parameter to improve creativity > Temperature affects randomness/creativity, not memory capabilities. Changing this parameter won't help the model remember previous interactions. 1. [x] Implement context management by storing and selectively including relevant conversation history > Correct! Since LLMs are stateless, you need to implement external memory by: 1) Storing previous exchanges, 2) Selectively including relevant parts of the conversation history in each new prompt, and 3) Managing token usage to prevent exceeding context limits. This might include summarizing older parts of the conversation to save tokens while preserving key information. 1. [ ] Switch from a standard model to a reasoning model > The choice between model types affects how responses are processed and constructed, and may affect the quality of the answer, but doesn't impact the model's ability to retain information between requests. ## A software developer is using an LLM to help debug complex code. Which approach would likely yield the most accurate assistance? > Hint: Think about how different LLMs process reasoning tasks. 1. [ ] Using high temperature settings (0.8-1.0) to get creative solutions > Higher temperatures increase randomness, which is generally counterproductive for debugging tasks where precision and accuracy are critical. This would likely introduce more errors. 1. [x] Using a modern reasoning model with a direct, concise prompt > Correct! For complex debugging tasks, modern reasoning models (like o3 or R1) with built-in verification capabilities would perform best when given concise prompts. These models automatically employ multi-step reasoning without needing explicit prompting instructions, making them ideal for code analysis where accuracy is crucial. 1. [ ] Providing detailed step-by-step instructions on how to analyze the code > For modern reasoning models, explicit step-by-step instructions can actually interfere with their built-in reasoning capabilities. Overly detailed prompting can constrain the model's approach and reduce effectiveness. 1. [ ] Using a Chat API with multiple detailed few-shot examples > While few-shot examples can help traditional LLMs, they can actually reduce the quality of responses from reasoning models on technical tasks like debugging. These advanced models perform better with direct questions that allow them to employ their internal reasoning processes. ## For an AI application that generates product descriptions for an e-commerce site, which parameter configuration would be most appropriate? > Hint: Consider the balance between consistency and creativity needed for this specific task. 1. [ ] Temperature: 0.1, Top-P: 0.5 > This combination creates highly deterministic, conservative outputs. While consistency is important for product descriptions, this setting might produce overly generic or repetitive content that doesn't effectively highlight product features. 1. [ ] Temperature: 1.0, Top-P: 1.0 > These settings maximize randomness and diversity, which could result in unpredictable and potentially inaccurate product descriptions. This level of creativity is inappropriate for factual content like product information. 1. [x] Temperature: 0.7, Top-P: 0.9 > Correct! This balanced configuration provides enough creativity to make engaging, varied product descriptions while maintaining sufficient consistency to ensure accuracy. It allows for some stylistic flair (beneficial for marketing) without straying into fabrication or extreme variability. 1. [ ] Temperature: 0, No Top-P filtering > Zero temperature produces completely deterministic results, essentially taking the most probable token at each step. This would create extremely rigid, mechanical descriptions lacking any engaging qualities needed for marketing.