Multi-Turn Chat Evals – Hamel’s Blog - Hamel Husain
Dec 5, 2024 · These resources can help you learn more about evaluating conversational AI: Your AI Product Needs Evals: A broader overview of evaluation approaches for AI products
Q: How do I debug multi-turn conversation traces? - hamel.dev
Jul 29, 2025 · Start simple. Check if the whole conversation met the user’s goal with a pass/fail judgment. Look at the entire trace and focus on the first upstream failure. Read the user-visible parts …
Your AI Product Needs Evals – Hamel's Blog - Hamel Husain
Mar 29, 2024 · Rechat’s AI assistant, Lucy, is a canonical AI product: a conversational interface that obviates the need to click, type, and navigate the software. During Lucy’s beginning stages, rapid …
claude – Hamel's Blog - Hamel Husain
Good (conversational): “The hard part is keeping everything accurate. That means either spending tokens on LLM calls, or getting domain experts to help.” Simplify jargon: Technical terms, especially …
The Chat Format – Hamel’s Blog
\ If it's a delivery, you ask for an address. \ Finally you collect the payment.\ Make sure to clarify all options, extras and sizes to uniquely \ identify the item from the menu.\ You respond in a short, very …
P6: Context Rot – Hamel’s Blog - Hamel Husain
Sep 10, 2025 · Models were tested under two conditions: a “focused” condition with only the relevant conversational history (around 100 tokens), and a “full” condition where the context was padded with …
Models were tested under two conditions: a “focused” condition with only the relevant conversational history (around 100 tokens), and a “full” condition where the context was padded with irrelevant …
P3: Optimizing Retrieval with Reasoning Models - hamel.dev
Jul 6, 2025 · (Timestamp: 00:02:17) This slide shows a modern “SearchGPT” style interface, which provides a generated, conversational answer. (Timestamp: 00:02:46) Despite the interface, Orion …