📄 Optional HW
Resources
Notes
Channels
-
[
announcements
](/parlance-labs/evals/2025-3/channel?channelId=283916d6-d0ce-4b62-9af9-b308a8eceecf)
- [general
](/parlance-labs/evals/2025-3/channel?channelId=6aa0dbf0-46ed-47e7-9ae8-cbbb4db531c9)
- [intros
](/parlance-labs/evals/2025-3/channel?channelId=c4b50f08-c5b5-4f3f-a9bd-c9684d5c7b3d)
- [💭
questions
](/parlance-labs/evals/2025-3/channel?channelId=0458bb33-ed98-42cf-b593-4161ac4cd2cd)
Messages
Channels
-
[
announcements
](/parlance-labs/evals/2025-3/channel?channelId=283916d6-d0ce-4b62-9af9-b308a8eceecf)
- [general
](/parlance-labs/evals/2025-3/channel?channelId=6aa0dbf0-46ed-47e7-9ae8-cbbb4db531c9)
- [intros
](/parlance-labs/evals/2025-3/channel?channelId=c4b50f08-c5b5-4f3f-a9bd-c9684d5c7b3d)
- [💭
questions
](/parlance-labs/evals/2025-3/channel?channelId=0458bb33-ed98-42cf-b593-4161ac4cd2cd)
Messages
Lesson 6: Complex Architectures
📄
Optional HW
Lesson 9
This week’s homework connects directly to Chapters 7–8 on Complex Architectures. You’ll complete two assignments that extend the evaluation methods we discussed.
In Homework 4: Recipe Bot Retrieval Evaluation, you’ll build and evaluate a Retrieval-Augmented Generation (RAG) pipeline using BM25 on recipe data. The goal is to measure how well your retriever surfaces relevant recipes for complex, real-world cooking questions—and optionally improve performance using an LLM-based query rewrite agent.
Then, in Homework 5: Failure Transition Heat-Map, you’ll analyze pre-labeled agent traces, constructing and visualizing a transition matrix that reveals where multi-step pipelines most often break down.
Remember, the homeworks are totally optional!
[
Previous
](/parlance-labs/evals/2025-3/syllabus/modules/9af422?item=tv6m93zztje)
CompleteComplete this lesson
0%
-
[
📄
Lesson 1
Complex Architectures (e.g., RAG)
](/parlance-labs/evals/2025-3/syllabus/modules/9af422?item=39ln5lt8o2x)
- [📄
Lesson 2
Chapter 1: Retrieval-Augmented Generation
](/parlance-labs/evals/2025-3/syllabus/modules/9af422?item=p92fmss4j5o)
- [📄
Lesson 3
Chapter 2: Evaluating Retrieval Quality
](/parlance-labs/evals/2025-3/syllabus/modules/9af422?item=36sq1vv0r75)
- [📄
Lesson 4
Chapter 3: Synthetic Data for RAG
](/parlance-labs/evals/2025-3/syllabus/modules/9af422?item=azvhhh8uibd)
- [📄
Lesson 5
Chapter 4: Retrieval Metrics
](/parlance-labs/evals/2025-3/syllabus/modules/9af422?item=0l7g1v9swxsq)
- [📄
Lesson 6
Chapter 5: Evaluating Generation Quality
](/parlance-labs/evals/2025-3/syllabus/modules/9af422?item=zouw53j69jf)
- [📄
Lesson 7
Chapter 6: Common Pitfalls With RAG
](/parlance-labs/evals/2025-3/syllabus/modules/9af422?item=sv12jvi1a2i)
- [📄
Lesson 8
Chapter 7: Complex Agents Beyond RAG
](/parlance-labs/evals/2025-3/syllabus/modules/9af422?item=tv6m93zztje)
- [📄
Lesson 9
Optional HW
](/parlance-labs/evals/2025-3/syllabus/modules/9af422?item=a80zgdqrbcc)
[
Home
](/parlance-labs/evals/2025-3/home)[
Community
](/parlance-labs/evals/2025-3)