📄 Recorded Lecture: Systematic Error Analysis
Resources
Notes
Channels
-
[
announcements
](/parlance-labs/evals/2025-3/channel?channelId=283916d6-d0ce-4b62-9af9-b308a8eceecf)
- [general
](/parlance-labs/evals/2025-3/channel?channelId=6aa0dbf0-46ed-47e7-9ae8-cbbb4db531c9)
- [intros
](/parlance-labs/evals/2025-3/channel?channelId=c4b50f08-c5b5-4f3f-a9bd-c9684d5c7b3d)
- [ðŸ’
questions
](/parlance-labs/evals/2025-3/channel?channelId=0458bb33-ed98-42cf-b593-4161ac4cd2cd)
Messages
Channels
-
[
announcements
](/parlance-labs/evals/2025-3/channel?channelId=283916d6-d0ce-4b62-9af9-b308a8eceecf)
- [general
](/parlance-labs/evals/2025-3/channel?channelId=6aa0dbf0-46ed-47e7-9ae8-cbbb4db531c9)
- [intros
](/parlance-labs/evals/2025-3/channel?channelId=c4b50f08-c5b5-4f3f-a9bd-c9684d5c7b3d)
- [ðŸ’
questions
](/parlance-labs/evals/2025-3/channel?channelId=0458bb33-ed98-42cf-b593-4161ac4cd2cd)
Messages
Lesson 2 & 3: Systematic Error Analysis
📄
Recorded Lecture: Systematic Error Analysis
Lesson 1
Topics covered:
- Bootstrap data through effective synthetic data generation
- Annotation strategies and quantitative analysis of qualitative data
- Avoiding common pitfalls in the error analysis process
Resources:
-
Chapter 3 in course reader
-
Slides (note that we jump around a bit in recorded lectures through the slides, and may not feature some of them). Links to Lesson 2 and Lesson 3 slides.
-
Homework: homeworks/hw2 in https://github.com/ai-evals-course/recipe-chatbot
CompleteComplete this lesson
[
Next
](/parlance-labs/evals/2025-3/syllabus/modules/a211ae?item=59wvq8x06kr)
0%
-
[
📄
Lesson 1
Recorded Lecture: Systematic Error Analysis
](/parlance-labs/evals/2025-3/syllabus/modules/a211ae?item=gqm0sw87aar)
- [📄
Lesson 2
Chapter 1: Introduction
](/parlance-labs/evals/2025-3/syllabus/modules/a211ae?item=59wvq8x06kr)
- [📄
Lesson 3
Chapter 2: Create Synthetic Data
](/parlance-labs/evals/2025-3/syllabus/modules/a211ae?item=75s8vsu2pi)
- [📄
Lesson 4
Chapter 3: Open Coding
](/parlance-labs/evals/2025-3/syllabus/modules/a211ae?item=f2m0dtd0e7j)
- [📄
Lesson 5
Chapter 4: Axial Coding
](/parlance-labs/evals/2025-3/syllabus/modules/a211ae?item=pn30kwmjzu)
- [📄
Lesson 6
Break & Reflection Question
](/parlance-labs/evals/2025-3/syllabus/modules/a211ae?item=pa5d8sjpsu)
- [📄
Lesson 7
Chapter 5: Error Analysis Recap
](/parlance-labs/evals/2025-3/syllabus/modules/a211ae?item=ofpfoc5s269)
- [📄
Lesson 8
Chapter 6: Multi-Turn Evaluation
](/parlance-labs/evals/2025-3/syllabus/modules/a211ae?item=m682v7ht9qg)
- [📄
Lesson 9
Chapter 7: Collaborative Evaluation
](/parlance-labs/evals/2025-3/syllabus/modules/a211ae?item=2anrs96yhwd)
- [📄
Lesson 10
Chapter 8: Rubrics for Collaborative Evaluation
](/parlance-labs/evals/2025-3/syllabus/modules/a211ae?item=oz9fny9wxe)
- [📄
Lesson 11
Optional HW
](/parlance-labs/evals/2025-3/syllabus/modules/a211ae?item=t4wzr7hvck)
[
Home
](/parlance-labs/evals/2025-3/home)[
Community
](/parlance-labs/evals/2025-3)