Notes

• Home
• Inbox
• People

Channels

• [

  announcements

  ](/parlance-labs/evals/2025-3/channel?channelId=283916d6-d0ce-4b62-9af9-b308a8eceecf)
  - [

  general

  ](/parlance-labs/evals/2025-3/channel?channelId=6aa0dbf0-46ed-47e7-9ae8-cbbb4db531c9)
  - [

  intros

  ](/parlance-labs/evals/2025-3/channel?channelId=c4b50f08-c5b5-4f3f-a9bd-c9684d5c7b3d)
  - [

  💭

  questions

  ](/parlance-labs/evals/2025-3/channel?channelId=0458bb33-ed98-42cf-b593-4161ac4cd2cd)

Messages

• Inbox
• People

Channels

• [

  announcements

  ](/parlance-labs/evals/2025-3/channel?channelId=283916d6-d0ce-4b62-9af9-b308a8eceecf)
  - [

  general

  ](/parlance-labs/evals/2025-3/channel?channelId=6aa0dbf0-46ed-47e7-9ae8-cbbb4db531c9)
  - [

  intros

  ](/parlance-labs/evals/2025-3/channel?channelId=c4b50f08-c5b5-4f3f-a9bd-c9684d5c7b3d)
  - [

  💭

  questions

  ](/parlance-labs/evals/2025-3/channel?channelId=0458bb33-ed98-42cf-b593-4161ac4cd2cd)

Messages

Back

Lesson 4&5: Automated Evaluators

📄

Chapter 7: LLM as Judge Coding Demo

Lesson 8

[

Previous

](/parlance-labs/evals/2025-3/syllabus/modules/b51ca3?item=j51tj9fyzsc)

CompleteComplete this lesson

[

Next

](/parlance-labs/evals/2025-3/syllabus/modules/b51ca3?item=wtd458l9bub)

0%

• [

  📄

  Lesson 1

  Recorded Lesson: Automated Evaluators

  ](/parlance-labs/evals/2025-3/syllabus/modules/b51ca3?item=oumjjqd57c)
  - [

  📄

  Lesson 2

  Chapter 1: Introduction

  ](/parlance-labs/evals/2025-3/syllabus/modules/b51ca3?item=ger3pdhumf)
  - [

  📄

  Lesson 3

  Chapter 2: Error Analysis Recap

  ](/parlance-labs/evals/2025-3/syllabus/modules/b51ca3?item=1kesnlso95e)
  - [

  📄

  Lesson 4

  Chapter 3: Code-based vs LLM-based Evaluators

  ](/parlance-labs/evals/2025-3/syllabus/modules/b51ca3?item=l0d8vp059j)
  - [

  📄

  Lesson 5

  Chapter 4: Overview of Creating a LLM-as-Judge Evaluator

  ](/parlance-labs/evals/2025-3/syllabus/modules/b51ca3?item=kwvk6ff9exn)
  - [

  📄

  Lesson 6

  Chapter 5: Example Criterion for LLM-as-a-Judge

  ](/parlance-labs/evals/2025-3/syllabus/modules/b51ca3?item=rpbapruasgn)
  - [

  📄

  Lesson 7

  Chapter 6: Crafting and Refining the LLM Judge Prompt

  ](/parlance-labs/evals/2025-3/syllabus/modules/b51ca3?item=j51tj9fyzsc)
  - [

  📄

  Lesson 8

  Chapter 7: LLM as Judge Coding Demo

  ](/parlance-labs/evals/2025-3/syllabus/modules/b51ca3?item=pt4ba03ydnb)
  - [

  📄

  Lesson 9

  Chapter 8: Correcting Bias in LLM-as-Judge Evaluators

  ](/parlance-labs/evals/2025-3/syllabus/modules/b51ca3?item=wtd458l9bub)
  - [

  📄

  Lesson 10

  Chapter 9: Pitfalls to Avoid When Building Automated Evaluators

  ](/parlance-labs/evals/2025-3/syllabus/modules/b51ca3?item=4tnvqupsv1g)
  - [

  📄

  Lesson 11

  Optional HW

  ](/parlance-labs/evals/2025-3/syllabus/modules/b51ca3?item=o8wmm10ftsg)

[

Home

](/parlance-labs/evals/2025-3/home)[

Community

](/parlance-labs/evals/2025-3)