Week 1

📄 Orientation: Required Reading, Discord & Course Materials

Resources

Notes

Messages

Messages

Back

Start Here

📄

Orientation: Required Reading, Discord & Course Materials

Lesson 1 • COMPLETED TUE. OCT 7

  1. We use a flipped-classroom format to ensure our live time is as valuable as possible. You watch lectures available on your own schedule, our 9+ hours of live office hours are dedicated entirely to interactive problem-solving, answering your specific questions. Information on adding these office hours to your calendar is included below. The recorded lessons will be made available starting on October 6th, with new lessons available weekly. We will send email announcements closer to the start of the course.

  2. Join this discord. This is a forum where you can interact with us and other students. There is a channel for each lesson as well as each guest speaker. For example, the name of the channel for the first lesson is called "lesson-1-fundamentals-lifecycle-llm-application-evaluation". Note that it can take up to 72 hours to add you in the discord, as you have to fill out a form and verify your Maven email address (we try to do this much faster than that). If you have any trouble with this, email alan@parlance-labs.com with your discord handle, maven email, etc so he can sort it out.

  3. We have optional hands-on exercises for students. The GitHub for the code for these exercises is located here. We will discuss this more in the lectures.

  4. We have a course reader. You can view and download it here. We will detail the chapters we are covering in the "Start Here" section of each week.

  5. We will be providing students 10 months of free access to our AI Evals assistant. This is a private AI chatbot that has access to everything we have ever created on the topic of evaluations: every course lecture, office hour recording, Discord chat, public talk, blog post, and paper. You can ask it unlimited questions, 24/7, and get instant, detailed answers based on our entire body of work. While it’s still experimental, students who have tested it found it incredibly helpful for getting unstuck on specific work projects. It’s the perfect complement to our live Discord channel and office hours, giving you a way to brainstorm and get answers anytime. We will send student invitations to this service, using your maven email, starting October 10th (or earlier), with instructions on how to use it.

  6. We sent out an email with the onboarding logistics. You can see the email content here: https://docs.google.com/document/d/1fj4NM2YsxXqCuDuVRA67W3P4Ys60O9vb8y5ThA3aFCY/edit?tab=t.0

CompleteComplete this lesson

[

Next

](/parlance-labs/evals/2025-3/syllabus/modules/54394c?item=bih00l9nqae)

75%

[

Home

](/parlance-labs/evals/2025-3/home)[

Community

](/parlance-labs/evals/2025-3)