Name

Tools Workshops HW 1 & 2

Tools Workshops HW 3

Tools Workshops HW 4

Tools Workshops HW 5

Week 1

Week 2

Week 3

Week 4

AI Evals.png

1. HW 1&2 walkthrough with Braintrust (pre-recorded) 1.mp4

1. HW 1&2 walkthrough with Braintrust (pre-recorded) 2.mp4

2. HW 1&2 walkthrough with Phoenix (pre-recorded).mp4

3. HW 1&2 walkthrough with LangSmith (pre-recorded).mp4

1. HW 3 walkthrough with Braintrust (pre-recorded).mp4

2. HW 3 walkthrough with Phoenix (pre-recorded).mp4

1. HW 4 walkthrough with Braintrust (pre-recorded).mp4

2. HW 4 walkthrough with Phoenix (pre-recorded).mp4

1. HW 5 walkthrough with Braintrust (pre-recorded).mp4

2. HW 5 walkthrough with Phoenix (pre-recorded).mp4

1. Lesson 1%3a Fundamentals & Lifecycle LLM Application Evaluation.mp4

2. Lesson 2%3a Systematic Error Analysis.mp4

3. Braintrust Tutorial w Wayde Gilliam.mp4

4. Optional%3a Office Hours.mp4

AIE - Braintrust Intro.pdf

Lesson 1.pptx

Lesson 2.pptx

5. Lesson 3%3a More Error Analysis & Collaborative Evaluation.mp4

6. Lesson 4%3a Automated Evaluators.mp4

7. Taming diffusion QR codes with evals and inference-time scaling w Charles Frye.mp4

8. 10x Your RAG Evaluation by Avoiding these Pitfalls w Skylar Payne.mp4

9. Optional%3a Office Hours.mp4

10. Optional%3a Office Hours.mp4

Lesson 3.pptx

Lesson 4.pptx

11. Lesson 5%3a More Automated Evaluators.mp4

12. Lesson 6%3a RAG & Complex Architectures.mp4

13. Scaling Inference-Time Compute for Better LLM Judges w Leonard Tang.mp4

14. Building custom eval tools with coding agents w Isaac Flath.mp4

15. From Vibe Checks to Evals to Feedback Loops - Case Studies in Al System Maturities w David Karam.mp4

16. A Playbook For Building Al Agents You Can Trust w Udi Menkes.mp4

17. Al Evals in Vertical Industries (such as healthcare, finance and law) w Dr Chris Lovejoy.mp4

18. Arize Phoenix tutorial W Mikyo King.mp4

19. Optional%3a Office Hours.mp4

20. Optional%3a Office Hours.mp4

21. Optional%3a Office Hours.mp4

Building Custom Eval Tools with cod.txt

Lesson 5.pptx

Lesson 6.pptx

22. Lesson 7%3a Efficient Continuous Human Review Systems.mp4

23. Lesson 8%3a Cost Optimization.mp4

24. Techniques for evaluating agents w SallyAnn DeLucia (Arize).mp4

25. LangSmith Tutorial w Harrison Chase.mp4

26. From Noob to 5 Automated Evals in 4 Weeks (as a PM) w Teresa Torres.mp4

27. Solvelt%3a The Thinking Developer's Environment w Jeremy Howard & Johno Whitaker.mp4

28. Testing Real Al Products LIVE w Robert Ta.mp4

29. Fireside Chat with DSP Creator w Omar Khattab.mp4

30. Optional%3a Office Hours.mp4

31. Optional%3a Office Hours (Bonus).mp4

Lesson 7.pptx

Lesson 8.pptx