Train Your Own LLM - companion code
Chapter 12: Is It Good Enough? Evaluating Your Model

HOW TO RUN
Use the Python environment from Chapter 4 (Python 3.12 or 3.13,
versions pinned in Appendix F). Unzip this folder, cd into it, and run each
script with: python <script>.py   (each example's caption in the book
gives its file name). Expected output is printed in the book.
Scripts download models and datasets from the Hugging Face Hub on first run.

CHAPTER NOTES
ch12_forgetting.sh needs lm-eval (pip install lm-eval==0.4.13). The model reply files
are included so the scoring scripts run without a model.

FILES THAT NEED A GPU
- None. Every script here ran on a CPU-only test machine,
  except any recipe the book marks as not run on our test machine.

FILES
- ch12_forgetting.sh
- ch12_judge.py
- ch12_loss.py
- ch12_overlap.py
- ch12_scorecard.py
- ch12_stats.py
- shop_style_train.jsonl
- shop_style_test.jsonl
- shop_messy_test.jsonl
- ch12_replies.jsonl
- ch12_messy_replies.jsonl
- ch12_judge_ref.jsonl
- eval_report_quietbrew.md

LICENSE
The code is released under the MIT License (see LICENSE).
Data files that scripts download keep their own licenses.
