Train Your Own LLM - companion code
Chapter 10: Teaching Preferences: DPO and a First Look at RL

HOW TO RUN
Use the Python environment from Chapter 4 (Python 3.12 or 3.13,
versions pinned in Appendix F). Unzip this folder, cd into it, and run each
script with: python <script>.py   (each example's caption in the book
gives its file name). Expected output is printed in the book.
Scripts download models and datasets from the Hugging Face Hub on first run.

CHAPTER NOTES
ch10_dpo.py needs prefs_train_v2.jsonl and prefs_eval.jsonl from Chapter 5
(ch05_project4_fix.py).

FILES THAT NEED A GPU
- None. Every script here ran on a CPU-only test machine,
  except any recipe the book marks as not run on our test machine.

FILES
- ch10_dpo.py
- ch10_dpo_by_hand.py
- ch10_dpo_diag.py
- ch10_grpo.py
- ch10_grpo_diag.py
- shop_json_train.jsonl
- shop_json_test.jsonl

LICENSE
The code is released under the MIT License (see LICENSE).
Data files that scripts download keep their own licenses.