https://www.youtube.com/watch?v=quh7z1q7-uc

3 ways of using LLM’s:

• Using a proprietary service (ChatGPT, gemini, claude, etc.)
• Running a Local LLM (GPU required)
• Deploying an LLM API end point (on a server)

<aside> 💡

Objective: Developing an LLM

</aside>

Steps for building LLM’s

Stage 1: Data preparation and architecture coding

Stage 2: Pre-Training

Stage 3: Fine-Tuning

This tutorial is derived from the book (Building a large language model from scratch) and contains more information that what is covered here.

The tutorial mainly focuses on developing steps 1, 3, 5, 7 from scratch

Step 9: Instruction Fine-tuning is covered in larger detail in the book but for the duration of this tutorial it is done using LitGPT.

Setup Links

Tokenizer