GPU

2 articles tagged with "GPU"

Part 2: The Local GPU Challenge and Defeat -- Taking on an 8B Model with 6 GB of VRAM on the RTX 3060 Series

Part 2: The Local GPU Challenge and Defeat -- Taking on an 8B Model with 6 GB of VRAM on the RTX 3060

My first attempt at fine-tuning ELYZA Llama-3-JP-8B on a local RTX 3060 GPU, and the hard lessons learned about VRAM limitations when training large language models.

Read →
Part 3: LoRA and Quantization Explained -- The Techniques That Make LLMs Lighter Series

Part 3: LoRA and Quantization Explained -- The Techniques That Make LLMs Lighter

A deep dive into LoRA, quantization, QLoRA, and SFTTrainer -- the key techniques that enable fine-tuning large language models on consumer hardware, validated through a Mount Fuji proof-of-concept experiment.

Read →