model-training
working prototypeModel Development
Two tracks for learning and using the model layer: a char-level GPT written in PyTorch and a QLoRA fine-tuning pipeline over a personal Obsidian vault, ending in GGUF export.
The problem
I wanted the model layer to stop feeling magical: one path to understand the mechanics, another to adapt an instruct model to private data without sending notes away.
Architecture
Track 1 implements attention, transformer blocks, LR scheduling, bf16 autocast, gradient clipping, and checkpointing in PyTorch. Track 2 wires Unsloth QLoRA fine-tuning with synthetic SFT data from a local teacher model, targeting native bf16 on Blackwell-class hardware.
What it demonstrates
The loop is data prep → train → eval → quantized GGUF export → local inference. Code is committed; training runs and large artifacts stay local so personal notes do not leave the machine.