Prompt2Shell

Prompt2Shell started from a small, well-bounded problem: translating a plain-English task into a shell command. The boundary matters. Shell syntax is unforgiving, but the domain is narrow enough that a small model can learn the shape of the output instead of relying on a general assistant's improvisation.

Fine-tuning a 3.8B model

I fine-tuned Phi-3-mini (3.8B) with QLoRA: the base model loads in 4-bit while low-rank adapters train in higher precision. That keeps the job inside a modest GPU while producing a small adapter that can be merged back into the base weights. The result improved ROUGE and BLEU by roughly 30% over the base model on the evaluation set.

Serving it cheaply

The merged model is converted to GGUF and served with llama.cpp on a Modal GPU. A FastAPI proxy on Render keeps the model endpoint private and handles the request shape the frontend expects. The React and Vite frontend lives on Vercel and deploys through Git-triggered CI/CD. Each piece scales or sleeps independently, which kept the project affordable to run.

Moving inference off the request server cut latency by about 80% compared with the first version. The interface is deliberately plain: type what you want, get one command, copy it or ask for an explanation. No chat history, no agent loop.

What it taught me

A narrow task is a good reason to fine-tune, and a bad reason to trust the output blindly. The useful engineering was in the contract around the model — input shape, output shape, evaluation, and a deployment that makes the whole path observable. That is the same pattern I use for production voice agents, only with audio instead of shell syntax.