Design note
Every third-party API call is a dependency you don't control. This is the decision record for self-hosting a fine-tuned model — and the honest case against it.
This is a design note, not a postmortem — Cobalt is in development, and I'd rather show the decision and its reasoning than pretend it's already behind me.
Every product I've shipped so far calls a third-party model API. That's the correct way to start: it's fast, it's cheap at low volume, and it let me ship Contract Explainer and Repo Explainer in days instead of months. But a frontier API is a dependency, and a dependency you don't control is a risk you've deferred, not avoided. Cobalt is my attempt to take that risk back on purpose — a self-hosted, fine-tuned open-weight model that my products can call instead of a paid API.
The case for doing it comes down to four things I don't control today. Cost: per-call pricing is a rounding error at demo volume and a real line item the moment anything grows, and it scales with usage rather than with value. Drift: the model behind an API changes under me — a silent version bump can quietly break a prompt I spent a week tuning, and I find out from a user, not a changelog. Limits: rate limits and availability are set by someone whose priorities aren't mine. And the boundary: every call ships my users' input to a third party, which for the kind of documents Contract Explainer handles is exactly the wrong direction — the same data-sovereignty argument that keeps my Security Scanner local.
Self-hosting answers all four. I own the cost curve (fixed infra instead of per-token), the model version is frozen until I choose to move it, there's no external rate limit, and the data stays inside a boundary I control. For the kind of company I want to work for, that last point isn't a nice-to-have; it's often the only deployment that's legally possible.
The honest case against — because a design note that only argues one side is marketing — is that I'm trading a problem I don't have to think about for several that I do. A fine-tuned open-weight model will, today, be worse at the hard stuff than a frontier model; I'm accepting a quality ceiling in exchange for control. I take on inference infrastructure, uptime, and the ops burden the API was quietly absorbing for me. And fine-tuning is only as good as the data I curate, which is now my job to get right. Self-hosting is not free; it's a different bill.
So the decision rule I'm building Cobalt to test is narrow: self-host when control, cost-at-scale, or data boundary outweigh raw capability — and keep calling the API when they don't. The endgame isn't "never use an API." It's being able to choose per product, instead of having the choice made for me.
I'll update this note with what the first real deployment teaches me. Design notes should age into postmortems.