This post was originally posted on Vaporlens Patreon
Hey folks!
A few posts ago I mentioned that I was migrating VaporLens processing to deepseek-v4-flash.
VaporLens already had a shared LLM wrapper. Most processing calls used the same interface, and the new model supported structured output. I expected to change the model name, run a few games, fix some provider differences, and enjoy much cheaper processing. The model name took one line to change. Getting the pipeline to work reliably took weeks of prompt changes, schema tweaks, output repair, retries, and firm instructions to return the object I asked for.
"Just swap the model" sounds simple because it leaves out most of the integration work.
The main reason was cost.
VaporLens does a lot of LLM work for every game. It analyzes reviews, summarizes clusters, finds causes of player sentiment, builds churn data, generates metadata, and runs several feature scans. Small differences in cost per call add up across review batches and games.
My ggex-bench experiments showed that deepseek-v4-flash could produce useful extraction results at a very low cost. It wasn't the cleanest model in every benchmark, but the price and results justified trying it in VaporLens.
The first requests succeeded. The model understood the reviews, and most outputs made sense when I read them. Production needed more than plausible output, though. Every response also had to match a strict schema.
Most VaporLens prompts had evolved around the models I used before. I hadn't written Mistral-specific instructions, or Gemini-specific ones before migrating to Mistral. But I had tuned the wording, schema descriptions, and output limits whenever those models made mistakes. After a while, those accommodations disappeared into "the prompt that works." Switching models made them visible again.
DeepSeek's output often made sense without matching the requested structure. It might return the array instead of the expected root object, rename a field with a sensible synonym, or expand a short label into a full explanation. A person could understand these answers. Zod rejected them, correctly, because they didn't match the schema.
That pattern repeated throughout the migration. The interface stayed the same, but the prompts and safeguards behind it had to change.
With Mistral, I could put the task instructions in the prompt and pass the structured-output schema with detailed field descriptions separately. DeepSeek ignored that schema in roughly half of the calls, so I started including the output shape in each prompt as well.
"Return JSON" wasn't enough. The prompt had to specify:
null or an empty arrayI first added those details to the unreliable calls, then to almost every processing prompt. Each output section became a small schema contract.
Concrete JSON examples introduced another problem: the model sometimes copied their values. I replaced them with shapes such as number, string, string[2-4], and enum<positive | negative>.
The prompt also states that these shapes describe constraints, not literal values. It had become a second type system, but it produced more reliable output.
Output length had to become part of that contract too. When I passed a large set of clusters for summarization, DeepSeek often returned more than the twenty items allowed by the schema. I first raised the generation-side allowance, then sorted and trimmed the result in code. That avoided schema failures, but it also allowed larger responses than I wanted. The prompts now call twenty a hard cap rather than a target. They tell the model to prioritize topics backed by the largest clusters, count the array, remove low-priority items if needed, and return only JSON.
I also shortened the cluster representation. Multiline blocks with repeated headings and separators became one-line records containing the cluster number, size, and points. This used less context, made cluster size easier to notice, and gave the model less formatting to copy. Models with the same advertised features can still need different input and output budgets.
VaporLens processes Steam reviews in many languages. It should understand each review in its original language, translate the meaning when needed, and store the extracted result in English.
After the migration, non-English text began appearing in generated labels, summaries, emotion names, and evidence fields. Chinese appeared most often, especially while I experimented with provider routing through OpenRouter. That wasn't surprising given DeepSeek's training, but the stored output still had to be English.
The old instruction, "write the output in English," didn't hold. I replaced it with explicit rules:
I added versions of these rules across feature scans, hardware summaries, review and churn analysis, game metadata, summary generation, emotion processing, curator blurbs, and podcast generation.
I also had to write a cleanup script for non-English emotion labels already stored in the database. Prompt fixes only affect future output. The model change had exposed assumptions that reached into persisted product data.
A response can match the schema and still contain unsupported claims.
The time-to-fun extraction asks whether a game is fun immediately, whether it clicks later, and whether reviews support an estimate of how long that takes. DeepSeek often converted vague comments about progression into precise time estimates. A reader might make the same inference, but the review hadn't provided the evidence.
I changed the prompt to accept an estimate only when a review contained an explicit time or count cue, and to return null otherwise.
It also forbids combining unrelated comments into an invented progression and limits the stance to a fixed set of values. I added post-generation normalization for timing values without supporting evidence.
Malformed JSON needed another safeguard. Most failures involved valid JSON that didn't match the schema, but occasionally DeepSeek returned JSON with a small syntax error that the parser couldn't read.
I added a fallback to the OpenRouter wrapper for those cases. On a JSON parsing error, it passes the raw output through jsonrepair package, parses it again, and validates the result against the original Zod schema.
The wrapper returns the repaired object only when validation passes. Otherwise, the call still fails as before.
The reliable sequence became prompt constraints, schema validation, then deterministic repair / normalization. None of those layers was sufficient by itself.
My earlier migration from Gemini to Mistral was much easier. Both were larger, more capable models that followed instructions and produced structured output reliably. They also cost considerably more.
That migration reinforced my assumption that models exposed through the same interface would behave similarly enough to swap. Moving to DeepSeek showed where that assumption broke down.
Changing the model affected:
DeepSeek offers a different tradeoff from Gemini or Mistral. Its low price reduces the cost of every processing run, but some of that cost moves into integration work. The pipeline needs stricter prompts, tighter validation, and more repair and normalization to get reliable results from the cheaper model. The model bill is lower, but the surrounding code has to do more.
For VaporLens, that tradeoff was worth it. Processing now costs much less, and the pipeline states and enforces assumptions that the previous models happened to satisfy. The final diff contained far more than a new model name because the model string was only one part of the integration.