Building it yourself vs. WhyUser

The weekend estimate is correct

Someone on your team has said this. They are right, and they are usually the most capable engineer in the room.

In a weekend you can have agents that read a landing page, personas that hold different briefs, structured JSON coming back, and a report that reads well. It will demo beautifully.

The estimate is honest. It is just an estimate of version one, and version one is not the part that costs.

1. What breaks in month two

Three things, in the order teams hit them.

Determinism. The obvious fix is temperature zero. It is not enough. Attention behaviour and batching mean the same prompt can still return different text, so your verdict moves when your page did not, and you can never tell your edit from the model. The fix is architectural: stop letting the model decide. Count the runs in code, compute the verdict algebraically, and let the model write only the narrative on top of a number it did not choose. That is a rebuild, not a patch.

Calibration. You will run each persona in a few mindsets and split them evenly, because that is the obvious choice. Even splits over-represent the buyer who is actively shopping by more than tenfold on cold social, and simulated click rates come back in the 20 to 30% range. Real cold LinkedIn is under 1%. The output is confident and wrong, and nothing in it warns you. Getting this right means deriving weights from published benchmarks per channel, then re-deriving them when a partner measures something different. See behavioural state.

The committee layer. Almost every in-house build stops at per-role output: five verdicts, blended into a score. That misses the failure that matters, because it lives between roles rather than inside one. See section 3.

2. What cannot be built at all

Everything above is engineering. Hard, but yours if you want it.

This one is not. A record of predictions sealed before the outcome existed accumulates at the speed of real campaigns and no other speed. A team starting today has zero. In twelve months they have twelve months. There is no way to buy back time, and no amount of talent shortens it.

That matters for one practical reason. The first time your in-house simulation says the hero is broken and a VP disagrees, someone will ask how often it has been right. "We built it in-house and it feels solid" is not an answer that survives that room. See evidence ledger.

3. The failure an in-house build almost never catches

Five roles read the page. Five roles approve. The blended score looks healthy, and the build works exactly as designed.

Catching this means modelling the handoff as a graph and scoring the edges: does the evidence one role collected intersect what the next role vetoes on? That is a different piece of software from the one your weekend produced. See conflict graph.

4. About the person proposing it

Usually your best engineer, and their instinct is sound. Worth noticing two things anyway.

The estimate is for the demo, not the instrument. Nobody scopes calibration drift, determinism, or a graded record, because those problems are invisible until you are inside them.

And in your own deal, that person is the technical decision maker. If they say "we could build this," the deal stalls and no objection is ever recorded. That is a silent veto, cast inside your own company. Worth surfacing rather than routing around.

When you genuinely should build it

Four cases, and they are real.

• Simulation is your product. Then this is core and you should own every line.
• You have a research function. A team that already owns methodology and calibration is not starting from zero.
• Your use case is far from B2B committees. Consumer, in-product, non-committee. Our evidence base does not help you.
• You need something we do not do. We read pages, ads and emails. Not onboarding flows or dashboards.

The bill nobody quotes

• Model updates, each one a re-baseline.
• Calibration against real campaign outcomes, ongoing.
• Prompt and schema drift as providers change formats.
• Evidence collection: reviews, community threads, competitor coverage, refreshed.
• The engineer who owns it, and what happens when they leave.

None of this is a reason not to build. It is the reason to price the build honestly against about 30 minutes a run.

Common questions

Can I build a buying committee simulation with Claude Code?

The orchestration is roughly a weekend and it will demo well. What takes longer is determinism, calibrating the behavioural mix against real channel benchmarks, modelling the handoffs between roles, and accumulating a graded track record.

Why is temperature zero not enough for a repeatable verdict?

Attention behaviour and batching mean the same prompt can still return different text. The architectural fix is to stop letting the model decide: count runs in code, compute the verdict algebraically, and let the model write only the narrative on top of a number it did not choose.

What do in-house simulations usually get wrong first?

The behavioural mix. Splitting mindsets evenly over-represents the actively shopping buyer by more than tenfold on cold social, so simulated click rates come back around 20 to 30% when real cold LinkedIn is under 1%. The output is confident and wrong with no warning.

What can an in-house build never replicate?

A record of predictions sealed before outcomes existed. It accumulates at the speed of real campaigns and cannot be backdated, so a team starting today has none, and that is exactly what gets asked for the first time the simulation contradicts a senior opinion.

When is building it yourself the right call?

When simulation is your product, when you already have a research function that owns methodology and calibration, when your use case sits outside B2B buying committees, or when you need something the vendor does not do.