Autopsy #8: The Performance Demo
Cause of death: ten records don’t behave like ten million, and nobody in the room was thinking in millions.
The presenter clicks a button. A report renders instantly. A search returns results before the loading spinner has time to spin. A batch job that would, in your world, run overnight, completes in the time it takes to say “and here’s the output.” The room absorbs all of this the way it absorbs everything else in a demo: as evidence of how the product performs.
It is not evidence of how the product performs. It is evidence of how the product performs against a database with four hundred customers, running on hardware provisioned for a demo, with exactly one user logged in, who is the only person generating any load at all. Your environment, on day one of production, will have none of those three things in common with it.
What actually happened
Performance is not a property of software in isolation. It’s a property of software under a specific load, against a specific dataset size, with a specific number of concurrent users doing specific things at the same time. A demo environment is engineered, whether deliberately or just by the natural economics of running a sales demo, to minimize all three of these variables simultaneously. Small dataset, dedicated hardware, single user. That is close to the best-case condition the software will ever run under, and it’s the condition you’re being shown as though it were representative.
The gap between that best case and your actual production reality is usually invisible in the room because nothing about the demo interface tells you the dataset is small. A grid with four hundred rows and a grid with four million rows look identical on screen, for the same reason they looked identical in the migration autopsy: you’re only ever looking at the same twenty rows at a time. Query performance, index behavior, and lock contention all degrade in ways that simply don’t exist yet at four hundred rows, and none of that shows up until the dataset, and the concurrent user count, both climb to something resembling your real operation.
Batch and integration jobs hide the same gap differently. A nightly job that processes four hundred records in three seconds tells you almost nothing about how it will behave processing four hundred thousand records at 2 a.m. while five other scheduled jobs are also competing for the same database connections. The three-second version and the six-hour version can be, technically, the exact same code.
Why it works on smart people
Performance is one of the few things a demo can show without narrating, which makes it feel more objective than the parts that require a presenter’s framing. Nobody has to make a claim about speed. You just watch it happen, in real time, and speed you watch with your own eyes reads as harder evidence than speed someone tells you about. That instinct is usually correct. It’s just being applied to a measurement taken under conditions that will never recur once the system goes live.
There’s also a scale-intuition gap that’s genuinely hard to close without direct experience. Most people don’t have a strong internal sense of how nonlinearly performance can degrade as data volume and concurrency grow. A system that feels instant at four hundred records doesn’t necessarily feel merely a little slower at four million. Depending on how indexes, queries, and locking are built, it can fall off a cliff at some threshold nobody in the demo room has any way of anticipating.
The actual damage
This is the one that shows up as a go-live incident rather than a slow discovery, because performance problems under real load tend to announce themselves immediately and all at once, usually during month-end close or the first day the full user base logs in simultaneously. Reports that took two seconds in the demo take four minutes. A batch process that ran in three seconds against sample data doesn’t finish before the next scheduled job needs the same resources, and the two start colliding every night.
The remediation at that point is expensive and disruptive in a way that early testing would not have been: emergency performance tuning, index rebuilding, sometimes an infrastructure upgrade that wasn’t budgeted, all happening under the worst possible conditions, with live users blocked and a go-live date already spent.
The fix, if you’re the one presenting
Test and show performance against something that resembles your prospect’s actual scale, not the vendor’s default demo dataset. If a true load test isn’t feasible in the sales cycle, at minimum say so explicitly: “this demo is running against four hundred sample records on dedicated hardware, here’s what we know about performance at your expected volume, and here’s how we’d validate it before go-live.” That sentence costs you the illusion of effortlessness. It buys you a prospect who understands what they’re actually being shown, and a performance conversation that happens in scoping instead of in a production incident.
Speed you watched with your own eyes is still only evidence of the conditions you watched it under. Ten records were never going to tell you what ten million would do.
I’ve argued in The Same Four Systems that the same organizational patterns show up whether you’re a corner store or a Fortune 500 company, just with higher stakes. That holds for structure. It doesn’t hold for performance. A query pattern that’s invisible at four hundred rows can become the whole story at four million, and no amount of pattern-matching from a small scale prepares you for exactly where that threshold sits.
A Tinkerers Notebook 