The Contoso Convergence, Episode 11: Character Sheets
An Advanced Dungeons & Dynamics 365 session. Episode 11 of 14.
Previously on: Episode 10, The Wayfinder’s Gap, the party discovered that Contoso’s ninety four percent training completion rate measured attendance, not competence, and that six of eight power users couldn’t complete a basic order to cash cycle without help, three weeks before go live.
A readiness assessment tells you who’s struggling. It doesn’t always tell you why. This episode, the party finds out that the worst score in the room might be measuring exactly the wrong thing.
The Session
GM: The party builds character sheets for each of the eight power users: role, current XP, gaps, and a coordinate on the Wayfinder’s Network showing where they stand versus where they need to be.
Sable: Denise scored lowest on the readiness test. Struggled with three of five scenarios.
Ai. Cassiopeia: I pulled her transaction history for context, as requested. She has the lowest error rate of any user in AP. By a wide margin.
Marge: How does someone with the lowest error rate score lowest on a scenario test?
Vex: Because the test measures speed and screen navigation. It doesn’t measure judgment.
Thorne: She’s not undertrained. She’s over cautious, because she’s the one who’s been quietly catching everyone else’s mistakes and never gets asked how she knew to check.
Denise: (overhearing from the doorway) Nobody asked, so I didn’t say anything.
The room turns. Denise steps in, arms crossed, more defensive than embarrassed.
Marge: Denise, walk us through why you struggled on the partial shipment scenario.
Denise: I didn’t struggle. I stopped. There’s a field on that screen that’s populated wrong about one time in six, and if you don’t catch it before you post, it throws off freight allocation two steps downstream. The test wanted me to move fast. I’ve learned not to move fast on that screen.
Sable: (quietly) That’s not a training gap. That’s institutional knowledge nobody documented.
Denise: I mentioned it once, in month two. It didn’t go anywhere, so I stopped mentioning things.
Ai. Cassiopeia: I recommend her proficiency path include a mentor tier, not a remedial tier.
Vex: Denise, that field, the one that’s wrong one time in six. What’s actually causing it?
Denise: (pausing, surprised to be asked) I always assumed it was a timing issue with the freight feed. Nobody ever asked me to look closer.
Vex: I have some time this afternoon. Want to look closer together?
Denise: (a beat, then) Yeah. Actually, yeah.
The party rebuilds the plan. Denise goes from “at risk” to “quiet expert” in one conversation, and the readiness map shifts under everyone’s feet.
What a readiness score actually measures
Every assessment measures something specific, and it’s worth asking exactly what before you trust the score it produces. Contoso’s readiness test measured speed and screen navigation under time pressure. That’s a reasonable thing to measure, but it’s not the same thing as competence, and Denise is the proof. Her caution wasn’t a skill gap. It was a defensible, earned response to a system that had already burned her once, and the test penalized the exact behavior that was protecting the business every single month.
The more troubling detail isn’t the test design, though. It’s that Denise had already raised this issue once, in month two, and it went nowhere. That’s the part that should sting a little, because it means the information the party needed was sitting in the building the entire time, just not in a form anyone had built a channel to receive. Nobody was hiding it from her. There was simply no mechanism that treated a frontline observation from someone without formal standing as worth following up on.
This is the actual argument for a proficiency system that includes mentor tiers alongside remedial ones. A readiness score that only sorts people into “needs help” and “doesn’t need help” throws away the most valuable signal in the data: the person whose caution is actually expertise wearing an unflattering score. Vex asking Denise to look closer together, instead of just logging her workaround and moving on, is the moment this episode is actually about. Competence that never gets asked a follow up question stays invisible indefinitely, no matter how many times the person tries to mention it.
What happens next
Denise’s insight reshapes the readiness map and buys the team a genuine mentor for the seven days that remain. But rebuilding proficiency around real expertise instead of raw scores means someone finally has to go back into the system Denise flagged, and what Vex finds when he looks closer is worse than a timing issue with the freight feed.
Next episode: Episode 12, The Customization Nobody Remembers (coming soon)
If someone on your team has mentioned a workaround once and then quietly stopped bringing it up, this episode is for them. Gamifying the Enterprise: Game Mechanics for Continuous Proficiency is available now on Amazon: https://www.amazon.com/dp/B0GY3VWLVX
And if you want a system built to actually catch signals like Denise’s instead of losing them in a scorecard, start here: adnd365.com/start
A Tinkerers Notebook 