← All thinking
Note 179 min readThe Judgment Problem

The first guess is supposed to be wrong

Experienced judgment isn’t knowing the answer sooner. It’s getting to a useful model faster, and that almost always starts with a guess that turns out to be wrong.

A pile of projects that happen to make money is not a business, and from the inside the two look almost identical. Every project has a customer on it. Every one has revenue attached. The P&L doesn’t complain.

I spent a stretch working alongside a software division built exactly that way. It kept producing little software products. Not little features. Products. I’d find another one every week or two: a spreadsheet somebody had been running for a year turned into a standalone application, something built for one customer turned into a thing a salesperson was now supposed to sell. Each one arrived with code, infrastructure, support, security, maintenance, and everything else that quietly shows up when a useful tool becomes software. Eight or ten of them, with the bandwidth to support two.

And almost none of them looked capable of making real money. Software only pays one way: build it once, sell it a thousand times, so the cost of serving the next customer rounds to nothing. One customer inverts that. The whole maintenance load sits against a single contract and never divides.

Then it compounds, because the cost doesn’t stop when the build does. Put rough numbers on it. Say one of these products brings in $50,000 a year, and keeping it alive runs 400 hours: support, security patches, a dependency that moved, a customer who needs it to talk to something new. At $100 an hour that’s $40,000 of engineering to hold onto $50,000 of revenue, and the $10,000 left over isn’t the interesting part.

The 400 hours is. That was the only capacity available to build something worth five million, and it’s committed for as long as the customer exists. Now run it eight times. Thirty-two hundred hours a year, better than a full engineer and a half, spent entirely on keeping yesterday alive. Little apps add up. Nothing about them scales.

So why build the next one at all, when the same people could be going after the five million?

My first explanation wasn’t sophisticated. I figured the guy running the division liked building things. Hobbyist with a budget. Nobody saying no. An engineering organization with no commercial discipline anywhere near it. From a distance it looked like somebody screwing around with company money, and most people I talked to had landed somewhere close to that.

It was plausible. It was also wrong, and wrong in the way that matters most: I had made it about a person’s character, which is the laziest place a diagnosis can come to rest.

So I kept asking. Nothing formal. Who owns this one? Where did that one come from? Who asked for it? Why did a spreadsheet become an application? Is anyone actually selling it?

Eventually I found someone who still carried a piece of the history almost nobody else had. The person running the division had been told, by someone well above him, to go make money selling software.

That was the whole thing. He was doing what he had been told to do.

Something doesn’t add up

I think we misunderstand what experience looks like when it meets a messy business problem. We picture someone walking in, studying the evidence, and recognizing the answer. That happens. Not often, and not usually first.

More often the first useful thought is much less impressive than that. It’s: that doesn’t make sense. The economics don’t work. The customer behaves differently than the model says they should. Everybody agrees something is important and nobody has touched it in six months. A department is hitting every target while the company misses its number. People keep making a decision that looks obviously irrational.

Those contradictions are the good stuff. They don’t tell you what’s happening. They tell you that your picture of what’s happening is missing a piece. I had no idea why all those little products existed. My theory was a placeholder, and I knew it. What I actually had was the useful thing: an explanation that didn’t cover what I could see.

The first guess isn’t the diagnosis

There’s a comfortable idea that good judgment means withholding judgment until you have enough information. It sounds responsible. Inside a real company it will leave you standing still for months.

You’re never going to have all the information. The data is incomplete, people remember the same meeting three different ways, the CRM tells one story and the financials tell another, half the decisions that matter were made verbally and never written down, and some of the people who made them don’t work there anymore. Waiting doesn’t fix any of that. It just costs you the quarter.

So form an explanation early and hold it loosely. Then ask what else would have to be true if it were right.

If he’s building pet projects because he enjoys building software, what should I expect to find? Projects clustered around his own interests. No commercial process anywhere near them. People who tried to stop him. A pattern of starting more of them despite years of evidence that they don’t sell.

Now go look. The hypothesis gives you somewhere to walk. The evidence tells you whether to keep walking.

The closest thing I know to this is gradient descent. You’re somewhere on a landscape trying to find the lowest point, and you don’t need to know where that point is before you move. You take a step in the direction that looks lower, see what you learned, adjust, step again.

Diagnosing a company feels like that. You start with a wide set of possible explanations and every conversation tells you a little more about the shape of the thing. Sales says the product is too hard to sell. Product says sales doesn’t understand the product. Customer success says the customers sales keeps bringing in were never a fit in the first place. Those aren’t three independent complaints. Put them next to each other and something starts to show, so you go back around. Which customers are hard to sell to? Who was this actually designed for? Which customers stay?

The questions narrow because the model is improving. I’m not collecting every available fact before I form an opinion. I’m forming opinions immediately and using the next question to try to kill them.

Five people, five stories

This is also why I compare conversations instead of treating each one as a source of truth. If five people tell me five different versions of the same event, I often don’t need to work out which one is right. The disagreement is the data.

Why does sales think the product is for one customer while product thinks it’s for another? Why does leadership call a project critical while the people staffed on it treat it as optional? Why does finance think a product makes money while the engineer maintaining it can name its three customers?

Sometimes one follow-up question resolves it. Sometimes you find two people using the same word to mean different things. Sometimes you find a decision made three years ago that got passed down the org and gradually turned into something nobody intended. And sometimes you find the missing fact that makes a whole series of strange decisions snap into place.

With the little software products it went: why are we building all this? Then, who owns these? Then, who decided they were products? Then, finally, what was this man actually asked to do?

Go make money selling software.

One piece of information changed the meaning of everything around it. That’s the moment you’re working toward. Not certainty. Coherence.

Companies are machines made out of people

We analyze companies as if they were machines. Inputs, outputs, processes, metrics, structures, capital allocation, funnels. All useful. But the machine is made out of people, and people respond to incentives, status, fear, history, identity, and what they believe their boss wants from them. Leave that out and perfectly rational behavior looks insane.

As a portfolio strategy, what I was looking at was indefensible. Build a standalone application, sell it to one or two customers, then support it, secure it, maintain it, and keep it compatible with everything around it, forever. Repeat, until the maintenance bill for the things nobody is buying is the reason you can’t build the thing somebody would. Seen through the eyes of a man who had been told to go sell software, it was the obvious thing to do.

He wasn’t the irrational part of the system. The system handed a reasonable person an instruction and then got the logical consequence of it.

That distinction matters because it changes what you fix. If my first theory had been right, the fix was governance: control what he’s allowed to build. That wouldn’t have touched this problem. The real question was one level up. What was the software business supposed to be? A software company? A place to build products that could become real businesses on their own? Software that existed to make the consulting business more valuable? Was software the thing being sold, or part of delivering something else?

Those are four different companies. Nobody had ever written down which one this was, so people filled in the blank themselves, reasonably, with the last clear instruction they’d been given. That’s a company roadmap question, and almost nobody has one of those. They have a product roadmap and they assume it’s the same document.

As far as I know, nobody ever answered it. I raised it. Then I moved on, and maybe I didn’t push hard enough.

Most bad decisions made sense to somebody

So this is the assumption I start from now. When you find something that looks stupid, assume for a minute that it isn’t, and ask what would have to be true for a reasonable person to do it.

Maybe their incentive was different from the company’s. Maybe the information they had was different from yours. Maybe it was the right call three years ago and nobody revisited it. Maybe they’re optimizing the metric they were handed. Maybe somebody important told them to.

You don’t have to excuse a bad decision. You do have to explain it. “Those people didn’t know what they were doing” is a remarkably low-information explanation, and it closes the investigation at exactly the point where it should be starting.

Sometimes you find the wrong bottom

There’s a problem with the gradient descent analogy, and it’s the same problem the real algorithm has. You can find a local minimum. You can follow the evidence downhill, arrive at an explanation that fits everything you’ve looked at, and still be wrong. The answer is coherent. It just isn’t the right one.

Companies are unusually good at manufacturing those, because information clusters. Talk only to engineering and you’ll build an excellent engineering explanation. Talk only to sales and the sales explanation gets more convincing every hour. Read twelve months of leadership decks and you’ll end up seeing the company roughly the way leadership already sees it. Every additional fact makes you more confident while keeping you in the same valley.

So every so often you have to jump. Talk to the customer nobody suggested. Chase the financial number that has no business appearing in an operational analysis. Ask somebody junior what everyone senior takes for granted. Find the person who was in the room when the decision was originally made. Ask what evidence would make your current explanation impossible.

The goal isn’t a more coherent model. It’s to keep giving reality chances to break the one you’ve got.

AI will make the wrong answer beautifully coherent

This matters more now, because a language model is extraordinarily good at taking the information you hand it and building a plausible explanation out of it. Give it your sales data and ask why conversion fell, and you’ll get a compelling answer. Give it employee surveys and ask about morale, and you’ll get themes. Give it customer calls and ask what to build, and it will hand you a roadmap.

The danger isn’t that the analysis is bad. It’s that it’s good enough to stop the investigation.

It can’t interview the person nobody thought to include. It doesn’t know about an instruction given verbally two years ago, because nobody wrote it down. It doesn’t know that “pipeline” means one thing in finance and another in marketing unless that contradiction happens to appear somewhere in its context. It will reason beautifully from an incomplete map. So will we. That part isn’t new.

I use it constantly. I just treat what it gives me the way I treat my own first theory. What would have to be true for this to be right? What evidence would contradict it? Who knows something that isn’t represented here? What does this explanation fail to explain? And the one I get the most out of: what would make this apparently irrational behavior rational?

Being wrong is cheap from outside

People avoid forming an opinion because they don’t want to jump to conclusions, and that’s sensible if the only alternative is making a snap judgment and defending it for a year. It isn’t. You can reach a conclusion quickly and attach almost no ego to it. I think it’s this. Let’s see. Nope. Then maybe it’s this. Let’s see.

That isn’t recklessness, it’s search. And the advantage experienced people have isn’t that their first guess is right more often. It’s that they generate better guesses, they notice the contradiction sooner, they know which question separates two explanations, and they’ve been wrong enough times that dropping a theory doesn’t feel like losing anything.

Which is the part that’s hard to do from inside your own company. Everyone in the building has a stake in the current explanation. They authored it, or they inherited it, or their team is named in it, or the budget was approved on the strength of it. Being wrong costs them something, so the first guess gets defended instead of tested, and the investigation stops at the first answer that lets everybody stay who they are.

I didn’t get to the software answer by gathering everything and deriving it. I noticed something that didn’t make economic sense. I guessed why. I was wrong. I asked another question, and another, asking and asking and asking, until the space got small enough that one fact changed the meaning of everything I’d already seen.

You don’t start with the map. You start with the thing that doesn’t fit the map you have, and you walk toward it.

The first guess is supposed to be wrong. Its job is to get you moving.

Know where this has to go, and not how?

Twenty minutes. Tell me where you want this company to be in two years and I’ll tell you what I’d go look at first.