Problem discovery framework: the parts worth your attention

Most discovery frameworks are a list of activities. Talk to users, map the journey, define the persona. They tell you what to do and never what would make you stop, so they can be completed in full while teaching you nothing.

This one is smaller. A problem statement is a bundle of claims. You test one at a time, cheapest fatal claim first, and rewrite the statement when a claim fails. That is the whole loop, and its value is that it terminates.

What to take away

Split the statement into claims

Write the problem in five parts and each part becomes a claim you can check on its own.

The five parts are the ones worth your attention: who has the problem, the situation in which it appears, the current method those people use, the cost of that method, and the frequency with which the problem comes up. Anything else in the statement is decoration, and decoration is what makes a problem impossible to test.

Sketched out for one problem, the five parts read: who — freelance designers; situation — a project ends and the invoice goes unpaid; current method — chasing the client by email; cost — the follow-up time and the awkwardness it creates; frequency — most months. Change one part and you are looking at a different problem, not a different wording of the same one.

Each part is a claim, and any one can be false while the other four hold, and each failure means something different. The five parts are not the only claims a statement carries: the table later in this piece lists the claim types you will meet most often, and they do not map one-to-one onto the parts. The U.S. Small Business Administration's guide to market research and competitive analysis asks for the same components under different names, which is a reasonable check that you have not left one out.

Which claim to test first

Not the cheapest. The cheapest fatal one.

Ask of each claim: if this were false, would the idea survive? Usually two or three would be fatal and the rest would only reshape things. Among the fatal ones, pick whichever can be checked for the least money and time, and check that.

This ordering is what people get wrong. Effort drifts toward the claims that are pleasant to investigate, which are the ones about the current method, because people enjoy talking about their workarounds. The claim about who pays sits untested for months, because testing it involves an uncomfortable question.

The cheapest check for each kind of claim

The right-hand column is the discipline. Each check settles one thing and leaves the next question open, and pretending otherwise is how three conversations turn into a business plan.

The findability claim is the one that rarely needs a conversation. Public directories do the work — professional association member lists, conference speaker rosters, and searches on a professional network like LinkedIn — and the test is only whether twenty names that fit the claim appear without strain.

Run the cycle

State the claim in one sentence, so that a specific observation could contradict it.

Name the observation that would count as failure before you look. Not a feeling: something you could show someone.

Run the smallest version of the check that could still produce that observation. Most of these checks are conversations, and how you run one so it recovers the past decides whether the answer is worth anything. Where the check is written rather than spoken, draft the wording of the question before you draft the answer options, because the wording is what decides the answer you get.

Write what happened, separating what you saw from what you concluded.

Rewrite the statement, or confirm it in writing with the date. A cycle that ends with no written change is where the record goes wrong, because in a month you will not remember whether the claim was tested or skipped.

Then pick the next claim. Most statements need four or five cycles before they stop moving.

When a claim fails

The failure of one claim is rarely fatal to the idea, and treating it as fatal is as much of a mistake as ignoring it.

The one thing not to do is soften the claim until it passes. A statement edited to survive its own test is no longer a test.

Where the loop ends

It ends when a cycle produces no change and you can predict the next conversation before you have it, which is the stopping condition problem discovery describes. At that point the remaining questions cannot be answered by asking anyone anything.

The next instrument is an offer somebody has to accept or refuse. That is a different loop with a different unit of evidence: see offer testing, and minimum viable offers for keeping the first one small without making it dishonest.

Common questions

Can I test two claims at once?

Only if a failure would tell you which one failed. Usually it would not, and you spend the week to buy one ambiguous number. If you can only afford one check, spend it on the fatal claim.

What if a claim cannot be checked cheaply?

Check something that correlates with it, and label the result as indirect. An indirect check is worth running and worth marking, so that a later decision does not treat it as the direct evidence it never was.

How is this different from just talking to people?

Talking to people is the method for several of these checks. The framework decides which check comes next and what would count as failure, which conversation alone does not. What happens when it is missing is cataloged in the discovery mistakes.

Does the statement ever get shorter?

Usually yes. Early statements try to describe everyone, and the useful ones end up narrower and more specific. That narrowing is the main output. A statement that got broader is normally a sign that contradicting evidence was absorbed rather than acted on.