Guides
Part of Competitor validation: what beginners should know
How to make sense of competitor validation framework
Competitor validation framework: a single page, three passes that fill it, a stop rule for each pass, and a closing list of what the research cannot settle.
The output of competitor work should fit on one page. If it does not, the work was collection rather than reading, and collection is the part anyone can do.
What follows is that page, section by section, with the three passes that fill it and a stop rule for each pass. Copy the headings into a document and work through them in order. The order matters: the later sections are useless without the earlier ones, and the earlier ones are the ones people skip.
What to take away
- Fill the substitute map before you look at a single competing product. Everything else is read against it.
- Each pass has a stop rule. Without one, competitor research expands to fill whatever time is available.
- The last section of the page is a list of what the research cannot settle. Writing it is the point of the exercise.
The page
| Section | What goes in it | Stop when |
|---|---|---|
| 1. Substitute map | What ten buyers do about the problem today | Ten buyers, or three consecutive answers you have already heard |
| 2. Artifact log | One line per thing a competitor's public material taught you | Five artifacts per competitor, three competitors |
| 3. Buyer's comparison set | What buyers say they would compare you to, unprompted | Five buyers, or two answers that name the same category |
| 4. Leave cost | What leaving the incumbent costs, itemized | Every line has a number or a name |
| 5. Open questions | What none of the above can settle | You have written at least three |
Pass one: the substitute map
Fill section 1. Ask ten people with the problem a single question: what did you do about this the last time it came up? Not what they think of the options. What they did.
Record each answer as a substitute: a product, a person, a spreadsheet, a phone call, or nothing. Starting from what buyers do rather than from what the category looks like is the same order the SBA's guidance on market research and competitive analysis sets out, and it is the order almost everyone reverses. Count how many times each appears. The substitute that appears most often is your real competitor, and it is frequently not a product. The pillar on competitors sets out why habits nobody has to justify beat features; this pass finds the habit.
Stop at ten people, or earlier if three answers in a row repeat what you already have.
Do not look at any competitor's website until this section is full. If you do, you will read their material as a description of the market, when it is a description of their customers.
Pass two: the artifact log
Fill section 2. For the three products that appeared most in section 1, read five artifacts each: the pricing page and an archived earlier version, a year of changelog, the current job listings, the support library's most-viewed articles, and the cancellation terms.
Where a competitor is large enough to file with a regulator, add a sixth: their own description of their market, in their own words, which you can reach through full text search of electronic filings.
For each artifact write one line in this form: what I saw, what it implies about their customers, how confident I am. Three columns. The third column keeps you honest, because most implications from a pricing page are guesses that would look like facts a month later.
Stop at five artifacts per competitor. The sixth teaches you less than the second conversation with one of their customers would.
Do not build a feature grid. If you want one for your own reference, keep it off the page; it is an inventory, not a reading.
Pass three: the buyer's comparison set
Fill section 3. Describe your offer to five buyers without naming any competitor, then ask what they would compare it to. Write down the exact words.
The category buyers put you in decides the price they expect and who inside their organization signs. If their category differs from yours, theirs wins, because it is the one the purchase will be justified in. This is the single most common surprise in the whole exercise, and it usually changes the price you can test.
Stop at five buyers, or when two independent answers name the same category.
Fill section 4: the leave cost
This section is written from the substitute in section 1 that you intend to replace, not from the most impressive product in section 2. Itemize what leaving it costs the buyer:
- Data that has to move, and who would move it.
- People who would need retraining, and who would do the training.
- Contract term remaining, and any notice period.
- Other things that depend on the current setup.
- The person who chose the current arrangement and what changing it says about them.
Every line gets a number, a name, or the word "unknown". The unknowns go to section 5. Your offer has to clear the total of the known lines before switching becomes rational, and you now know whether it does.
Fill section 5: open questions
Write what the page cannot tell you. There are always at least three, and they are usually variants of these:
- Whether anyone will buy from you at a price above your floor.
- Whether the buyers in section 3 can be reached through a channel you can repeat.
- Whether you can deliver what the substitute delivers, reliably, for a stranger.
None of those is answerable by reading. Each is answerable by putting something in front of a buyer who can refuse it, which is what the offer test is for. Section 5 is the handoff: it says which test comes next and why.
Keeping the page alive
Date it. Revisit sections 2 and 4 every month for ten minutes: check the pricing page, the job listings and the changelog, and add a line for each change. Rewrite section 1 whenever you enter a new segment, because the substitutes change even when the problem does not.
Common questions
Can I skip the substitute map if I already know the market?
No, and the people most certain they know it are the ones the map most often surprises. Ten conversations cost a few days. Building for the wrong competitor costs a year.
What if the three biggest substitutes are all "nothing"?
Then the page is finished early and the finding is clear: the problem is tolerated. That is not a reason to stop, but it is a reason to test the price before anything else, because the only way to beat "nothing" is to cost less than the discomfort.
How is this different from a market sizing exercise?
A sizing exercise asks how many buyers exist. This page asks what they do now and what it would cost them to change. The two answer different questions and market validation covers the first.
Do I need to fill every section?
Yes, including section 5. A page with no open questions is a page that claims competitor research settled things it cannot settle, and that is the reading error this structure exists to prevent.
