What a thermometer cannot see, and why I published that on the tool

Hacker News reads 47 degrees. Wikipedia reads 81 with an AI reading of 65. Any instrument that measures one page from outside has a specific, nameable blind spot, and hiding it would cost more than admitting it.

Updated

This thermometer cannot see links, traffic, rankings, rendered content, or whether the writing is any good. Those are most of what decides whether a page wins. I have published that limit on the tool itself rather than in a footnote, because the alternative is a reader discovering it on their own and concluding the whole thing was a sales pitch.

The evidence for the blind spot is in my own dataset

Hacker News: 47 degrees. Search Visibility 33. It is one of the most linked-to pages in technology and it reads two degrees below the midpoint.

Schema.org: 69 degrees, entity strength 48, on 224 words. It is the vocabulary that structured data itself is built on.

W3.org: 62 degrees. The organisation that has been shaping the web since 1994.

I could have left those out of the published dataset. Nobody would have known. A set of seventeen readings where every reference site scored respectably and every portfolio site scored well would have been a much better advertisement.

It would also have been the single most fragile thing on the site, because the first reader who tested Hacker News themselves would have found the number I chose not to publish, and everything else here would have become suspect at once.

The five things it genuinely cannot do

It cannot see links. No backlink data, no reference to who cites the page. This is the largest single omission and it explains the Hacker News result completely.

It cannot see performance. It measures time to the document, not time to a usable page. A site can return HTML in 90 milliseconds and take nine seconds to become interactive, and this reads the first number.

It cannot see rendered content. By design, because that is the question the AI reading is asking. But it means the reading is about the response rather than the experience, and those diverge.

It cannot see the rest of the site. One page. The homepage, which is the page least representative of any site and the one most likely to have been looked after.

It cannot tell whether the writing is any good. Not even slightly. It counts words. A page of 6000 excellent words and a page of 6000 generated ones read identically, and the second one is more common than it used to be.

What it is left with

Three uses, and they are genuinely useful.

Deciding whether to look closer. That is what a thermometer is for in the room it was invented for. A reading of 41 does not diagnose anything and it absolutely tells you to keep going.

Comparing two pages on the same basis. The absolute number carries the blind spot; the gap between two numbers measured the same way mostly cancels it out. This is the version people actually use.

Watching one page change. The most valuable and the slowest. Same page, same instrument, different dates. Everything the thermometer cannot see stays constant between two readings of the same URL, which means the difference is signal.

Why publishing the limits is not modesty

It is the only durable position, and I would take it even if I thought nobody would check.

A tool that implies completeness makes a promise it has to keep on every single reading, forever, with no way to recover from being caught once. A tool that states its boundary makes a much smaller promise and keeps it easily. When someone runs this against a site they know is doing well and gets 51 degrees, the page explaining what it cannot see is already there, and the tool stays credible through a result that would otherwise have ended it.

There is a version of this argument that is just marketing about being honest, and I want to be careful not to write that one. So, concretely: the full signal registry is generated from the engine, so it cannot describe a check that no longer runs. The dataset includes the results that embarrass the instrument. The method page lists the limits before it lists the features.

None of that is generous. It is the cheapest possible insurance against the one failure this kind of site cannot survive, which is a reader working out that the number was never the point.

Filed under: limitations, honesty, measurement


Take your site's temperature