Reference checker: paste the citations from a sales page

Somewhere below the order buttons on almost every supplement page there is a heading that says Scientific References, followed by a numbered list. The checker above reads that list and tells you what each entry can be held against. The rest of this page is what it is doing, so you can do it without the tool — and the part the tool cannot do, which is the part that decides anything.

What a reference list on a sales page is for

It exists to be scrolled past. That is not cynicism; it is a description of the design. The list sits under the buy buttons, in small type, with no links, and the copy above it says “backed by science” or “clinically proven” in large type. The reader is meant to register that a list exists, feel the weight of fourteen numbered items, and move on. Almost nobody opens one.

That makes the list a signal about the seller rather than about the product. A seller who expected the references to be read would link them, quote the finding, and say which claim each one supports. When we read all fourteen on one page, five were papers in peer-reviewed journals and two of the remainder did not support the claim they were attached to. The list was assembled to look authoritative rather than to be checked — which is a reason to discount the claims stacked on top of it, and the reason this tool exists.

The five kinds of citation you will meet

The checker uses seven buckets, but five of them are the ones that turn up on a sales page, and they are worth knowing on sight.

1. A journal paper with an identifier. A DOI (10.1111/jcpe.12155) or a PubMed ID. This is the only kind that can be opened, read and held against the sentence above it. It is the start of the check, not the end: the paper exists, and now you read what it measured.
2. A journal paper without one. Author, title, journal, year, no link. Usually real, sometimes garbled, occasionally invented. The checker recognises about two hundred journal names and builds a PubMed search for the title; if the search finds nothing, treat the line as unverifiable.
3. A press release or news story. EurekAlert, PR Newswire, a newspaper. These report that a study exists; they are not the study. A press release is written by the people who paid for the research, and the checker files it accordingly.
4. A page on the seller’s own site, a retailer, or a consumer health site. Amazon listings, the brand’s own “science” page, Healthline. Not evidence of anything except that someone wrote a page. Enter the seller’s domain in the optional field and the checker will catch self-citation.
5. Nothing checkable at all. “Studies show…”, “Dr Anderson, leading researcher”, “data on file”. A citation with no identifier, no link and no recognisable journal is a sentence formatted to look like one. This bucket, and how full it is, is usually the most useful number on the scorecard.

The other two buckets — preprints and guideline bodies — are rarer on sales pages. A preprint is a manuscript nobody has yet reviewed, which is not nothing but is not a finding either. A guideline from the NHS, the CDC or a professional body is often the strongest thing on the list, and also the thing least likely to say what the seller says it says.

How to check one reference by hand in two minutes

1. Find the identifier. A DOI begins 10.; put it after https://doi.org/. A PubMed ID is a number after PMID; put it after https://pubmed.ncbi.nlm.nih.gov/. No identifier? Search the title on PubMed. If PubMed has never heard of it, stop — you cannot check what you cannot find.
2. Read the abstract for the population. Who was studied — people or cells or rats; how many; healthy or ill; for how long? A sales page for a chewable tablet citing a study of a different strain in a yoghurt, in forty Japanese adults, for four weeks, is citing something real that does not transfer.
3. Read the abstract for the outcome. What did they actually measure? A change in plaque score is not fewer cavities; a drop in a bacterial count is not better breath; a marker in blood is not a symptom in a person. Then look back at the sentence the citation hangs under and ask whether it names the same thing. Most of the time it does not, and that is the whole finding.

This is the check the tool cannot run for you, and it is the one that matters. The scorecard tells you how many lines are worth opening. The two minutes tell you whether the seller is telling the truth.

What “clinically proven” is allowed to mean

In the United States the phrase has a definition, and it is stricter than the way it is used. The Federal Trade Commission’s guidance for health products says that claims about the benefits of supplements “require substantiation in the form of competent and reliable scientific evidence”, and defines that as “tests, analyses, research, or studies that (1) have been conducted and evaluated in an objective manner by experts in the relevant disease, condition, or function to which the representation relates; and (2) are generally accepted in the profession to yield accurate and reliable results.” It then says what that usually means in practice: “substantiation of health-related benefits will need to be in the form of randomized, controlled human clinical testing.”

The guidance also gives the failure mode this tool is built around. Its worked example is software advertised as “clinically proven to improve memory” on the strength of a trial that tested three laboratory tasks of working memory; the FTC’s reading is that “these results don’t support a general memory improvement claim because there are other types of memory that weren’t tested.” A real study, a real result, and a claim it does not support. Swap “memory” for “gum health” and you have most of the supplement pages we have read. Elsewhere in the guidance, in a footnote on a well-known juice case, the point is made the other way round: merely citing clinical studies can itself amount to making a “clinically proven” claim — which is to say the reference list is not decoration in law, whatever it is in practice.

That is US guidance and binds US advertisers. Rules elsewhere differ in detail, but the shape is the same everywhere we have looked: the phrase is a claim about the quality of evidence, not about its existence, and a list of citations is not the same as a randomised trial of the thing being sold. For what the evidence actually supports in this category, see do oral probiotics actually work?

What this checker does not do

• It does not read the paper. It sorts citations by what they can be checked against. Whether a paper supports the sentence above it is the two-minute check described above, and no pattern match can do it.
• It does not score or grade a page. There is no single number, because a single number would hide exactly the distinction worth seeing: five real papers and nine empty lines is a different page from fourteen real papers that all studied something else.
• It does not know whether a journal is predatory, or whether a DOI resolves to anything. It builds the link; you open it.
• It does not know every journal. About two hundred names and abbreviations, weighted toward dental, periodontal, microbiology and nutrition titles. A real journal it does not recognise will land in “unverifiable” when there is no identifier — which is cautious in the right direction.
• It can be wrong. A hostname says where a page lives, not what it is. Read the rule it fired (the why under each classification) and disagree with it when you should.

Our illustrations and tools are free to reuse under CC BY 4.0 — see the diagrams page.

References

1. Federal Trade Commission. Health Products Compliance Guidance (December 2022). Official page, accessed 15 Sep 2026. · Funding: see ledger