Version 1.17 · locked 2026-11-19
Methodology
This page is generated from the repository’s README, which is the governing text. Every change to it is in the commit history.
Scoring rubric
Each product is scored 0 or 1 on each of six criteria. No weighting, no partial credit. Maximum score: 6.
| # | Criterion | Scores 1 when |
|---|---|---|
| 1 | Exact amount disclosed for every active ingredient | No proprietary blend appears anywhere on the Supplement Facts panel |
| 2 | Standardization percentage stated where potency depends on it | e.g. % withanolides, % curcuminoids |
| 3 | Specific chemical form named, or strain designation for live organisms | "Magnesium glycinate," not "magnesium"; "L. rhamnosus GG," not "L. rhamnosus" |
| 4 | Disclosed dose falls within the trial range for the outcome the product claims | Cross-checked against clinical-doses.csv |
| 5 | Third-party certificate of analysis publicly accessible | No email gate, no account required |
| 6 | Amounts stated per serving, with serving size and servings per container both visible | Both figures present on the panel |
How criterion 2 is applied
Criterion 2 applies only where potency depends on a marker compound: standardized herbal or botanical extracts (e.g., % withanolides, % curcuminoids). Products containing no such ingredient are not penalized for its absence. Criterion 2 scores 1 by default when nothing in the formula requires a stated percentage.
What counts as standardization-dependent. Criterion 2 applies wherever a stated percentage is required to compare the label to a reference range — not only to botanical marker compounds. An ingredient sold at graded purities, such as xylooligosaccharide (marketed anywhere from roughly 35% to 95% XOS), declared only as an extract weight cannot be compared to a range expressed in the active compound. The percentage is therefore potency-determining and its absence scores 0. The test is functional: if the disclosed figure cannot be converted into the unit the evidence uses, the standardization is missing.
Ruling on named compounds versus extract weights (v1.15, 2026-11-19). The functional test turns on what the label declares, not on how the ingredient is sold. Where the label names the compound itself with an amount, as in "Xylooligosaccharides (XOS) 80 mg", "Inulin (Chicory Root Extract) 200 mg" or "Prebiotic FOS (Fructo-oligosaccharides) 50 mg", the amount is read as the compound and can be compared to a range expressed in that compound, so no percentage is required and the point stands. Where the label declares a source material by weight, as in "Xylooligosaccharide from Corn Cob Extract 100 mg", the active content of that weight is unknown and the point is lost. Rank 13, Lemme Debloat, forced this: it had been scored 0 alongside BIOMA, while Culturelle Pro Strength scored 1 for the same kind of declaration, and one rule cannot produce both. Lemme Debloat now scores 1 and BIOMA stays 0.
The ruling also requires a reference range to exist before a missing percentage can be potency-determining. Garden of Life Dr Formulated Probiotics for Men had been scored 0 because resistant starch is sold at graded content, but the reference table holds no resistant starch row, so there is no unit to convert into. It now scores 1, provisionally: if a resistant starch range is added, its undisclosed starch weight inside a blend total fails the test and the 0 returns, for it and for any other product in the same position.
The objection is recorded openly. A brand naming "XOS" may be declaring the weight of a commercial XOS ingredient that is itself 35 to 95 percent pure, and the label cannot show which. That is true, and it applies equally to every named compound in the sample. The audit scores what the label states, and a label that states the compound has made a claim a certificate of analysis can check.
Unlike the probiotic exclusion under criterion 4, this is not a unit-mismatch problem; the requirement simply does not apply. The criterion therefore remains a scored point rather than reducing the total possible score, keeping every product comparable on the same 6-point scale.
How criterion 3 is applied
Criterion 3 asks whether the specific chemical form is named, because the form determines what the body actually receives. "Magnesium glycinate" scores 1; "magnesium" scores 0.
Probiotics. The equivalent of chemical form for a live microorganism is the strain designation. Probiotic effects are strain-specific and do not generalize across a species: Lactobacillus rhamnosus GG has been trialled for outcomes that other L. rhamnosus strains have not, and evidence for one Bifidobacterium longum strain is not evidence for another. A label naming only genus and species does not identify what is in the bottle, and the published literature cannot be matched to it.
Criterion 3 therefore scores 1 for a probiotic product only when every listed organism carries a strain designation — for example Lactobacillus rhamnosus GG or Bifidobacterium longum 35624. Genus and species alone — "Lactobacillus acidophilus" — scores 0, and a single undesignated organism anywhere in the formula is enough to score 0. A strain designation means a specific identifier assigned by a culture depositary or the manufacturer, not a marketing name for a blend.
This rule governs the criterion 3 point only. Probiotics remain excluded from criterion 4 for the separate unit-mismatch reason given below.
How criterion 4 is applied
Clinical dose ranges are outcome-specific. Glutamine trials measuring intestinal permeability and glutamine trials measuring IBS symptom severity used different doses; zinc carnosine trials for permeability and for gastric ulcer healing used different doses. There is no single "clinical dose" for most ingredients.
Criterion 4 is therefore scored against the outcome the product claims on its label or product page. A product marketed for intestinal permeability is scored against permeability trials. A product marketed for symptom relief is scored against symptom trials.
Where the literature is contested, the reference table records the range and the disagreement in the notes field, and the product is given the benefit of the doubt: falling within any credible published range for the claimed outcome scores 1.
Where no adequate human trial exists for an ingredient at all, that ingredient is marked "no adequate human trial identified" and excluded from the product's criterion 4 assessment. A product scores 0 on criterion 4 only if at least one of its actives has a reference range and falls outside it.
Human trials only. Animal and in vitro studies are excluded from the reference table. Criterion 4 requires published human-trial evidence for the claimed outcome. An ingredient with substantial preclinical support but no human trial is treated as having no reference range.
Positive-outcome trials only. Reference dose ranges are drawn from trials that found a significant effect on the outcome measured. Trials that found no effect are recorded in the notes field but do not set the range. A dose shown not to work is not evidence that the dose is correct.
Negative findings carry their search. "No adequate human trial identified" is a claim about the literature, and it is what excludes an ingredient from criterion 4 — so a wrong one silently spares a product a point it should have lost. Rows making that claim record the date the search was run and what it returned, the same standard criterion 5 applies to a missing certificate. Absence of evidence is recorded as of a date, not asserted in general.
Absorption enhancers. Ingredients included to increase the bioavailability of other ingredients rather than to produce an independent effect are excluded from criterion 4. Their doses are recorded in the reference table where trials state them, but no trial tests them against the outcome the product claims.
Evidence hierarchy. Where more than one trial could set a range, they are not equal. Two rules apply, in order, after the human-trials-only and positive-outcome-only rules above:
- Attribution. A trial of a multi-ingredient formula cannot set a range for any single component, because the effect cannot be attributed to that component. This already governs the multi-herb blends in the reference table, and it applies identically to a finished commercial product: a synbiotic trialled whole does not establish the dose of its prebiotic.
- Independence. Where an independent trial of the ingredient exists, it sets the range. A trial authored or funded by the manufacturer of the ingredient or product is recorded in the notes, and sets a range only where no independent trial does — flagged in the row as the weaker basis.
Without the first rule a brand could establish its own dose by trialling the formula rather than the ingredient; without the second, by funding the only study. Both would let the audited party set the standard it is audited against.
Probiotics. A probiotic dose is compared to a trial dose only where both are stated in the same unit and the label's figure is one the brand commits to delivering. Trials report probiotic doses in CFU or in milligrams of preparation, and the two are not reliably interconvertible, since CFU per milligram varies by manufacturer and by viability at manufacture versus expiry. In practice that gives three cases, set out in the rulings below and restated here in v1.15 so this paragraph and the scores agree: a CFU count guaranteed through the best-by date is compared to trials dosing in CFU (v1.13); a CFU count with no through-expiry guarantee establishes no delivered dose and scores 0 on that ingredient (v1.13); a dose in milligrams is compared to trials dosing in milligrams (v1.12). A probiotic that fits none of these, such as a CFU-declared strain whose only trials dose in milligrams, or an organism inside a blend total, is excluded from criterion 4, and the v1.11 ruling governs a formula in which every active is excluded. Earlier text said every CFU-declared probiotic was excluded; the rulings had already narrowed that, and v1.15 brings the paragraph in line without changing any score.
Ruling on probiotic-only formulas (v1.11, 2026-09-26). Where every active ingredient in a product is a probiotic, criterion 4 scores 0. The exclusion above removes probiotic ingredients from the comparison; it does not award the point in their absence. Criterion 4 asks whether the disclosed dose can be checked against the trial range for the outcome claimed, and for a probiotic-only formula it cannot be, so the product does not deliver what the criterion tests. This is deliberately different from the criterion 2 treatment, where a requirement that does not arise leaves the point standing: there the label is complete as printed and nothing is withheld, whereas here the check is impossible. The objection this invites is recorded openly: a brand may fairly argue it is marked down for a units problem in the evidence base rather than for anything on its own label. That objection is sound as far as it goes, and the answer is that this audit scores what a buyer can verify, not what a manufacturer intended.
Ruling on probiotic ingredients declared in milligrams (v1.12, 2026-09-26). Where a label declares a probiotic in milligrams of preparation rather than CFU, the unit mismatch does not arise and the ingredient is scored on criterion 4 like any other ingredient. The exclusion was written for a stated reason and extends only as far as that reason does. Applying it to a label that reports its dose in the same unit the trials use would mark a product down for a problem it does not have.
Rank 9, Florastor, forced this. Its panel declares Saccharomyces boulardii CNCM I-745 at 500 mg per serving, not in CFU. A search of the literature found that independent adult randomised trials of this strain also dose in milligrams, across a range of 500 to 600 mg per day: 250 mg twice daily (Seddik 2019, PMID 30694338) and 600 mg daily (Salazar-Parra 2023, PMID 37400812). Corrected 2026-11-19: this paragraph first gave the range as 500 to 1000 mg, counting 500 mg twice daily from Sjomina 2023 (PMID 37942999). That trial concluded the probiotic did not increase eradication, and its positive subgroup results lost significance after Type I error adjustment, so under the positive-outcome rule it does not set the range. Florastor's 500 mg is inside either range and its score is unchanged. Both halves of the mismatch fail for this label, the disclosed dose can be checked, and it falls inside the range. The product scores 1.
One trial was set aside under the independence rule rather than counted: Maev 2025 (PMID 40884341) dosed 500 mg twice daily but administered Enterol, a Biocodex product, and Biocodex is the manufacturer of Florastor. It is recorded in the reference table and does not set the range. The trials above carry only academic and hospital affiliations; their funding statements were not verifiable from the abstracts and are recorded as unverified rather than asserted to be independent.
The v1.11 ruling is unchanged and still governs the ordinary case, where the label declares CFU. Nature's Bounty Acidophilus, declared as 0.5 mg (100 million CFU) against a literature reported in CFU, remains 0.
The objection this invites is recorded openly. The trials that set the range studied antibiotic-associated diarrhoea, Helicobacter pylori eradication support and acute viral diarrhoea, in patients who were being treated or were symptomatic, while Florastor is sold for digestive and immune health in daily use by people who are neither. A reader could fairly argue that a range drawn from those trials does not establish a dose for that claim, and that criterion 4 should have scored 0 on outcome grounds even with the units matching. That argument is sound enough to publish. The answer is that every one of those trials measures a digestive outcome, and that the benefit-of-the-doubt rule stated above scores 1 for falling within any credible published range for the claimed outcome. Applying a stricter outcome test to this product than to the rest of the sample would be the larger inconsistency.
Ruling on CFU declared at manufacture versus through expiry (v1.13, 2026-09-27). Where a label declares a probiotic count both at the time of manufacture and as an amount guaranteed through the best-by date, criterion 4 is scored against the through-expiry figure. That is the only quantity the brand commits to delivering to the person swallowing the capsule, and this audit scores what a buyer receives rather than what leaves the factory. Scoring the higher figure would also make the criterion easy to pass by construction, since viability at manufacture can be set as high as the process allows and says nothing about the dose taken months later.
The rule has a second half, and it is the half that makes it fair. Where a label declares only an at-manufacture count and gives no expiry guarantee, the delivered dose cannot be established at all, and criterion 4 scores 0 on that ingredient. Without this, a brand printing a single unqualified "1 billion CFU" would score better than one that disclosed its own decay, and the rubric would reward withholding information.
Applied to rank 13, Align Pro Formula. The panel declares 1 x 10^9 CFU at manufacture and 1 x 10^7 through the best-by date. The pivotal encapsulated trial of this strain found 1 x 10^8 CFU significantly superior to placebo, while 1 x 10^6 and 1 x 10^10 were not (Whorwell 2006, PMID 16863564, n=362). The guaranteed dose is ten times below the only capsule dose shown to work, and one order of magnitude above a dose shown not to work. Criterion 4 scores 0.
The objection is recorded openly. Align discloses more than most of this sample: many labels state a single CFU figure and never say what survives to the best-by date. It is uncomfortable that the more candid label is the one whose shortfall becomes visible. The answer is the second half of the rule above, which scores the less candid label no better, and the observation that the disclosure is what makes the shortfall checkable rather than what creates it.
How criterion 5 is applied
Criterion 5 asks whether a third-party certificate of analysis for the product is publicly accessible: reachable without an email address, an account, or a support request. A batch-number lookup that returns a certificate scores 1, provided no email or account is required to use it.
Three things are not a certificate of analysis:
- A "third-party tested" claim, badge, or seal with no document behind it. The criterion is about the document being accessible, not the claim being made.
- A GMP or facility registration certificate, which covers the manufacturing site rather than the product.
- A certificate issued by the manufacturer's own laboratory, which is not third-party.
A zero is a negative finding and is recorded as one. The search covers the product page, the brand site (footer links commonly named Quality, Testing, Lab Results, Transparency, or Certificate of Analysis), a site-scoped web search for "certificate of analysis", and the retail listing's images. Where that search returns nothing, the row records that no publicly accessible certificate was located on the capture date — not that none exists. A certificate published later, or reachable by a route this search did not cover, would change the score, and the dated record makes that revision auditable rather than silent.
When a label changes
Ruling on label changes (v1.16, 2026-11-19). A score describes the label a buyer
can see now, so when a re-capture shows that a scored product's Supplement Facts panel
has changed, the product is re-scored in full on the new panel, criterion 5 search
included, and the row's capture date moves to the re-capture. The earlier capture stays
in captures/ and the earlier score stays in the commit history, so the change is
visible rather than overwritten, and the product page shows both captures with their
dates. A change that touches no criterion still triggers the re-score: the point is that
the row is read from the current label, not that the score necessarily moves. Rank 21,
Digestive Advantage, forced this: its 2026-11-19 re-capture shows Total Sugars 4 g and
added sugars at 16% / 8% where the scored label showed 3 g and 12% / 6%.
Criteria deliberately excluded
The following were considered and rejected as scoring criteria:
- Use of branded or trademarked ingredients. Would reward marketing spend rather than transparency. A product using generic ashwagandha at a clinically studied dose scores identically to one using a branded equivalent.
- Price or value. Not a transparency measure.
- Taste, packaging, brand reputation. Subjective.
Sample selection
The top 30 gut-health supplements ranked in Amazon Best Sellers in Probiotic
Nutritional Supplements (category node 3774071, reached via Health & Household >
Vitamins, Minerals & Supplements > Digestive Supplements > Probiotics), captured
on 09/26/2026. The capture is committed at
captures/amazon-probiotics-bestsellers-top30-2026-09-26.png. This list is fixed
at capture and not revised as rankings change afterward.
A product is included only if the digestive/gut-health benefit is the first-listed claim in its product title or primary bullet. Products whose first-listed claim is vaginal/urinary tract health, oral/dental health, or an unrelated function (e.g., pre-alcohol support) are excluded, even if digestive support is mentioned secondarily. This test is applied to every product in the category ranking, in rank order, without exception.
What "first-listed" means. The test asks whether the product is presented as a gut-health product, not how a marketing string is punctuated. Where a title opens with a compound benefit phrase naming several systems together — "Immune, Digestive & Gut Health" — digestive health counts as first-listed. A product is excluded only where the first benefit named is wholly unrelated to the gut and digestive support appears later as a secondary claim. The listed exclusions are the guide: vaginal or urinary tract health, oral or dental health, and unrelated functions such as pre-alcohol support are products that are fundamentally about something else, which is what this test is for.
Ruling on women's multi-system probiotics (v1.9, 2026-09-26). Where a product names gut health first inside a benefit string but the product line itself is built around another system, the exclusion test governs and the product is excluded. Where gut health is named first and the product is not specific to another system, the literal first-listed test governs and the product is included. Applied to the 09/26 sample: rank 7 (vH essentials Probiotics for Women) is excluded, since the line is a feminine-health line and gut appears only inside a later benefit string; rank 18 (Doctor's Recipes Women's Probiotic) is included, since digestive is the first benefit named and the product is not vaginal-specific. Both were carried as undecided until this ruling rather than being resolved silently.
Ruling on listings with no health claim in the title or primary bullet (v1.10, 2026-09-26). The inclusion test reads the title first, then the primary bullet. Where neither carries any health claim, the test reads down the bullets in order and applies to the first bullet that names a health benefit. A bullet about manufacturing, format, certification or guarantee is not a health claim and is skipped. Applied to the 09/26 sample: rank 20 (NOW Probiotic-10) passes on its primary bullet, which names the human GI tract and no competing system; rank 25 (NewRhythm) has a title of format and quality attributes only and a primary bullet claiming manufacture in New York, so the test reads down to its third bullet, which opens "Digestive Health and Immune Health", and the product passes. This extends the rule rather than restating it: the v1.7 text assumed the primary bullet would carry a benefit phrase, as it did at rank 4.
What "top 30" counts. The sample is the first 30 rows of the category ranking as captured, examined in rank order. Exclusions are not backfilled from further down the ranking: a captured boundary is only fixed if it does not move with the number of products that fail the test. The scored set will therefore be smaller than 30.
The captured ranking is recorded in sample.csv, one row per product
in rank order, carrying the first-listed claim the test was applied to, the
include/exclude decision, and the reason for every exclusion. Excluded
products stay in the file: a sample is only auditable if the rejections are
visible alongside the acceptances. Without this file the claim that the list
was fixed at capture cannot be checked by anyone, including its author.
Note on the pilot sample (added 2026-09-26). Methodology v1.7 stated a sample captured on 09/05/2026. The selection was made from the live category page on that date, but the ranking screenshot was never saved. No such file exists in this repository or anywhere on the author's machine. By the standard set in the paragraph above, a sample whose capture was not retained cannot be checked, so it is not published as one. The sample was re-captured on 09/26/2026 and that capture is the sample of record. The three products scored during the pilot are all present in it, at ranks 1, 4 and 15, and all pass the inclusion test, so their scores carry forward unchanged. Their label capture dates remain 09/07/2026, recorded separately per product.
Ruling on products scored from outside the captured sample (v1.14,
2026-10-26). A product that is not in the captured ranking can be scored under
the identical rubric, and some are worth scoring: a reader choosing between a
best seller and a direct-to-consumer brand that never appears in an Amazon
category is choosing between two labels, and this audit can read both. What such
a product cannot do is enter pilot-scores.csv.
The reason is that the ranked file is the scored subset of a sample fixed at capture, and every count, mean and "highest score" computed over it is a statement about that sample. Admitting a product selected on any other basis — reader demand, a brand's prominence, the author's interest — would make those figures describe a set assembled after the fact while still being presented as results from a fixed ranking. The selection rule and the aggregate have to match or the aggregate means nothing.
Off-sample rows therefore live in offsample-scores.csv, sharing the ranked
file's header: the same physical separation the conflict-of-interest appendix
uses, for the same reason, so that no total computed over the ranked data can
include one without a deliberate choice to combine two files. An off-sample row
may back a product page and a head-to-head comparison, because each of those is
a statement about one or two specific labels rather than about a sample. It may
never be counted in a ranking, a mean, a "products scored" figure, or the
highest-score claim. Each row records why the product was added, and every other
rule — live-label verification, dated captures, the criterion rulings above —
applies to it unchanged.
Ritual Synbiotic+ is the first such row. It is scored because Seed DS-01, at rank 4 of the captured sample, is the comparison a gut-health shopper reaches for most often, and Ritual is the product on the other side of it; Ritual sells direct and so cannot appear in an Amazon category ranking at all.
Second category: fiber supplements (scored 2026-11-19). The top 30
products in Amazon Best Sellers in Fiber Supplements, captured on 11/19/2026, are
listed in fiber-sample.csv; the capture, which shows the top 38, is committed at
captures/amazon-fiber-bestsellers-top38-2026-11-19.png, and only the first 30 are used. The same first-listed
claim test applies, with regularity or digestive health as the gut-health claim.
Fiber results are reported on their own and are never combined with the probiotic
sample in any total, mean or ranking, because the two lists come from different
categories and were captured on different dates.
Ruling on duplicate formulas (2026-11-19). A best-seller list often ranks the same formula several times: different sizes, counts or flavors of one product. Each distinct formula is scored once, at the rank of its highest-placed listing. A later listing whose label panel matches it ingredient for ingredient and amount for amount is recorded in the sample file as a duplicate, with the rank of the listing it duplicates, and is not scored again or counted a second time in any total. A listing whose panel differs in any ingredient or amount (a sugar-free version, an added ingredient) is a separate formula and is scored on its own. Scoring duplicates separately would let one product's label count several times in a category average just because it sells in several sizes.
Ruling on fiber amounts (v1.17, 2026-11-19). Most fiber labels print no ingredient weight. They state "Dietary Fiber 6 g" on the panel and name the fiber in the ingredient list. Five rules govern how the fiber sample is read:
- One fiber source. Where a single fiber ingredient supplies all the fiber, the dietary fiber line is read as that ingredient's disclosed amount for criteria 1 and 4. Pectin, agar and gelatin listed in a gummy after the declared fiber are read as gelling agents, not actives.
- Several fiber sources, no split. Where two or more fiber ingredients share one fiber line with no per-ingredient amount, the amounts are undisclosed: criterion 1 scores 0, as it would for a blend total, and any of those fibers with a reference range scores 0 on criterion 4, as the prebiotic blends in the probiotic sample did.
- Daily dose. The dose compared is the daily amount the label directs for the gut outcome it claims. Where directions allow a range ("2 to 3 gummies", "up to 3 times daily"), the product scores 1 if any directed daily amount falls inside the reference range, under the benefit-of-the-doubt rule; this matters because several reference rows are single doses, which a larger amount can overshoot. Directions given for a different outcome, such as appetite control, are not used. Where directions are read from listing text rather than the captured panel, the row says so.
- Nothing checkable. Where every active in a product is excluded from criterion 4 because no qualifying trial exists, criterion 4 scores 0. This extends the v1.11 reasoning beyond probiotics: the exclusion removes the comparison, it does not award the point. No probiotic-sample score changes, because only Florastor holds the point there and it has a range.
- No panel. A listing whose Supplement Facts panel cannot be located on the retail listing or the brand site is recorded in the sample file as not scored, with the search, rather than scored on what it omits.
The objection to rule 1 is recorded openly. Fiber grams are not ingredient weight: psyllium husk is roughly 85 to 90 percent fiber, and psyllium trials dose husk weight, so reading the fiber line understates a psyllium product's dose by about a tenth. Two rows, the Metamucil 4-in-1 powders (sugar-free and real sugar), reach 9 g/d of fiber, just under the 10 g/d psyllium floor, and would sit at about 10.2 g/d on the husk weight P&G states for its OTC version; both are flagged in their notes. The two psyllium capsule rows reach 8 g/d of fiber and stay below the floor on either reading. The answer is that the panels do not state husk weight, and the audit scores what the label states. A brand can close the gap by printing it, as Konsyl's single-ingredient serving effectively does.
Data sources
Primary: the NIH Office of Dietary Supplements Dietary Supplement Label Database (DSLD).
Important limitations of DSLD, stated openly:
- Manufacturer submission is voluntary, so coverage is incomplete.
- The database contains both on-market and off-market products.
- NIH does not verify that label information conforms to FDA requirements.
Accordingly, every scored product is verified against its current live label, captured by screenshot with a capture date recorded. Where the live label differs from the DSLD record, the live label governs and the discrepancy is noted in the dataset.
Products that cannot be verified against a current label are excluded from scoring.
Label captures are stored in captures/, named
<brand>-<product>-<view>-<YYYY-MM-DD>.png, where <view> is panel
for the Supplement Facts panel or listing for the product page carrying
the claims that criterion 4 is scored against. Both views are captured for
each product, and the same date is recorded in the capture_date field of
the row that cites them. A scored row without a
corresponding capture is incomplete, not merely undocumented.