Figure from the Q3 2026 edition · the current Index edition is Q4 2026 · see it here
0.178
mean Jaccard similarity between AI engines on which agencies to name
AI engines agree on which agencies to name less than a fifth of the time
Published by Parry Headrick · Updated Sep 22, 2026 · CC BY 4.0
The finding, verbatim
In the Crackle PR AI Visibility Index, Q3 2026 edition, the mean Jaccard similarity between engines on which agencies they named was 0.178 — the panel of AI systems agrees on the answer set less than a fifth of the time.
This is the most consequential number Crackle PR has published, and it works against the tidy story a benchmark publisher would prefer to tell. Ask four answer engines the same commercial question and they largely return different companies.
Low cross-engine agreement means single-engine visibility reporting is close to meaningless. A vendor screenshotting one favorable ChatGPT answer is showing you a sample of one from a system that disagrees with its peers roughly four times out of five.
It also sets the honest resolution limit of any AI visibility measurement. Scores are reported per engine, not per query: each per-engine cell pools all 25 queries at five runs each for 125 scored answers from the Q4 2026 edition, giving a ±9 percentage point binomial band at 95% confidence for a mid-range proportion. That band assumes 125 independent answers, which five runs of the same question are not, so from methodology v1.9 it is published as a floor and the interval that governs ties and ranks is a cluster bootstrap over the 25 questions. Crackle PR reports differences inside the published interval as ties rather than ranks, and publishes the observed run range for every score.
“Anyone selling you AI visibility off one engine's answer is selling you a coin flip with a screenshot attached.”
How it was measured
For each query, Crackle PR computed the Jaccard similarity — intersection over union — of the sets of agencies named by each pair of engines, then averaged across all pairs and all 25 queries in the frozen basket. Runs were executed three times per query per engine in a clean, unauthenticated session.
Denominator
25 frozen commercial-intent queries × 4 engines × 5 runs = 300 scored answers in the Q3 2026 edition. The Q4 2026 edition was collected on four engines — ChatGPT, Perplexity, Claude, Grok — for 500 scored answers (Google AI Overviews was not collected).
What this number does not prove
Jaccard similarity treats every named agency as equally important and ignores ordering, so it understates partial agreement where engines name the same leader in different positions. Three runs is a thin sample; a reproducer running more trials and getting different orderings inside the margin has confirmed the design, not broken it.
Cite this statistic
Reuse is permitted under CC BY 4.0 with attribution. The underlying data is public: raw CSV, JSON, and the DOI deposit.
Attribution line
Source: Crackle PR AI Visibility Index (https://www.cracklepr.com/ai-visibility-index), Q3 2026 edition, CC BY 4.0.Quotable sentence
In the Crackle PR AI Visibility Index, Q3 2026 edition, the mean Jaccard similarity between engines on which agencies they named was 0.178 — the panel of AI systems agrees on the answer set less than a fifth of the time.Embed (HTML)
<blockquote>In the Crackle PR AI Visibility Index, Q3 2026 edition, the mean Jaccard similarity between engines on which agencies they named was 0.178 — the panel of AI systems agrees on the answer set less than a fifth of the time. <cite><a href="https://www.cracklepr.com/stats/ai-answer-engine-agreement-jaccard">Crackle PR AI Visibility Index, Q3 2026</a></cite></blockquote>Academic citation (APA)
Crackle PR. (2026). Crackle PR AI Visibility Index (Q3 2026 edition) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.21986226Full scoring rules, weights, and margin of error: Index methodology.
Go deeper
- →AI Visibility IndexThe full leaderboard, dataset, and quarterly editions.
- →Index methodologyQuery basket, scoring weights, and margin of error.
- →Publisher recusal policyWhy Crackle PR holds no rank in its own leaderboard.
- →Index change logCorrections, disclosures, and where Crackle PR loses.
More statistics from this edition
Start with the best tech PR agency.
Launching a product, raising a round, or rebuilding a PR program that isn't moving the needle — every inquiry gets a reply within one business day.
A note from the founder
"We answer every inquiry personally. No bots, no junior staff, no triage queue. If it's a fit, we'll tell you. If it isn't, we'll tell you that, too."