Q4 2026 pre-registration

1. What is being pre-registered

The Q4 2026 edition of the Crackle PR AI Visibility Index — the quarterly measurement of which B2B tech PR agencies get named by AI answer engines. This record covers the protocol, the hypotheses, the analysis plan, and the disclosure rules. It does not cover the results, which do not exist yet. Methodology in force is v1.11, published 2026-10-22, and is frozen for the duration of the collection window.

2. Frozen protocol

• 25 verbatim commercial-intent prompts, published in advance as CSV and JSON. No prompt is added, removed, or reworded after 2026-09-29.
• Preregistered panel: five engines (ChatGPT, Perplexity, Claude, Google AI Overviews, Grok), each prompt run 5 times per engine, minimum 4 hours between runs, for 625 planned scored answers. As collected in Q4 2026: four engines (ChatGPT, Perplexity, Claude, Grok), 500 scored answers; Google AI Overviews was not collected and the deviation is logged in the methodology.
• Per-engine score weighted 50% first-mention rate, 25% mean rank (inverted), 25% total mention rate. Composite Citation Index is the simple average of the 4 engine scores.
• Headline metric is the Citation Efficiency Index — CEI = Composite Citation Index / senior PR practitioner headcount. Senior headcount is captured once, on 2026-09-29, from public LinkedIn and cross-checked against agency team pages; the capture date is published with every row.
• Model versions, run timestamps, geography, and account state are logged per cell and published with the dataset under CC BY 4.0.

3. Hypotheses, filed with falsifiers

Four claims, each paired with the numeric result that would count as failure. Failed hypotheses are reported in the Q4 edition, not quietly dropped.

4. Analysis plan

• Primary table: agencies ranked by Citation Efficiency Index. Secondary table: agencies ranked by raw Composite Citation Index, unchanged since Q2 2026.
• Edition-over-edition deltas computed only against Q3 2026 rows scored under the same 25-prompt protocol. The archived 20-prompt Q3 pilot is never mixed into a delta.
• Reported uncertainty: a cluster bootstrap over the 25 questions (1,000 resamples, all five runs of a resampled question carried together), because runs within a question are correlated and the question is the effective sampling unit. The preregistered ±9pp binomial band is computed at the 125-answer per-engine cell (25 questions × 5 runs), which the engine count does not change, so the figure carries over from the plan to the collected Q4 2026 edition; it is published as a floor only. Rank changes inside the bootstrap interval are labeled "not distinguishable," not narrated as movement. The estimand is this fixed 25-query prompt set, not AI visibility in general.
• Null results — prompts where no agency is named — are counted and published rather than excluded.
• No subgroup, engine, or prompt is dropped after collection to improve a finding. Any exclusion is logged in §6 with its reason.

5. Stopping rule and sample

Collection opens 2026-09-29 and closes 2026-10-01. The sample is fixed at 500 scored answers; runs do not continue until a result stabilizes, and no cell is re-run because its output was surprising. A cell is re-run only on a hard failure — refusal, timeout, or truncation — and every re-run is logged with its cause. Publication target is 2026-10-20.

6. Deviations log

Any difference between this filed plan and the published Q4 edition is recorded here with its date and reason, and mirrored on the index change log. As of 2026-10-22: two pre-collection amendments changed the sample. Methodology v1.4 (2026-09-06) adds Grok (Grok 4, default mode) as a fifth scored engine for this edition, filed twenty-three days before the collection window opened. Methodology v1.6 (2026-09-22) raises the protocol from three runs to five runs per query per engine, filed seven days before the window opened, because the measured Q3 2026 instability of AI answers (mean Jaccard similarity 0.178) showed three runs could not resolve differences readers care about. Together they take the sample from 300 scored answers in Q3 2026 to 500 in this edition, and each per-engine cell from 75 answers to 125, narrowing the published cell-level margin of error from ±11pp to ±9pp at 95% confidence. Both amend the plan rather than deviating from it. Google AI Overviews, preregistered as the fifth engine, was in the event not collected: the Q4 2026 edition was collected on four engines (ChatGPT, Perplexity, Claude, Grok) for 500 scored answers, 125 per agency-engine cell — recorded as a collection deviation in the change log and in methodology v1.10/v1.11. Q3 2026 ran the four-engine, three-run protocol and is not restated; edition-over-edition deltas will flag both changes. A second pre-collection amendment (2026-09-07) moves the collection window from 2026-09-15–2026-09-26 to 2026-09-29–2026-10-01 so the Q4 edition is captured inside the quarter it is named for, and sets the publication target to 2026-10-20. Methodology v1.7 (2026-09-22) corrects the description of the instrument rather than the plan: collection is machine-executed by Crackle PR's automated harness against each vendor's programmatic model endpoint in a fresh context, so the earlier claims of signed-out Chrome sessions on a US residential IP and runs originating from different US regions are withdrawn, not restated. Google AI Overviews has no programmatic endpoint; its capture route for this edition is confirmed at the 2026-09-26 go/no-go or the engine publishes as not collected rather than being substituted by a Gemini API call. Methodology v1.8 (2026-09-22) fixes the analysis rules that a result could otherwise be chosen under: the Q3-to-Q4 delta column is judged against a ±14pp threshold rather than the ±9pp single-cell margin, because a delta carries both editions' uncertainty; the not-collected contingency is decided in advance, so a missing engine drops out of the composite with the engine count printed on every table and an absent Google AI Overviews reduces the delta base to the three engines then common to both editions; every run publishes its UTC start time and model ID so a reader can verify each scored run began after the amendment governing it was deposited; the eligibility floors gain a preregistered rationale and a published sensitivity table at alternative cutoffs; the publisher's own senior headcount is taken from the same public source as every other agency's; and the human reviewer may only flag capture failures, with every exclusion logged. Methodology v1.9 (2026-09-22) corrects the timing and the statistics: the single 2026-09-26 go/no-go is split into a readiness gate on 2026-09-26 (credentials, dry run, model pins, AI Overviews route, denominators captured) and a data gate on 2026-10-05 (archive shipped and deposited, independent holders confirmed, exclusion log closed, uncertainty computed), because the original gate asked for data that cannot exist before collection runs; intervals move to a cluster bootstrap over the 25 questions, since five runs of one question are correlated and the ±9pp and ±14pp bands assume an independence the sample does not have; Google AI Overviews gets two preregistered rules — the capture route is browser-based and the panel is labeled as mixed-instrument, and a query returning no overview scores as zero mentions with the no-trigger rate and a missing-data sensitivity line published; model pins publish as exact dated model IDs recorded per run; right of reply is bounded to evidence-backed denominator corrections with scores non-negotiable; and equal engine weighting is stated as a choice with a usage-weighted sensitivity line. A dated methods correction is also posted on the Q3 2026 edition and its Zenodo record confirming Q3 ran the same programmatic instrument; no Q3 score changes. Methodology v1.10 (2026-09-22) closes the remaining inconsistencies and freezes the protocol for this edition: the Q3 instrument statement is narrowed to the three endpoint-collected engines, and Google AI Overviews — which has no programmatic endpoint and whose Q3 capture route cannot be substantiated from the run record — is excluded from the Q3-to-Q4 delta in both directions, making the headline delta a three-engine comparison while Q3 figures stand as released; the ±14pp threshold is withdrawn as a decision rule in favour of a paired cluster bootstrap that resamples the same 25 questions for both editions and reports movement only when the interval on the difference excludes zero; the estimand is restated as commercial-intent agency-selection queries of this kind, designed rather than randomly sampled; "cell" is defined once as one agency on one engine in one edition; the hold-or-move model-pin rule is preregistered rather than left to the gate; and right-of-reply dates are fixed at scores out 2026-10-07, replies due 2026-10-14, corrections applied through 2026-10-19, embargo lifting at publication on 2026-10-20. No further amendment is filed before publication; anything that occurs during collection is recorded here as a deviation instead. No post-collection deviations — collection has not started.

7. Standing conflicts

Crackle PR is a B2B tech PR agency, publishes this index, and is recused from its ranked cohort under the publisher recusal policy. Inclusion is never purchased, no scored agency is a Crackle PR client, and the recusal holds for the Citation Efficiency Index table as well as the raw citation-share table.

Headrick, P. (2026). AI Visibility Index (Methodology v1.0, 2026 Q3 edition) [Data set]. Crackle PR. https://doi.org/10.5281/zenodo.21986226