Protocol / methodology v1.3
The ranking pipeline
Six auditable stagesEach active provider receives the same item type, research question, and initial list size from 1 to 50.
OpenAI retains every unique candidate, normalizes names, merges only clear duplicates, and logs every merge or removal.
Every model scores the complete shared field in three fresh calls. Each result includes up to five terse reasons.
For each item, every provider’s middle score becomes its official model score.
Provider medians combine into the AI Consensus Score. The requested top N becomes official; a shorter field is published in full.
OpenAI turns the score evidence into one direct explanation per result. Optional top-ten Spotlight Insights surface one strong, well-established detail without changing the ranking.
Calculation layer
Equal model weightProvider score =
median of samples 1, 2, and 3
AI Consensus Score =
median of active provider medians
Using the median at both levels limits the influence of one unusual sample and one unusually generous or harsh model.
Shared scoring language
Any subjectSystem limits
Read before interpretingModels use internal knowledge only. Results can be outdated, incomplete, or wrong.
The ranking captures the collective judgment of the active model panel.
A high score means the panel judged an item strongly against the exact research question.
Changing a provider or model version may change the answer. Each run names its exact panel.
Research intent repeats across years, but panels and shortlist sizes may change and are disclosed.
