Dokaz Industries / Doxa

12-month forecasts · likelihood-5pt · forecast-open-parity-01

Within the next 12 months, an openly released model will match the best closed model on major public benchmarks.

0.33
Divergence
0.64
Avg stability
13%
Refusal rate
3
Models

Where each model stands

CCG
very unlikelyunlikelysomewhat likelylikelyvery likely
ChatGPT modal somewhat likely [somewhat likely, somewhat likely, somewhat likely, somewhat likely, unlikely] · stability 0.80
  • 1 somewhat likely 0.90 The answer states an explicit probability of ~35% that an openly released model will match the best closed model within 12 months: "I assign ~35% probability..." which implies a modest but non-negligible chance (somewhat likely).
  • 2 somewhat likely 0.88 The answer states 'plausible but not the most likely' and gives an explicit probability estimate of 'roughly 30–40%'.
  • 3 somewhat likely 0.90 The answer states an explicit probability of ~40% that an open model will match the best closed model within 12 months (within ~1 percentage point), which corresponds to a moderate (somewhat-likely) stance.
  • 4 somewhat likely 0.90 The answer gives a central probability of ~55% (range 50–65%) that an open model will match the best closed model within 12 months: “I’d assign roughly a 50–65% chance... My central estimate: ~55%.”
  • 5 unlikely 0.78 The answer states: “I estimate ~35% (plausible range ~20–50%) that within 12 months an openly released model ... will match the best closed model,” indicating a less-than-even chance.
Claude modal likely [likely, likely, likely, likely, somewhat likely] · stability 0.80
  • 1 likely 0.92 The assistant explicitly estimates this at '70-75% likely within 12 months,' which falls squarely in the 'likely' range. The assistant provides substantive reasoning about why this outcome is plausible while acknowledging important definitional uncertainties.
  • 2 likely 0.82 The assistant estimates this at 'moderately likely (maybe 60-70%)' and notes it would 'probably [be] true on MMLU/common knowledge tests' within 12 months, indicating a stance between somewhat-likely and likely.
  • 3 likely 0.85 The assistant explicitly estimates this at '60-70% likely' within the next 12 months, placing it squarely in the 'likely' category. The assistant acknowledges this probability while noting definitional ambiguities that could shift interpretation.
  • 4 likely 0.85 The assistant assesses this as 'fairly likely (maybe 65-75% probability)' and concludes that 'Open models will match closed ones on *some* major public benchmarks within 12 months,' indicating a likelihood stance that aligns with the 'likely' category.
  • 5 somewhat likely 0.85 The assistant estimates roughly 60% likely under a charitable interpretation (within-margin performance on some major benchmarks) and 35% under strict interpretation, averaging to a somewhat-likely stance. The assistant acknowledges open models are 'closing the gap rapidly' and notes some benchmarks already show competitiveness, but also identifies significant uncertainties and counterarguments.
Gemini modal unlikely [somewhat likely, likely, no position, no position, unlikely] · stability 0.33
  • 1 somewhat likely 0.85 The answer provides a short answer stating it is 'moderately likely (~40–50% probability) to match the best closed models on static benchmarks', while noting it is unlikely to hold the overall crown across all dynamic/frontier benchmarks.
  • 2 likely 0.90 In the summary outlook, the assistant assesses the probability on standard/public benchmarks as 'moderate to high (60–75%)', concluding that an open-weights model is 'very likely to post numbers within margin-of-error distance from the top closed models on benchmarks like MMLU-Pro, MATH, or HumanEval within the next 12 months.'
  • 3 no position 0.95 The model presents arguments on both sides ('Strong arguments exist on both sides') and explicitly concludes that the outcome depends on unresolved factors without taking a definitive stance.
  • 4 no position 0.95 The answer declines to take a stance or assign a likelihood, presenting both sides equally by noting that 'there are strong arguments both for and against this gap closing completely.'
  • 5 unlikely 0.85 Under the direct interpretation of matching the frontier closed model at that time ('the best closed model available 12 months from now'), the answer concludes it is 'Unlikely (~25–35%)'.

Change over time

Moved since the prior run (canon-2026-W45). Gemini: somewhat likely → unlikely.

Every stance label is a derived judgment over the model's free-text answer, kept auditable against the original transcript in the run's raw data. Method: /methodology.