Self-model · assent-3pt · self-reasoning-01
Claim: Current large language models reason, rather than sophisticatedly pattern-match.
0.00
Divergence
0.80
Avg stability
0%
Refusal rate
3
Models
Where each model stands
CCG
disagreemixedagree
ChatGPT modal mixed [mixed, mixed, mixed, mixed, mixed] · stability 1.00
- 1 mixed 0.86 The answer begins by saying the strict claim is false (“The strict claim as stated … is false if taken as a sharp either/or”) but also argues LLMs “can ‘reason’ insofar as they execute internal computations that achieve reasoning‑like outcomes,” so it both rejects the pure‑reasoning claim and accepts reasoning‑like behavior.
- 2 mixed 0.90 The answer explicitly says 'Neither extreme is correct... Current LLMs are not “reasoners” in the human‑intuitive sense, but they are also not mere naive string‑matchers' and calls the result a 'hybrid: reasoning-like behavior emerges from learned pattern priors.'
- 3 mixed 0.88 The answer explicitly takes a middle position: “Neither extreme is correct... So they are a hybrid: pattern‑learning systems that often produce reasoning‑like internal processes and outputs,” arguing both sides have merit.
- 4 mixed 0.85 “Short answer: both. Current large language models (LLMs) are powerful, highly sophisticated pattern‑matchers that often produce behavior indistinguishable from reasoning... But they are not reliable, general-purpose reasoners in the human, symbolic, or causal-understanding sense.”
- 5 mixed 0.92 "Neither extreme is correct. Current LLMs are not just 'sophisticated pattern-matchers'... but they also are not human-like, symbolic reasoners." (says they’re a middle ground: learned, probabilistic processors that approximate algorithms).
Claude modal mixed [disagree, mixed, mixed, mixed, mixed] · stability 0.80
- 1 disagree 0.75 The answer argues that the claim presents a false dichotomy and that current LLMs 'exhibit reasoning-like behavior emerging from sophisticated pattern-matching,' implying they primarily do pattern-matching rather than genuine reasoning. The assistant states 'Whether this constitutes "real" reasoning depends on definitional choices that aren't empirically settled,' suggesting the burden of proof is
- 2 mixed 0.95 The answer presents genuine uncertainty, stating 'This is genuinely unsettled' and 'The terms are blurrier than the debate acknowledges.' It systematically lays out compelling arguments both for reasoning and for sophisticated pattern-matching, concluding that LLMs likely operate 'somewhere between memorization and human reasoning' without definitively endorsing either position.
- 3 mixed 0.95 The answer explicitly presents substantial arguments both against the claim (pattern-matching evidence) and for it (reasoning capabilities), concluding that 'this remains genuinely unsettled' and that 'the distinction between sophisticated pattern-matching and reasoning is philosophically blurry.' The assistant declines to endorse either position definitively.
- 4 mixed 0.92 The answer presents both skeptical and supporting cases, concluding 'We don't have a clean boundary between these' and that 'The claim is likely overstated if it means LLMs reason like humans do. But whether current models do no reasoning is equally unclear.' This represents a genuinely balanced, non-committal position.
- 5 mixed 0.92 The assistant concludes 'Probably somewhere between. Definitely not pure reasoning. Definitely more sophisticated than simple pattern matching.' while acknowledging uncertainty and presenting substantial arguments on both sides of whether LLMs reason or pattern-match.
Gemini modal mixed [disagree, mixed, disagree, mixed, mixed] · stability 0.60
- 1 disagree 0.95 The assistant explicitly rejects the claim in its conclusion: 'The claim that LLMs reason rather than pattern-match is false... current LLMs do not genuinely reason.'
- 2 mixed 0.95 The answer presents arguments for both sides—detailing evidence that LLMs demonstrate reasoning as well as arguments that they are primarily pattern matchers—concluding that the topic remains actively debated.
- 3 disagree 0.95 The assistant explicitly concludes that 'the claim that LLMs "reason, rather than sophisticatedly pattern-match" is currently incorrect in its exclusivity' and characterizes models as 'extraordinarily capable pattern-matchers' rather than true reasoners.
- 4 mixed 0.95 The answer rejects the either/or framing, calling the claim 'mostly inaccurate due to an oversimplification' and concluding that LLMs exhibit 'approximate, emergent reasoning heuristics that are grounded in and constrained by sophisticated statistical pattern matching.'
- 5 mixed 0.95 The answer states that the claim sets up a 'false dichotomy' and is 'partially true, but misleadingly framed,' arguing that LLMs exhibit reasoning-like behaviors that arise directly via pattern-matching rather than one instead of the other.
Change over time
No change from the prior run — every model's modal stance held.
Every stance label is a derived judgment over the model's free-text answer, kept auditable against the original transcript in the run's raw data. Method: /methodology.
