2DSD: confidence from evidence that keeps arriving

1Two stages, one stream of evidence

On a Space Observer trial the question is whether there are more green or more orange aliens. In the model, the observer keeps a running total of evidence for "green" over "orange". Every few milliseconds the total moves a little: on average in the direction of the true answer, by an amount that grows with the colour difference, but with a lot of random jitter on top.

x(t + dt) = x(t) + v · dt + σ · √dt · ε,   v = k · s

Here s is the colour difference (n_green − n_orange) / 68, k converts it into a drift rate v, and ε is fresh standard normal noise at every step. The total starts at z (zero for an unbiased observer).

Stage 1, the choice. As soon as the total reaches +θ the observer answers "green", and as soon as it reaches −θ, "orange". This is Ratcliff's diffusion model, and the time it takes is the decision time.

Stage 2, the confidence. The process does not stop at the choice. Evidence keeps arriving for an extra period τ, and confidence is read off the total at the end of it: the further the total has moved in the direction of the choice, the higher the confidence. The confidence criteria cut that final value into levels, just as in signal detection theory.

Run several trials and look at where stage 2 begins. Every path that chose "green" starts its second stage at exactly +θ, whether it got there in 0.3 seconds or in 3, straight up or after wandering. Reaching the threshold wipes out the history. This one property decides almost everything that follows, including why the static version in Part A is not an approximation.

A1Why time can be dropped exactly

Stage 2 always starts at the threshold, so the confidence evidence does not depend on how long stage 1 took. What remains are two closed-form pieces.

The choice. The probability of hitting +θ before −θ is a classic result for the diffusion process (with σ = 1):

P(green | s) = [1 − e^{−2v(θ + z)}] / [1 − e^−4vθ]

For an unbiased start (z = 0) this simplifies to 1 / (1 + e^−2vθ): a logistic psychometric function, where SDT gives a probit one.

The confidence evidence. Write e for the final total, signed so that positive means "in favour of the option chosen". It starts at θ and then gains |v|τ on average if the choice was right, or loses |v|τ if it was wrong, plus noise with variance τ:

e | correct ~ Normal(θ + |v|τ, τ)    e | error ~ Normal(θ − |v|τ, τ)

The confidence level is the interval between criteria g₁ < g₂ < g₃ that e falls into (fitted as g₁ and the gaps g₂ − g₁ and g₃ − g₂). Decision time appears in neither formula, so integrating it out costs nothing: as long as there is no trial-to-trial variability (Part B), these are exactly the full model's predictions for choices and confidence. A simulation of the full diffusion with 100,000 trials matches them to within about 0.002 in every response cell.

A2The parameters we fitted

Three adaptations separate our fit from the paper. The paper fits a separate drift for each difficulty level, while we let the drift grow linearly with the colour difference (v = k · s), because the staircase produces many levels with few trials each. The paper fits separate criteria for each response, while we share one set, which saves three parameters with 80 trials. And the paper's experiments used 4-point ratings (or continuous ratings binned into 6 levels), while we cut each person's 0–100 ratings into four quantile bins. One further difference is only cosmetic: the paper writes the thresholds as ±θ and its code runs the diffusion between 0 and 1, while we use −1 and +1. That rescales the drift, τ and the criteria (their drift is twice ours, their τ a quarter of ours) without changing any prediction.

A3What the model predicts for one display

The figure shows where the total ends up after stage 2, with trials answered "green" above the line and trials answered "orange" below it, each scaled by how often that answer is given. Green answers start stage 2 at +θ and orange answers at −θ, and both then drift in the direction of the true majority. Each answer has its own confidence levels: for a green answer e is the total itself, for an orange answer it is the total with its sign flipped, so the shared criteria appear mirrored around the midpoint. The bars show the resulting share of trials in each of the eight possible responses.

Four things to try. Set τ to 0: every trial ends exactly on +θ or −θ, all in the same confidence level, and confidence says nothing at all about accuracy. Then raise τ slowly: both clouds spread and drift towards the true majority, the separation grows as 2|v|√τ, and so does meta-I. Watch the choice accuracy while you move τ: it never changes, because τ acts only after the choice. Finally, flip the difference to the orange side and move z: a bias changes how often each answer is given, but not where each answer's cloud ends up, because stage 2 always starts on the threshold.

A4Across difficulty

Because the mean of e is θ + |v|τ after a correct choice and θ − |v|τ after an error, easier displays push the two apart. Confidence on correct trials rises with the colour difference, and confidence on errors falls: the folded X pattern. The accuracy curve on the left does not move at all when you change τ.

A5How this differs from SDT

What sets 2DSD apart is easiest to see by comparing what confidence gets to read in three related models.

This has three consequences. With no time after the choice, SDT's confidence still tracks accuracy, while 2DSD's carries no information at all. After the choice, 2DSD's total can swing back past the midpoint towards the option that wasn't chosen (e < 0), most often after an error: a change of mind in all but name, which our shared criteria put in the lowest level. And in the terms of Dayan's metacognitive information theory, 2DSD's rater has no copy of the actor's evidence (B → ∞) and only post-decisional information, with noise G = 1 / (|v|√τ).

At the fitted values the two models end up close. At the staircase level, with both kinds of display mixed as in the task, the fitted 2DSD's confidence carries 0.056 bits of information about accuracy, against 0.064 bits for the fitted SDT model. A τ of about 0.56 instead of 0.48 would close the gap. Two models that make confidence this similarly informative are hard to tell apart with 80 trials per person, which fits what the model comparison found: SDT's estimated frequency was 70% (161 of 200 participants assigned to it), 2DSD's 6.5% (13 of 200).

One caution about τ in our data: in the first parameter-recovery run (REC_SOURCE = "fitted"), tau_post did not recover (r ≈ −0.05). That run drew its true values from shrunk HBI estimates, which also flattened every other model's recovery, so it is not conclusive. Check τ in the population-based recovery (REC_SOURCE = "population") before reading anything into individual τ values.

B1What the full model adds

Response times also fix what the static version could not. Thresholds and drift trade off in choices alone, but they shape decision times differently, so θ can be estimated instead of fixed. One quantity still sets the scale; here it is σ = 1 per second. Pleskac and Busemeyer write the thresholds as θ_A and −θ_B, which may differ; a shifted starting point, as here, does the same job. Implementations such as the R package dynConfiR also let the non-decision time vary.

B2Decision times and confidence together

The left chart shows when responses happen: correct responses above the line, errors below it, each scaled by how often it occurs. The right chart shows the mean confidence level of responses made at each moment.

Work through the presets in order.

No variability. With an unbiased start, correct and wrong responses have exactly the same response-time distribution, and the confidence lines are flat: confidence does not depend on how long the decision took. Both follow from the threshold erasing the history. Moving the starting point z breaks the first property but not the second.

Drift variability. Now some trials have weak drift and some strong. Weak-drift trials are slow, more often wrong, and they also gain less evidence after the choice. So errors become slower than correct responses, and confidence falls with response time on both correct and wrong trials. Within one difficulty level, this is how the full 2DSD produces the familiar finding that slow decisions come with low confidence, and it needs drift variability to do so. Across difficulty levels the same finding comes for free: harder displays are both slower and less confident. Drift variability also produces confident errors: on trials where the drift happens to point the wrong way, the evidence after the choice keeps supporting the wrong answer, so mean confidence on errors rises.

Start-point variability. Starts close to a threshold give fast responses, and fast errors. Confidence does not care: stage 2 still begins at the threshold, so the confidence lines stay flat.

Speed and accuracy emphasis. A lower threshold gives faster, less accurate responses. It also gives lower confidence when the criteria stay put, because every trial starts stage 2 closer to the midpoint. A higher threshold does the opposite.

B3What the static version keeps and loses

It keeps the full model's predictions for choices and confidence exactly, as long as there is no trial-to-trial variability. Integrating out decision time is then not an approximation, since nothing that confidence depends on depends on time.

It loses three things. First, the response times themselves, which separate the threshold from the drift and give τ a real time scale. Second, the trial-to-trial variability: the static formulas assume a single drift for each difficulty level, so they cannot express the slow errors and the falling confidence-time relation above, nor the way drift variability reshapes the confidence distribution. Third, the link between τ and observable confidence response times.

For Space Observer the loss is smaller than it sounds. The colour display lasts only 250 ms, and the choice is made afterwards on a separate screen, so response times are not a clean record of evidence accumulation. For the same reason, "evidence arriving after the choice" cannot come from the screen itself: in our fit, τ is best read as how much extra information confidence receives, not as a duration.

B4Known limits

2DSD predicts only the folded X: confidence in errors falls as the stimulus gets easier. Without trial variability this follows directly from the formulas above, and it held in every setting with drift and start-point variability that we tried. Some data sets show confidence in errors rising with stimulus strength instead. Hellmann, Zehetleitner and Rausch (2023) found that 2DSD cannot produce this, while a model with an extra visibility process (dynWEV) can; in their two experiments dynWEV fitted best and 2DSD came second.

Two further limits concern time. With drift variability, 2DSD gets the direction of the confidence and response-time relation right but predicts a weaker relation than observed (Calder-Travis et al., 2024). And with a fixed τ it predicts the same confidence response time for every confidence level, whereas people report fastest the rating they use most often (Chen & Rahnev, 2023).

In Shekhar and Rahnev's static comparison, 2DSD did not come out on top in any of their three experiments: LogN fitted best in two of them and PE in the third.