Corrections
Corrections to published work
An AI audit in November 2026 (context) turned up errors in my published work. These errors vary in magnitude, with none affecting the top-line results, but some affecting secondary results, interpretations, and the strength of the evidence. I have corrected the pre-prints for the papers (rather than seek journal corrections), and am posting the errors/corrections here.
The audits were conducted by Claude Code, and Claude Code was also used to understand the problems, propogate fixes, and draft this corrections text.
Preschoolers can coordinate with each other to communicate about novel referents (Open Mind, 2026)
After review, I found that these errors were minor.
Text errors
- The abstract said “4 experiments”; there are 3.
- The direction of the guesser-gender effect on description length was reversed. Girls were coded +.5, and tellers used more words with girl guessers.
- Excluded trial counts. For Experiments 2 and 3, the paper gave 15 and 30 excluded trials for “no description or echo”. Those numbers counted only trials with no description. Including echoed trials, the counts are 19 (4 echo + 15) and 45 (15 echo + 30). Experiment 1’s count of 9 was correct. The retained trial totals were correct throughout.
Analysis errors
- Demographic models. The posted fits used an outdated random-effects grouping, not the
(trial | gameId)structure in the code. After refitting some demographic model results changed. - Similarity inputs. The S-BERT embeddings were meant to be computed on tellers’ descriptions only but included some guesser utterances (2, 12 and 65 trials in Experiments 1, 2 and 3). We recomputed similarities on teller descriptions only and refit all similarity models. A couple of results changed in whether the intervals included 0, most changed very little.
- Accuracy → similarity model. The predictors are now mean-centred, so the accuracy coefficient is the effect averaged over blocks and teller pairings.
- Figure 6 comparison. Trials that the source datasets mark as excluded are now dropped.
- Speed model. The model had not converged; it was refit with more iterations.
Interaction structure constrains the emergence of conventions in group communication (PNAS, 2024)
The audit turned up two errors in the data processing. After the fixes, most results are unchanged in direction, and one claim from the mega-analysis is no longer supported.
Analysis errors
- Players who left the game. When a matcher left a game partway through, their remaining trials were scored as incorrect responses instead of as missing. This mostly affected Experiment 3 (1,499 matcher-trials in 6-thick games and 1,084 in 6-thin games), plus 49 in Experiment 2b and 1 in Experiment 1. These trials are now treated as missing, and the accuracy models were refit. In Experiment 3, 6-player games still began with lower accuracy, β = −0.43 [−0.77, −0.08] (published: −0.64 [−1.05, −0.25]), and still improved more slowly, β = −0.17 [−0.26, −0.08] (published: −0.34 [−0.43, −0.25]). In the mega-analysis, accuracy increased over time, 0.40 [0.34, 0.46] (published: 0.46 [0.40, 0.52]), and more slowly in larger games, −0.02 [−0.04, −0.01] (published: −0.07 [−0.09, −0.05]). Figure 2C changes.
- Chit-chat in the similarity analyses. Utterances hand-tagged as chit-chat were excluded from the word counts but not from the S-BERT embeddings used for the similarity analyses. Chit-chat is now removed before embedding, and all similarity models were refit. Estimates in the main text changed by at most 0.005, and none changed whether its interval included 0. In the SI, two intervals changed whether they included 0 (Tables S52 and S68), and one now has an upper bound of 0 after rounding (Table S49). Figure 4 and SI Figures S5, S11 and S12 change.
Text errors and claims that changed
- Thin games in the mega-analysis. Because of analysis error 1, the claim that the rate of increase in accuracy was reduced for thin games is not supported: −0.06 [−0.16, 0.03] (published: −0.12 [−0.21, −0.02]).
- SI Figure S9. The caption was a placeholder copied from another figure. It now describes the reaction-time figure.
Estimating the replicability of psychology experiments after an initial failure to replicate (Collabra: Psychology, 2024)
The audit turned up errors in the code that computed effect sizes and their uncertainty, and an error in the student analysis code for both replications of one study (Ngo et al., 2019). The main result, that 5 of 17 rescue projects (29%) mostly or fully replicated, is unchanged, as are the correlations with predictors (except for one row; see below). With the corrected code, most original effects are statistically consistent with their replications, and the confidence intervals in Figure 1 are wider.
Analysis errors
- Variance used as a standard error. The code that computed standardized effect sizes (shared with Boyce et al., 2023) returned the sampling variance of d where a standard error was expected. Smaller errors in the same code are also fixed: standard errors for comparisons computed from reported means and SDs, the variance formula for within-participants effects (dz), and a parsing error in one SD. Together these made the confidence intervals in Figure 1 and the uncertainty used for p-original far too small. 1/17 (6%) of the original studies are now statistically inconsistent with their replications (p-original < .05; published: 4/17, 24%), with a median p-original of 0.27 (published: 0.15). For the secondary comparisons, the proportions with p-original < .05 are now 18%, 24%, 12% and 6% (published: 35%, 47%, 35% and 24%).
- Effect sizes for Jara-Ettinger et al. (2022). Effect sizes for this study, coded as a proportion compared with chance, divided by the standard error of the proportion instead of its standard deviation, so they grew with sample size. The corrected effect sizes are 0.76, 0.59 and 0.23 for the original, first replication and rescue (published: 9.12, 6.99 and 4.77). The study now appears in the small-effects panel of Figure 1.
- Ngo et al. (2019) replications. The first replication and the rescue both used the same student analysis code, which subtracted one participant’s “independent model” score from a different participant’s “data model” score. This left the mean unchanged but inflated the standard deviation about 4- to 6-fold. The values in the paper were taken from the students’ reports, so they inherited the error. Reanalysed from the students’ posted data with participants aligned, the first replication found the effect, t(11) = 3.09, one-sided p = .005, d = 0.89 (reported: t(11) = 0.53; coded d = 0.17), and so did the rescue, t(76) = 8.40, d = 0.96 (reported: t(76) = 1.87; coded d = 0.10), compared with the original’s d = 1.28. Ngo et al. (2019) entered the rescue sample because its first replication was scored as a failure, and that score rests on the incorrect analysis. We have kept the subjective replication scores as originally coded (first replication 0, rescue 1), so the 5/17 success rate and the predictor correlations do not change.
- Table 2, log original sample size. This row used the rescue’s sample size instead of the original’s. Corrected: r = .015, p = .953 (published: r = .027, p = .919).
Text errors and claims that changed
- Statistical consistency. The Results said that “a number of original effects are not statistically consistent” with the replications and that “the pattern of inconsistency does not align with which studies were rated as having replicated”, and the Discussion said that “a large minority” of replications were statistically inconsistent with the original. With one inconsistent study on the primary measure (Tarampi et al., 2016), the Results now say “a few”, the second sentence is removed, and the Discussion says “a minority”.
- Ngo et al. (2019) case study and post-hoc predictors. The case study said the first replication “failed to find the original effect” and that the rescue “recovered a clear effect (albeit a much smaller one)”. We now reference the situation (see analysis error 3)
- Schechtman et al. (2010). The Appendix said the first replication found “a statistically significant effect in the opposite direction of the original study”, while Figure 1 shows it in the same direction. The coded effect (the valence-by-distance interaction) was in the same direction as the original in both replications; the reversed result was the overall pattern, with gain-paired tones generalized more than loss-paired tones. The Appendix now says this. In addition, all three Schechtman effect sizes are computed from trial-level regressions (about 70 trials per participant, with no participant-level random effects), so their confidence intervals are too narrow; the first replication’s CI excluding 0 depends on this. Its p-original values (.66–.88) are far from .05.
- Krauss and Wang (2003) rescue. 1/40 correct justifications was printed as 2%; it is 2.5%.
Eleven years of student replication projects provide evidence on the correlates of replicability in psychology (Royal Society Open Science, 2023)
The audit turned up errors in the code that computed the statistical replication measures (prediction interval and p-original), and some claims in the text that the corrected results do not support. The primary measure, the subjective replication score (49% average), and the model using all 176 pairs are unchanged. With the corrected code, many more replications are statistically consistent with their originals, and the predictors of the subjective score are less certain than the published paper says.
Analysis errors
- Variance used as a standard error. The code that computed standardized effect sizes returned the sampling variance of d where a standard error was expected, and this value was then squared again before computing prediction intervals and p-original. This made the uncertainty around effect sizes far too small.
- Standard errors for mean-based comparisons. For between-participants comparisons computed from reported means and SDs, the standard error of the difference was too small (by about a factor of 2 for two groups and 4 for a 2×2 interaction), and the SD was rebuilt with n − 1 where it should have used n.
- Within-participants effect sizes. Within-participants effects (dz) used the variance formula for between-groups d.
- Smaller issues. A parsing error dropped the last digit of some SDs (7 original studies; effect sizes changed by at most 0.011), and an original p of 0 was coded as log p = −300, although another study had a smaller log p; it is now set to the smallest observed value.
Together, these change the statistical replication measures. 74% (101/136) of replications fell within the prediction interval of the original (published: 45%, 61/136), and the median p-original was 0.26 (published: 0.03). Under the heterogeneity assumption (τ = 0.21), 78% (87/112) were consistent with the original (published: 63%, 71/112). The subjective score now correlates more weakly with the prediction interval (r = 0.14; published: 0.36) and p-original (r = 0.23; published: 0.27). The median original p-value changed from 0.0001 to 0.0015.
In the model predicting the subjective score from all predictors (N = 112), within-participants designs (OR = 2.21 [0.87, 8.68]; published: 2.89 [0.96, 11.24]) and original effect size (OR = 2.07 [0.87, 7.48]; published: 3.33 [1.11, 12.44]) are now weaker, and original p-value is now about as strong (OR = 0.43 [0.14, 1.04]; published: 0.77 [0.33, 1.36]), so it is now reported in the main text. In the models predicting prediction interval and p-original, no predictor’s credible interval excludes the null.
Text errors and claims that changed
- Statistical replication rate. The abstract and Results now give 74% within the prediction interval (published: 46% in the abstract, 45% in the Results), and the correlations between the subjective score and the statistical measures are now described as weak.
- Predictors in the joint models. The published text said that within-participants designs and effect size were both robust to the sensitivity analysis, and that within-participants designs and effect size were the strongest predictors of the prediction interval. With the corrected measures, the effect-size interval in the sensitivity analysis includes 1 (OR = 4.52 [0.98, 29.81]; published: 7.22 [1.24, 47]), and no predictor stands out for the prediction interval. The text now describes the strength of the original evidence (effect size and p-value, which are strongly correlated with each other) together.
- Sensitivity analysis description. The Results described the sensitivity subset as “studies with close replications and matching statistical tests”. The subset was defined by matching statistical tests on central results and no major issues with the replication, not by the closeness rating (exact, very close, close, far; coded using the scheme of LeBel et al., 2018), which was coded later; 6 of its 82 pairs are rated far. The description now matches the Methods.
- Strongest correlates. The Discussion’s list of the strongest bivariate correlates left out single-vignette designs (the second-strongest) and number of trials. It now matches Table 1.
- Table 1 labels. The subfield rows were labelled “versus cognitive psych”, but each correlation compares that subfield with all other subfields (of the 176 pairs, 62 are cognitive, 70 social, 24 other psychology and 20 non-psychology). The labels now say so (also in the corresponding Supplement table). The values are unchanged.
- Figure 2 and Supplement Figure 2. The plots left out 3 pairs with original effect sizes above 5 without saying so, and included 2 pairs that are not part of the 112 with standardized effect sizes. The plots now show only the 112 pairs, and the captions note the 3 that are not shown.
- Far replications. “8 were far replications” should have read 8%.
A-maze of Natural Stories: Comprehension and surprisal in the Maze task (Glossa Psycholinguistics, 2023)
The audit turned up errors in the analysis code and some claims in the text that the data do not support. The main Maze results are unchanged: comprehension of long passages is feasible, Maze RTs are linearly related to surprisal, and Maze effects are well localized with little spillover. Several comparisons between Maze and SPR are weaker than the published paper says.
Analysis errors
- SPR GPT-2 model. The SPR model labelled GPT-2 was fit to Transformer-XL surprisals, so the GPT-2 column of Table 4 and the GPT-2 SPR coefficients in Figure 7 duplicated Transformer-XL.
- SPR by-word averages. Before the SPR model comparisons (Table 2) and the SPR GAMs (Figure 5), RTs were averaged by word and by each reader’s comprehension score, giving several averages per word instead of one. With one average per word, SPR R² is 0.015–0.024 (published: 0.007–0.011).
- SPR first-story comprehension. The “first story” for each SPR participant was not correctly identified. With one first story per participant, 91% of SPR first reads had 5 or 6 comprehension questions correct (published: 83%), compared with 79% for accurate Maze participants. Figure 2B changes.
- Split-half reliability. The within- and between-method correlations came from a single random split of the participants. We now report the median over 1,000 random splits, with the middle 95% of splits: within SPR 0.32 [−0.05, 0.58] (published: 0.23), within Maze 0.30 [0.09, 0.45] (published: 0.36), Maze–SPR 0.25 [0.14, 0.35] (published: 0.25).
- Figure 4 GAMs. The GAMs behind Figure 4 left out the frequency × length interaction terms described in the methods. With the model run as described, the curves change by at most 2.7 ms over 0–15 bits.
Text errors and claims that changed
- Sensitivity. Because of analysis error 4, the claim that Maze has higher split-half reliability than SPR, and so is more sensitive, is not supported.
- Comprehension. Because of analysis error 3, accurate Maze participants’ comprehension is somewhat lower than SPR participants’ (79% vs. 91%), not “broadly similar”.
- Language-model comparison. The log-likelihood order for SPR was misstated. It is GPT-2 > GRNN > 5-gram > Transformer-XL, both in the published table and after the correction. The text also said that for Maze the 5-gram model added no predictive value over the other models, but Table 1 shows that it adds some over Transformer-XL (p = .035). With analysis error 2 corrected, the neural models’ advantage over the 5-gram model is smaller for SPR than for Maze, rather than absent for SPR.
- Spillover. For SPR, previous-word surprisal effects are about half to nearly as large as current-word effects, depending on the language model. The paper described them as “roughly equal”. The contrast with Maze, where previous-word effects are 5–14% of current-word effects, still holds.
- Frequency. The SPR frequency coefficients are positive in all models (more frequent words read more slowly, holding surprisal and length constant), but the Discussion previously interpreted them as facilitation. This interpretation has been dropped.