🌏 中文版
This is part 13 of the series "Writing Articles That Teach," in the third section, "What readers bring with them." Prior knowledge in general will get its own part; this one is about an especially awkward kind of it: things the reader already believes that are wrong. The Chinese edition of Part 1 used a word closer to "myth" and this part's Chinese edition uses one closer to "misconception"; both mean the same thing, and this English edition says "misconception" throughout, as the research mostly does.
Part 1 already covered the average result. Write out a common misconception, say plainly that it is wrong, then give the correct explanation: in two meta-analyses, this kind of refutation text beat text that only gave the correct version. That part also noted that the two meta-analyses share many of the same studies, and listed several caveats: neither tested which way of writing works better, the estimate shrinks once possibly unpublished studies are accounted for, the texts were mostly short, and most studies were not in real teaching settings. This part does not repeat the average. It asks three things Part 1 did not answer:
- Who it helps. Does it work the same for readers who hold the misconception and readers who already understand?
- Whether it plants the error. Can writing out a false claim make someone who never thought of it remember it instead?
- How to write it. Where should the false claim and the correct claim go, should either be repeated, and when should you leave the misconception out altogether?
The trigger was this site's writing spec for course guides. The spec asks concept guides to point out common misconceptions (my translation of the Chinese spec), and the part on questions and feedback suggested adding one line: write out the wrong idea, then explain why it is wrong. In practice I get stuck in three places. First, I do not actually know whether readers hold a given misconception; this site has no reader data at all. Second, writing out a misconception sometimes means inventing an error the reader never had. Third, this site's own old drafts stated rules wrong, and the 2026-10-13 review reports recorded several cases.
How the sources were chosen, and their biases. The source list came from a scan of the literature, followed by picking studies to read closely; for each question I deliberately included at least one study with an opposite or null result. In the end I read two refutation-text meta-analyses, two experiments that measured how many misconceptions readers held, a methods review and a replication experiment from misinformation research, a short commentary on when prior knowledge hurts, a theoretical paper that treats misconceptions as material to refine, and a website cataloging programming misconceptions. The biases: English-language literature, science and math topics, college students or adults, and studies with legally available full texts. Misinformation research is about public issues such as vaccines and politics, not teaching articles; I flag that gap wherever I use it. Every paper in the references was read in full, with an extraction record. For the programming catalog I read only the about page and six entry pages, and I only skimmed the reference list of Smith and colleagues; the reference entries say so in parentheses.
The short version
| Question | Evidence | How far it goes |
|---|---|---|
| Does refutation text still work if the reader already understands? | Danielson and colleagues' meta-analysis: no measured difference in effect whether or not studies screened out participants who already answered correctly. Schroeder and Kucera's meta-analysis says the primary studies reported too little to analyze this | No measured difference, but the authors say power may have been too low |
| Do more misconceptions hurt understanding more? | A 92-person experiment: in the control group (standard text plus a list of the misconceptions), more misconceptions went with poorer comprehension (p = .050); no such relation in the refutation-text or think-sheet groups; nothing was significant 6 to 8 days later | Consistent direction, most results at the edge of significance |
| Does it only work for people who hold the misconception? | An experiment with 336 teachers and student teachers: no measured difference in the full sample; among the 96 who held both misconceptions at pretest, personalized refutation text did better | One experiment, short texts, a value-laden topic |
| Does repeating a misconception make people believe it more? | A misinformation methods review: few studies report this backfire, and most of those used single-item measures. A vaccine-myth replication: no group believed the myths more than the control group | For people who have already heard the misconception, no robust evidence at the group level |
| If the reader never heard the misconception, does writing it out plant it? | The methods review calls this a legitimate concern that has not been tested; the teacher experiment above offers an after-the-fact explanation | No study has tested it directly |
| Where do the false claim and the correct claim go? | The methods review recommends pairing them prominently and not letting the misconception stand alone in a headline. In the replication, the group shown myths labeled "myth" with no facts did not differ significantly from control | The advice comes from news and public health; applying it to article headings is my inference |
| Does a misconception have to be "knocked down"? | Smith, diSessa, and Roschelle, a theoretical paper: most common misconceptions are useful knowledge stretched beyond the range where it works | A position, no experiment |
Who it helps: whether the reader holds the misconception
Refutation text starts from the assumption that the reader arrives with a wrong idea. If the reader already understands, an extra passage stating the false claim might be useless or even harmful. Three kinds of data bear on this: moderators in the meta-analyses, an experiment that measured how many misconceptions each reader held, and an experiment that gave different text depending on a reader's pretest.
Meta-analysis: screening out those who already answered correctly made no measured difference
Danielson and colleagues' 2024 meta-analysis (71 articles, 76 studies) treated whether a study excluded participants whose beliefs were already accurate as a confirmatory moderator, expecting larger effects in screened studies. The two kinds of studies showed no significant difference, so the hypothesis was not supported.
Read this result together with the authors' own caveats. None of the 26 moderators in the analysis was significant at the .05 level. The authors say that because the moderators did not explain the variation between studies, they could not definitively answer under what circumstances refutation texts work; they also note that some moderator analyses had too few studies and may have lacked power. They do say when it is worth targeting a misconception: when it is especially harmful, causes confusion, or gets in the way of understanding a concept. They add that when background knowledge is limited or absent, building accurate knowledge is a worthy goal in itself.
Schroeder and Kucera's 2022 meta-analysis (33 studies, 44 comparisons) cannot answer this question. In 37 of the 44 comparisons, the prior knowledge of participants was not reported, and the authors write that "how and to what degree learners' preconceptions or misconceptions impact learning outcomes when using refutation text is still an open question." Their practical advice carries a precondition: "a common misconception must be known to write a refutation text."
Number of misconceptions and comprehension: three ways of presenting misconceptions
Anja Prinz, Stefanie Golke, and Jörg Wittwer's 2021 experiment did not manipulate misconceptions. It first measured how many misconceptions about covariance each participant held (0 to 4), then looked at how that number related to comprehension after reading. Ninety-two German university students who had taken a statistics course were split into three groups, each with 12 minutes of study time: the refutation-text group read for 12 minutes; the other two groups first looked at a sheet for 2 minutes, then read for 10.
- Refutation text: each misconception stated in one sentence, one sentence saying it is wrong, then three sentences of correct information.
- Think sheet: read the standard text while filling in a three-column table. The first two columns were printed with questions and the typical misconceptions; readers filled in the correct explanation in the third.
- Control: read the standard text, but first received a sheet listing the four misconceptions.
The results come in three layers:
- In all three groups, misconceptions dropped after reading; instructional method had no main effect.
- In the control group, the more misconceptions a person held, the poorer their comprehension (p = .050, which the authors call marginally significant). In the refutation-text and think-sheet groups, the number of misconceptions was unrelated to comprehension. Across all three groups combined, the correlation between misconceptions and comprehension was close to zero.
- At a retest 6 to 8 days later, none of these relations was significant. In the refutation-text group, misconceptions rose significantly from the immediate test to the delayed one; they were still numerically fewer than in the other two groups, but at the delayed test the three groups showed no measured difference in misconceptions.
This experiment is easy to read as showing that, when reading standard text, people with more misconceptions understand less. But its control group did not read just standard text; it read standard text plus a list of the misconceptions. So it does not answer whether adding a common-misconceptions passage beats leaving it out. It compares three ways of presenting misconceptions, and the slope difference between the refutation-text group and the control group (p = .063) did not reach significance; only the think-sheet group did. In the discussion, the authors say that simply giving a list of misconceptions before the text with the correct information may already be effective.
Personalized refutation won only in the subsample that held the misconceptions
Anna-Sophia Dersch, Ian Thacker, and Alexander Eitel's 2026 experiment used each reader's pretest to decide what they got. 336 in-service teachers and student teachers in Germany first answered a misconception scale on math and gender stereotypes, then were randomly assigned to four groups (2×2): a standard expository text; the same text preceded by a refutation ("Many people think that … However, these assumptions are incorrect and not in line with the scientific consensus."); the same text preceded by personalized feedback based on the pretest (saying whether the reader's answers matched the scientific evidence); or both. Each of the two expository texts was under 200 German words, and the refutation passages were shorter; the posttest came right after reading, with no delayed test.
- Full sample of 336: none of the three pre-specified comparisons was significant. Plain refutation text did not beat plain expository text either.
- Only the 96 who held both misconceptions at pretest: personalized refutation text beat the other three groups (p = .044, explaining about 5% of the variance); plain refutation text still did not beat expository text.
The authors' guess is that no effect showed up in the full sample because people without the misconceptions reacted to the texts differently. Their practical advice: pair personalized feedback with a judgment, giving refutation text only to readers who actually hold the misconception and expository text to everyone else.
The topic here is a value-laden belief about gender, the participants are teachers, and the texts are very short. All three are gaps when applying it to a "common misconceptions" passage in an ordinary teaching article.
When prior knowledge hurts: a framework
Garvin Brod's 2021 commentary proposes a framework: prior knowledge reliably helps only when it is activated, relevant to the task at hand, and congruent with what is to be learned. When it is activated and relevant but incongruent with the new material, it mostly hinders learning; the bigger the reorganization required, the harder the learning. Brod also notes an exception: highly incongruent new information can be learned well when it strongly surprises the learner.
This is a commentary with no new data. The supporting examples are lab memory tasks such as word lists and matching names to faces, and the author also says the three factors do not map one-to-one onto learning success. What it offers this part is a reason misconceptions deserve separate treatment: they are the kind of prior knowledge that is activated, relevant, and incongruent.
Whether it plants the error
The worry that "writing out the false claim will hurt the reader" goes by the name backfire effect in misinformation research. Briony Swire-Thompson, Joseph DeGutis, and David Lazer's 2020 methods review splits it into two kinds. A worldview backfire effect is when a correction challenges someone's belief system and they defend their position by believing more. A familiarity backfire effect is when a correction repeats the false claim, making it more familiar, and familiar things tend to be taken as true. My judgment is that the second kind is the main worry for teaching articles.
The review splits "does it plant the error" into two questions, and I follow its split.
People who have already heard the misconception: no robust evidence at the group level
The review's authors list the familiarity-backfire literature study by study: 3 of 11 articles reported a backfire effect, all three on vaccines and all with single-item measures, and one of the three unpublished. Taking both kinds of backfire together, 81% of the backfire effects found in the review came from single-item measures. The authors point out that single items are unreliable and prone to regression to the mean, and that this may be particularly problematic for people with low pre-correction belief in pre-post designs, where it could produce spurious backfire. Their conclusion is that backfire effects are "not a robust empirical phenomenon." They also report a meta-analysis of 10 studies: belief updating did not differ significantly depending on whether the correction repeated the false claim.
Ullrich Ecker, Caitlin Sharkey, and Briony Swire-Thompson's 2023 experiment replicated one of those three. 380 adults in the UK were split into five groups. One group read 10 vaccine myths, each paired with a fact; another read only the same 10 myths, each labeled "myth" with no fact, which the authors call the condition theoretically most likely to produce a familiarity backfire. Tested immediately and a week later, no group believed the myths more than the control group at any time point. The myth-only group did not backfire either; the authors call this "particularly strong evidence against the notion of familiarity backfire."
Both papers add a caveat: unnecessary repetition should still be avoided, because it may raise familiarity with no benefit. Ecker and colleagues also write that false claims with little traction and low risk of harm are sometimes better left uncorrected.
Readers who never heard the misconception: no study has tested it directly
All the evidence above comes from settings where readers had mostly heard the claim already. For someone writing a teaching article, the more pressing case is different: the article lists a false claim the reader never thought of.
Swire-Thompson and colleagues leave this open. They say that publicly correcting a misconception few people believe may spread the false claim to people who did not know it, without the benefit of reducing belief; they call this a "legitimate concern" that still needs empirical testing. Ecker and colleagues' experiment did not manipulate whether participants had heard the myths before, so it cannot answer either. Danielson and colleagues list the same possibility in their introduction (a reader who did not hold the misconception learns it from the refutation text), and their discussion says they "found no evidence of widespread backfire effects." That sentence is inferred from the overall positive effect; the paper did not run a separate test for backfire. They also list as not yet evaluated whether a text with several refutation statements leads readers to remember the misconception and double down on it.
The closest data come from the Dersch experiment. Of the 336 participants, 12.5% had higher misconception scores after reading than before (the paper says they "backfired"); analyzing only this group, the personalized refutation group's scores got worse by the most. Most of the people whose scores got worse held only one misconception or none at pretest. The authors offered familiarity backfire as an explanation after the fact, and say plainly that it was not among their preregistered hypotheses and that the existing evidence is mixed (the paper they cite is Ecker and colleagues' failed replication). Their abstract is stronger than their results section: it says personalized refutation text "triggered the most backfire among those who did not" hold misconceptions, but no analysis took people without misconceptions at pretest as the population and compared the four groups.
Three things to know when reading this result:
- The paper does not report how many of the people whose scores got worse (42 of them) were in each of the four groups. What was compared is how much worse their scores got, not how many people. Reading from the figure, those whose scores got worse in the plain expository group also dropped by about 1 point on average.
- They were selected by outcome, not compared across randomly assigned groups. Each score combines whether the person agreed or disagreed with how certain they were, so someone already at the bottom of the scale (near −4) at pretest can only stay level or rise. This resembles the regression to the mean that Swire-Thompson and colleagues describe, but the analogy is weaker: each of Dersch's scales has 4 items, not the single-item measures that review mainly criticizes.
- The discussion says the misconception was mentioned twice in the personalized refutation condition, once in the feedback and once in the refutation passage. But in the full materials in the appendix, all three kinds of personalized feedback (no misconception, uncertain, misconception) mention only the topic of the questions, not the content of the misconception.
All three points are things I noticed reading the paper, including the full materials in the appendix; the authors do not discuss the last two.
So whether writing out a misconception the reader does not hold leads them to learn something wrong has not been tested directly. That does not mean there is no risk, and it does not mean there is one.
How to write it: leads from the research
Part 1 already noted that neither meta-analysis tested what makes something a refutation text. Most of the leads below come from misinformation research and single experiments; applying them to teaching articles is inference, and I mark each one.
Keep the false claim and the correct claim together
One of Swire-Thompson and colleagues' practical recommendations applies directly to writing: always pair the false claim with the correction clearly and prominently. Their example is news headlines. Many people read only the headline, so do not put the misconception in the headline with the correction in small print below; put the correction in the headline itself, even if it is only a prominent "myth" label.
Applying this to article headings is my inference. Article pages on this site have a table of contents fixed in the sidebar that lists every second- and third-level heading, so headings really do appear apart from the body, which is close to the "headline only" situation. There is also a difference: a news reader may never get to the correction at all, while in an article the correction comes right under the heading. Following the original advice, a heading can contain the false claim as long as the correction appears with it: "A tiny learning rate is not the safe choice," or a heading marked "Common misconception." The original says nothing about questions; treating a question heading such as "Is a smaller learning rate always safer?" as "stating only the false claim" is also my inference.
Give the correct claim, not just "this is wrong"
In Ecker and colleagues' experiment, the group that read each myth paired with a fact showed less concern about side effects and less belief that vaccines cause autism than the control group right after reading. The group shown myths labeled "myth" with no facts did not differ significantly from control (and did not backfire). The authors' reading is that a clear, detailed alternative account may be the most important ingredient of a correction. Note that these two groups were not compared with each other directly; each was compared with the control group.
A limitation to carry along: a week later, even the myths-with-facts group's advantage over control was no longer significant under the multiple-comparison adjustment the authors had specified in advance, and before that adjustment the p-values of the myths-with-facts group and the myth-only group look similar. The abstract says the condition reduced vaccine misconceptions without giving a time point.
Say it once, in one place
Ecker and colleagues and Swire-Thompson and colleagues both argue that restating the false claim in a correction largely need not be avoided and may even help, and both add that unnecessary repetition should still be avoided. Danielson and colleagues list the risk of multiple refutation statements as not yet studied. Putting these together, my practice is to have each misconception appear in only one section: the heading can state it together with the correction, and the body states it once and corrects it once; the summary, the tldr, the conclusion, and other sections do not repeat the false claim. The heading counts as part of that section, because the previous recommendation asks precisely for the false claim and the correction to appear together in the heading. This is inference; no study has compared numbers of repetitions.
Misconception before or after the correct explanation, and how many per article
I found no study to cite. Swire-Thompson and colleagues mention two common recommendations: the "truth sandwich" (fact, falsehood, fact) has not been tested, and preliminary results from a conference presentation suggest that stating the misconception first and then two factual statements may work better than the truth sandwich, which the authors say is not yet enough to recommend. I have not read the conference presentation.
Let readers fill in the correct claim themselves
In Prinz and colleagues' think-sheet group, readers wrote the correct explanation next to each misconception themselves. Comprehension showed no measured difference from the refutation-text group. Some other outcomes did differ; for example, the think-sheet group's confidence judgments about whether each answer was right were more accurate (an after-the-fact comparison). The authors list as limitations that they do not know why think sheets work, and that they did not test having learners first write down their own prior ideas.
In an article, the closest equivalent is to turn the misconception into a question: "Decide whether this statement is true before reading on." This is my extrapolation; the experiment was in a lab with participants required to fill in the table, and readers of an article can skip it.
A model from programming: a catalog of misconceptions
Programming Misconceptions is a catalog maintained by programming education researchers including Luca Chiodini and Matthias Hauswirth. As of 2026-10-14 it had 104 public entries across four languages: Java, Python, Scratch, and JavaScript. It is a reference for programming teachers, not a study, and nobody has tested whether writing these entries for learners helps. I read its about page, its field statistics page, and six entry pages; several things about how it writes are worth borrowing:
- The wrong claim and the right claim each in one sentence, side by side. For example, "Assignment copies the object" versus "Assignment copies the reference to the object." Every entry has both.
- A short piece of code that lets readers see the error for themselves. AssignmentCopiesObject uses two assignment lines plus
id()printing the same number. - Where the misconception comes from. The same entry says teachers may explain that an assignment "copies the number contained in the variable," but variables store references, and the difference becomes crucial once students meet mutable objects. In other words, some misconceptions come from earlier simplified explanations.
- How to tell a reader holds it, and how to build on it. AssignmentCopiesObject treats the misconception as a starting point for discussing how "everything is an object" in Python; another entry, VariablesHoldObjects, uses it to start discussing two variables that refer to the same object.
Its scope, stated plainly: of the 104 public entries, 55 have a literature field and 49 list no literature at all; the Origin, Symptoms, and Value fields (where it comes from, how to tell, how to build on it) are each filled in for only about half the entries. The site itself does not describe these entries as "common." In a teacher survey it summarizes, 51 of 97 teachers said they had never seen IfIsLoop (believing the body of an if statement runs repeatedly while the condition holds) in their students. These survey figures are the site's summary; I did not check them against the original paper.
Another view: misconceptions as material to refine
The research above assumes a misconception is something to be replaced. John Smith, Andrea diSessa, and Jeremy Roschelle's 1993 theoretical paper (I read the authors' fourth draft) takes a different view:
Most, if not all commonly reported misconceptions represent knowledge that is functional but has been extended beyond its productive range of application.
Their example is fractions. Several elementary school students who had just learned fractions judged 4/6 larger than 6/9 because "6ths are larger than 9ths." The rule itself is right (a smaller denominator means larger parts); they just had not restricted it to cases where the numerators are equal. Proficient middle and high school students used the same rule but only when the numerators were equal, and some extended it to numerators that were roughly equal. So the authors argue that learning is mainly a matter of adjusting where existing knowledge applies and reorganizing its role in the whole knowledge system, not swapping it out one for one.
The paper is easy to read as saying you should not directly refute students' wrong ideas. The original is more nuanced:
- The authors accept the empirical findings of misconceptions research, saying it has shown that students' ideas fail in many situations; what they object to is treating misconceptions only as mistakes. They also acknowledge that some flawed ideas play no role at all in expert reasoning.
- A section heading says instruction that confronts misconceptions is "misguided and unlikely to succeed," while the body text says something weaker, that it "is not the most promising pedagogy"; neither is backed by data. The three cases are a classroom description, a summary of an unpublished dissertation (10 novices, 11 proficient students), and an argument about expert reasoning in physics (history of science, a reinterpretation of two think-aloud excerpts published by someone else, and a thought example). None compares two ways of teaching.
- They are talking about classroom discussion. Nowhere do they discuss how to write text materials, and they do not discuss refutation-text experiments. So the paper cannot be said to "oppose" those experimental results; the two are at different levels.
What it offers someone writing a teaching article is a reminder that a "common misconceptions" passage can be written another way: besides explaining where the idea goes wrong, also say when it is right. No study has compared the two approaches.
This site's example: where the materials themselves were wrong
This site's seven-week guides to CS50 AI went through an independent review on 2026-10-13. The project content the old drafts got wrong (for example, the rule for who wins Nim in week 4 was reversed), how each week was confirmed, and the fact that the review reports have no reader data are already covered in the part on code. Here I add only three things that relate to readers coming away with wrong ideas.
The largest group of must-fix items was saying more than the source. Across the series there were 25 must-fix items. I grouped them myself (the reports did not), and the largest group, with 7 items, was: attributing to the lecturer what the source did not say, stating something more strongly than the source, or making a claim with no basis. Examples include presenting the article's own inference as something the lecturer said, and turning a source's "one of the first" into a claim that it was the first. Readers would take these as what the lecturer or the source meant.
The official materials also have spots readers can misread. In week 4, the lecturer calls drawing a line to classify data "linear regression," while defining regression elsewhere as predicting a real value. In week 2, the probabilities the lecturer reads out (31%, 23%, 64%) differ from what you get by hand with the enumeration method the lecture teaches (30.0%, 23.9%, 63.4%). The reviewer judged that the library uses an approximate algorithm but did not run the version used at the time of the lecture. A reader who works it out by hand might think they made a mistake; that is my inference.
The common-mistakes part of the project sections is this site's version of a common-misconceptions passage. One of the examples in the rewrite spec (written in Chinese) is that the player who takes the last one loses, which is the same Nim rule the old week-4 draft got backwards: something this site once got wrong became an entry warning readers. These entries come from review and actual runs, not from readers, and none of the "common misconceptions" in this site's articles was measured in readers.
Where the sources differ
The following results point in different directions, or look contradictory while actually measuring different things. I list them side by side without taking a side.
Is restating the false claim harmful? Ecker and colleagues and Swire-Thompson and colleagues consider the worry about restating it largely unwarranted, and say restating may even help. Danielson and colleagues, discussing future research, write that the literature also suggests repeating misinformation can be harmful, since readers may remember the misinformation without remembering that it is wrong. Dersch and colleagues explain their own after-the-fact result with familiarity backfire. The first two rest on misinformation about public issues and belief measures; Dersch's comes from teachers reading short texts, and it is an after-the-fact explanation.
Does plain refutation text beat expository text? On average across both meta-analyses, yes. In the Dersch experiment, plain refutation text did not beat expository text in the full sample or in the subsample holding the misconceptions. In the Prinz experiment, refutation text, think sheet, and expository text plus a list of misconceptions showed no measured difference in comprehension. In neither of these two experiments was the comparison condition plain expository text alone: Dersch's four groups all read the same expository text, with the refutation only a short passage in front; Prinz's control group also saw the misconceptions.
Should you write only for readers who hold the misconception? In Danielson and colleagues' meta-analysis, screening out those who already answered correctly made no measured difference. In the Dersch experiment, personalized refutation text won only among people holding both misconceptions. The former is a cross-study moderator analysis whose authors say power may have been too low; the latter is a subsample analysis within one experiment.
Is "this is wrong" enough? Swire-Thompson and colleagues report a 2019 study by Ecker and colleagues: simply tagging misinformation as false, without explaining why, substantially reduced belief. In Ecker and colleagues' 2023 experiment, the group shown myths labeled "myth" with no facts had no significant effect relative to control. I have not read the 2019 study.
Is direct refutation good teaching? Smith and colleagues say confronting misconceptions head-on is not the most promising pedagogy, with no data. The refutation texts in both meta-analyses do exactly that: state the misconception and say it is wrong. The two are talking about different things: Smith and colleagues about classroom confrontation that pits the student's idea against the expert's, the meta-analyses about how a short text is written.
Checking this site's practices
The comparison below uses the versions in the working directory as of 2026-10-14. What research supports is marked "basis"; what this site decided on its own is marked "this site's choice."
| What this site does now | Reason | What research can say |
|---|---|---|
Course guide spec course-guide-contract.md (in Chinese): concept guides should point out common misconceptions | The spec gives no reason; none of the entries in its design-basis section is about misconceptions | Basis: both meta-analyses support stating and refuting a misconception over giving only the correct version. The spec does not say how to choose misconceptions or where to put them, and research has not compared ways of writing them |
| The part on questions and feedback suggested adding a line asking writers to write out the wrong idea and then explain why it is wrong (not yet in the spec as of 2026-10-14) | The meta-analyses | Basis as above; Ecker and colleagues' experiment also supports always giving the correct claim |
| CS50 AI project-section rewrite spec (in Chinese): common-mistakes entries state what happens, why, and how to notice | Entries come from review and actual runs (the part on code already checked the reasons) | From this part's angle: it deals with implementation errors, not conceptual misconceptions readers bring with them, so refutation-text research does not map onto it; noticing the problem is close to the catalog's Symptoms field (how to tell a reader holds the misconception). This site's choice; untested |
| The spec does not ask where a misconception in an article comes from | — | Schroeder and Kucera say writing a refutation text presupposes knowing the misconception; this site has no reader data, and its current misconceptions come from the author's judgment, review, and actual runs |
| The spec says nothing on how misconceptions go into headings | — | Only the news-headline advice (Swire-Thompson and colleagues); this site has a sidebar table of contents of headings; applying it to headings is my inference |
What you can do
These are my judgments, each with its basis and strength.
- Before writing a misconception, write down why you think readers hold it: a simplified explanation from the lecturer or the materials, an error found by running things during review, an entry in the programming misconceptions catalog, a mistake you made yourself while learning. If you cannot name a source, consider leaving it out. Basis: Schroeder and Kucera say you need to know the misconception first; Swire-Thompson and colleagues call correcting a misconception few people believe a legitimate concern (untested); the catalog's Origin field. Strength: weak, practice and inference.
- Put the false claim and the correct claim in the same passage, and do not let a heading state only the false claim. Write "A tiny learning rate is not the safe choice," or mark the heading "Common misconception." Basis: Swire-Thompson and colleagues' advice on news headlines, plus this site's sidebar table of contents listing headings on their own. Strength: weak, extrapolated from news.
- Always give the correct claim and one sentence of reasoning, not just "this is wrong." Basis: in Ecker and colleagues' experiment, the group given facts beat control (immediately) and the myth-only group did not; the two were not compared directly. Strength: weak to moderate, immediate effects only, on a public health topic.
- Let the false claim appear in only one section (a heading that carries the correction, plus the body), not repeated in the summary, the tldr, the conclusion, or other sections. Basis: the caveats in the two misinformation papers and the open question Danielson and colleagues list. Strength: weak; no study has compared numbers of repetitions.
- When a misconception can be checked, let readers see it fail for themselves: a piece of code they can run, a small example they can work by hand. Basis: how the programming catalog writes entries. Strength: practice, no effectiveness research.
- Say when the idea is right. For example, a smaller denominator means larger parts, which settles the comparison when the numerators are equal. Basis: Smith and colleagues' theoretical position. Strength: no experiment.
- When readers may already understand, turn the misconception into a question for them to judge, such as "Decide whether this statement is true," with the answer right after. Basis: Dersch and colleagues' personalized refutation won only among people holding the misconceptions; Prinz and colleagues' think sheet had readers write the correct claim themselves. Neither is a "judge first, then read the answer" format. Indirect support comes from a 2020 study by Brashier and colleagues, reported by Swire-Thompson and colleagues: simply asking participants to rate statements for accuracy wiped out the effect by which repetition makes things seem true; that concerns repetition without correction, and I have not read the original. Strength: my inference.
Three things you can do tonight
Pick a teaching article you have written that has a "common misconceptions" passage:
- Next to each misconception (in your draft, not the published version), write one line on where it comes from. For any you cannot source, delete it or rewrite it as a question for the reader to judge.
- Change any heading that states only the false claim. For example, change "Is a smaller learning rate always safer?" to "A tiny learning rate is not the safe choice." Then search the whole article to confirm the false claim appears only in that one section (heading plus body) and is not repeated in the summary or conclusion.
- For one misconception, add a sentence on when it is right, then add a small example readers can run or work by hand to see it fail.
Limits of this evidence
- No study has directly tested a "common misconceptions" passage in a teaching article. The two meta-analyses cover mainly science and math topics; in Schroeder and Kucera the longest text with a reported length is 1,542 English words, and Danielson and colleagues did not report text length. I found no research on refutation writing in programming articles or long math articles.
- Evidence on planting the error comes almost entirely from public issues. By my reading, the studies Swire-Thompson and colleagues reviewed are almost all about politics, vaccines, climate, health rumors, and fictional news; the only one using refutation text appears as a single row in the appendix of the published version. Ecker and colleagues measured beliefs and vaccination intentions, not learning outcomes.
- No direct research on readers who never heard the misconception. Dersch and colleagues' data come closest, but that is an after-the-fact analysis.
- Delayed effects are thin. In Prinz and colleagues, nothing was significant 6 to 8 days later; in Ecker and colleagues, nothing was significant against control after a week; Dersch and colleagues had no delayed test.
- Most experimental results sit at the edge of significance. Most of Prinz and colleagues' main results fall between p = .05 and .08, and the authors say their power was below .80; Dersch and colleagues' subsample result is p = .044.
- Individual-level "backfire" rates cannot be compared directly. In Ecker and colleagues, 5% to 20% of each group had rising scores, including the control group that read no correction at all (20% immediately). My judgment is that this mainly reflects retest noise; Ecker and colleagues do not interpret that table. Dersch and colleagues' 12.5% uses a different scale and baseline.
- My inferences: applying the news-headline advice to article headings, and treating question headings as stating only the false claim; having each misconception appear in only one section; that familiarity backfire is the main worry for teaching articles; that the two spots in the official materials could make readers think they were wrong; turning a misconception into a question for readers to judge; saying when a misconception is right; and each item in "What you can do" and "Three things you can do tonight." All are marked in the text.
- Known to exist, but I could not read a legally available full text: Zengilowski and colleagues' 2021 critical review of the refutation-text literature (Danielson and colleagues say earlier conclusions have been met with skepticism because of it); Prinz and colleagues' 2018 and 2019 papers on whether people with more misconceptions also overestimate how much they understand; Qian and Lehman's 2017 review of programming misconceptions; and the ITiCSE 2021 paper behind the programming catalog. I do not cite their content or list them in the references.
- Secondhand: the Walter and Tukachinsky meta-analysis, Ecker and colleagues 2019, Brashier and colleagues 2020, and the truth-sandwich conference presentation, all as reported by Swire-Thompson and colleagues; the teacher survey as reported by the programming catalog. I have not read any of the originals.
- Readers' metacomprehension (whether people with more misconceptions overestimate how much they understand) will come in a later part.
Appendix: study designs and numbers
Danielson, Jacobson, Patall, Sinatra, Adesope, et al. (2024), Educational Psychologist, read as the author manuscript archived on NSF PAR. Preregistered; 71 articles, 76 studies, 111 samples, 294 effect sizes, total sample 10,265; only science topics, studies assigning participants to separate groups, and no interactive instruction. Accurate beliefs g = 0.37 (posttest only, 67 studies) and 0.36 (pretest-adjusted, 49 studies); the 95% prediction interval for accurate beliefs crosses zero (posttest only, −0.53 to 1.26). Screening (confirmatory moderator): posttest only, not screened 0.42 versus excluding participants with highly accurate beliefs 0.37, b = −0.04 (SE 0.09); pretest-adjusted, 0.42 versus 0.45, b = 0.02 (SE 0.08); neither significant, so the hypothesis that screened studies show larger effects was not supported. None of the 26 moderators was significant at .05; the authors say some analyses had too few studies (for example, only 5 studies outside North America and Europe). Effects on positive emotions, negative emotions, and attitudes were all nonsignificant (7 studies each).
Schroeder and Kucera (2022), Educational Psychology Review 34, read as the published version on PMC. 33 studies, 44 comparisons, 3,869 participants; g = 0.41 (0.30–0.51). Prior knowledge: not reported in 37 comparisons; low prior knowledge g = 0.40 (6 comparisons); mixed −0.03 (1); the coding scheme had a "high" category, but the table has no such row. In 5 of the 44 comparisons the control group did better.
Prinz, Golke, and Wittwer (2021), Instructional Science 49, published version. 92 German university students: refutation text 31, think sheet 29, control 32; the text was adapted from a German statistics textbook on covariance and targeted four misconceptions (covariance indicates causation, is a standardized statistic, is related to the slope of the fitted line, and a covariance of zero proves no relationship at all). Standard version 536 words, refutation version 665 words. 12 minutes of study. The misconception test and the comprehension test were the same 4 items. Reduction in misconceptions: main effect of time F(1.63, 143.52) = 44.59, p < .001; main effect of instructional method p = .507; all three groups had fewer misconceptions than before reading, both immediately and delayed (ps ≤ .010); the refutation-text group rose from immediate to delayed (p = .045). Comprehension: overall interaction p = .053; control slope b = −0.40 (p = .050), refutation text b = 0.15 (p = .487), think sheet b = 0.27 (p = .204); slope differences: refutation text versus control p = .063, think sheet versus control p = .024; at the mean number of misconceptions the three groups did not differ (p = .202, .971); across all three groups the correlation between misconceptions and comprehension was r = .01. Transfer: in the control group, misconceptions were unrelated to transfer (p = .430), so that hypothesis was not supported. Delayed comprehension and delayed transfer: interaction p = .391 and .190, nothing significant. Response-confidence accuracy: think sheet gamma .72, refutation text .43, control .38; post hoc contrast p = .039.
Dersch, Thacker, and Eitel (2026), Educational Psychology Review 38, published version, CC BY 4.0. 2×2 between-subjects (refutation versus expository × personalized feedback versus none), online, preregistered (the authors note that the wording and order of hypotheses changed from the registration). 336 participants: in-service teachers 25.3%, student teachers already teaching 14.9%, student teachers with no teaching experience 59.8%. Group sizes: personalized refutation 93, personalized expository 86, plain refutation 74, plain expository 83. The two expository texts were 163 and 190 words (German), refutation passages 32 to 77 words, personalized feedback passages 43 to 52 words; at least 35 seconds per passage. Misconception score = agree or disagree × certainty, −4 to +4. Full sample: none of the three contrasts was significant (printed as "(t > 1)" in the paper; by context it should be t < 1). The 96 who held both misconceptions at pretest: personalized refutation versus the other three groups b = 0.26, SE = 0.13, t(92) = 2.04, p = .044, R² = 0.05. In the same subsample, a behavioral measure (noticing teachers' stereotyping behavior in classroom videos, n = 82) was lower in the personalized refutation group (p = .047); the authors say this measure did not correlate with misconception scores and counts only as exploratory. 42 people had worse scores after reading (12.5%; the paper says they "backfired"): 8 who held both misconceptions at pretest and 34 who held one or none. A 2×2 ANOVA on these 42 alone found a personalization × refutation interaction, F(1, 38) = 8.11, p = .007; approximate values read from the figure: personalized refutation about −1.75, the other three groups about −0.35 to −1.05. The interaction remained significant after excluding the 8 who held both misconceptions (p = .020). A few other numbers in the paper are inconsistent (group sizes differ between the flow chart and the text, and one indirect effect falls outside its own confidence interval); I follow the text and Table 1.
Brod (2021), npj Science of Learning 6, commentary, published version on PMC. No data. Three factors: activation, relevance, congruence; the author says "at least three," with no one-to-one mapping to learning success.
Swire-Thompson, DeGutis, and Lazer (2020), Journal of Applied Research in Memory and Cognition 9, a narrative review, not a meta-analysis, with no account of how the literature was found. Numbers follow the published version on PMC; the preprint's numbers differ (single versus multiple items 36% versus 9%; the meta-analysis as 11 studies). Appendix C (familiarity): 11 articles, 3 reporting backfire (1 unpublished manuscript, Pluviano and colleagues 2017, Pluviano and colleagues 2019), all on vaccines with single-item measures. Appendix B (worldview): 20 articles, 7 reporting backfire in some subgroup. Share of backfire effects from single-item measures: 81%; rate of backfire with single-item measures 37% versus 8% with multiple-item measures (Z = 2.96, p = .003), with the denominator not stated. The meta-analysis they report: 10 studies, no significant difference in belief updating whether or not the correction repeated the false claim.
Ecker, Sharkey, and Swire-Thompson (2023), PLOS ONE 18, published version. Replicates and extends Pluviano and colleagues 2017. 380 adults in the UK, online in May and June 2021; five groups: myths with facts, myths only (labeled "myth"), visual correction, fear appeal, and control (unrelated medical tips). Three dependent variables (concern about side effects, belief in a vaccine–autism link, vaccine hesitancy), 7 items each, immediately and after a week. Myths-with-facts group immediately: side-effect concern F(1, 370) = 13.43, p < .001; autism link F = 6.90, p = .009; both significant after adjustment. After a week the two were p = .017 and .026, not significant under Holm-Bonferroni correction with families of 4 contrasts. The myths-only group was not significant against control at any time point. No group differed on vaccine hesitancy. No backfire either in the third of participants with the most negative vaccine attitudes. Misbelief in the sample was low to begin with (all means below 1.6 on a 0–5 scale). At the individual level, the share whose scores rose numerically: control 20.00% immediately and 10.00% after a week; myths with facts 5.13% and 11.54%; myths only 20.25% and 16.46%. The authors say power was insufficient to detect small effects.
Smith, diSessa, and Roschelle (1993), Journal of the Learning Sciences 3(2). Theoretical paper, read as the authors' fourth draft; quotations and numbers come from that draft. The fractions case summarizes Smith's unpublished 1990 dissertation: the novices were 10 elementary school students who had just learned fractions, the proficient group 8 high school and 3 middle school students; on 8 comparison items the proficient students used 7.6 different strategies on average, the novices 7.0. No effect sizes, no statistical tests, no comparison of teaching methods.
Programming Misconceptions (progmiscon.org), read 2026-10-14. 104 public entries (Java 55, Python 30, Scratch 13, JavaScript 6) and 143 drafts (hidden by default); the site says the number of entries per language does not reflect how hard the language is. Required fields for each entry: language, incorrect statement, correct statement, concepts, the matching section of the language specification, correction, image. Optional fields filled in among public entries: where it comes from 47, how to tell 57, how to build on it 58, literature 55. A 2025 survey by Chiodini and colleagues as summarized on the site (97 Swiss high school teachers): for AssignmentCopiesObject, 26 teachers themselves held the misconception; for IfIsLoop, 51 said they had never seen it in students. These are teachers' self-reports and their impressions of students, not direct measures of students; I did not check them against the original paper.
References
Papers were read in full, with extraction records; for the website, only the pages listed in parentheses were read. The version read and anything I could not read are noted in parentheses.
- Danielson et al. (2024), The effectiveness of refutation text in confronting scientific misconceptions: A meta-analysis (author manuscript archived on NSF PAR; supplementary materials not read).
- Schroeder & Kucera (2022), Refutation Text Facilitates Learning: a Meta-Analysis of Between-Subjects Experiments (published version on PMC, DOI 10.1007/s10648-021-09656-z; supplementary materials not read).
- Prinz, Golke & Wittwer (2021), Counteracting detrimental effects of misconceptions on learning and metacomprehension accuracy: The utility of refutation texts and think sheets (published version, open access).
- Dersch, Thacker & Eitel (2026), Should we Tell Teachers That They Are (In)Correct? How Personalized Refutation Text Promotes or Hampers Conceptual Change (published version, open access; the original PDF layout was not obtained, and tables were checked against the publisher's table pages; OSF data not opened).
- Brod (2021), Toward an understanding of when prior knowledge helps or hinders learning (published version on PMC; for Figure 1 only the caption was read).
- Swire-Thompson, DeGutis & Lazer (2020), Searching for the Backfire Effect: Measurement and Design Considerations (both the published version on PMC and the PsyArXiv preprint were read; numbers follow the published version; for Figure 1 only the caption was read).
- Ecker, Sharkey & Swire-Thompson (2023), Correcting vaccine misinformation: A failure to replicate familiarity or fear-driven backfire effects (published version; the supplementary file on OSF not read).
- Smith, diSessa & Roschelle (1993), Misconceptions Reconceived: A Constructivist Analysis of Knowledge in Transition (read as the authors' fourth draft archived from SRI's publications page, whose wording differs from the published version; references only skimmed).
- Programming Misconceptions (progmiscon.org): the index page, the about page, the field statistics page, and six entry pages including AssignmentCopiesObject and IfIsLoop (read 2026-10-14; the other 98 entry pages not read).
- Posts on this site: Writing the Examples and Openings of a Teaching Article, Writing Questions and Wording in a Teaching Article, How Far Should the Code in a Teaching Article Go, Harvard CS50 AI course overview.
Loading...