Posted on
Driving out the snake oil in education
Category: Cognitive Science
Beware cognitive science-y sounding snake oil.
Last month, the Federal Trade Commission announced a major settlement against Lumosity, one of many âbrain trainingâ outfits to crop up in the past couple of years (tagline: âWe transform science into delightful gamesâ). If you use Facebook and have children, youâve likely seen advertisements for Lumosity and similar companies. Perhaps you or your children have used their products yourself. Maybe you even found them useful.
These companies have been of personal and professional interest to me because, in the wake of Deans for Impact releasing The Science of Learning, many friends and colleagues have pointed me toward them and suggested âtheyâre doing what youâre doing!â I understand why they think so â after all, these supposed âbrain trainersâ sprinkle terms from cognitive science into their marketing, and make overt claims about improving working-memory capacity and cognitive functioning (to name only two examples). The Science of Learning focuses on cognition, these companies want to improve cognition, so we must be aligned, right?
Wrong. Organizations such as Lumosity are not part of the cognitive-science revolution. They are the snake-oil salesmen of our day, promising quick fixes that (at best) are devoid of scientific support and (at worst) often contradict what science we do know about how learning and cognition improve.
Or at least, that was true until the FTC cracked down on Lumosity and others for making false and misleading claims about whether their product offerings are truly backed by science. And while most of the press coverage has focused on the hefty fine that FTC imposed upon Lumosity ($2 million), what most interested me was the terms of the injunction that prohibits Lumosity from making dubious claims about its products.
There are two prohibitions in the court order. The first prevents Lumosity from making specific scientific claims about the value of its products â for example, that they help ward off Alzheimerâs disease â unless it has âhard scienceâ (my term) to back up its claims. Hard science means:
Human clinical testing of such product that is sufficient in quality and quantity, based on standards generally accepted by experts in the relevant field, when considered in light of the entire body of relevant and reliable scientific evidence, to substantiate that the representation is true. Such testing shall be (1) randomized, adequately controlled, and blinded to the maximum extent practicable; and (2) be conducted by researchers qualified by training and experience to conduct such testing.
The second prohibition blocks Lumosity from making gentler claims about the general benefits or efficacy of its products unless they have âsoft scienceâ (again, my term) to support them. This means Lumosity must be able to cite âtests, analyses, research, or studies (1) that have been conducted and evaluated in an objective manner by qualified persons; [and] (2) that are generally accepted in the profession to yield accurate and reliable results.â
Strong medicine, and while the courtâs order applies only to Lumosity, hereâs an interesting thought exercise: What if these prohibitions were expanded to apply to all products and services offered to teachers under the banner of âprofessional developmentâ? How many would survive an analysis applying these rules of hard and soft science? Might we drive out cognitive science-y snake oil and a great deal of other crap thatâs peddled to teachers, parents and kids as being âresearch basedâ?
This may not be a complete fantasy. While the FTCâs reach is limited, under the Every Student Succeeds Act, the new federal education law that will go into effect next year, states are nudged in the direction of using âevidence-basedâ strategies and interventions in support of various education policies. ESSA identify four tiers of evidence: âstrong,â âmoderate,â âpromisingâ and activities that have a researched-based rationale but lack existing empirical support. The fourth tier may appear to be a loophole, but the law further requires that there must be âongoing efforts to examine the effectsâ that any intervention falling into this category is likely to improve student or other relevant outcomes. In other words, put up or shut up.
As Martin West from Harvard recently observed, the emphasis on evidence and outcomes in ESSA âhold the potential to create and provide resources to sustain a new model for decision-making within state education agencies and school districtsâa model that benefits students and taxpayers and, over time, enhances our knowledge of what works in education.â I agree, but to make that potential the reality will require a much closer nexus between institutes of higher education (which possess research expertise) and practicing teachers and school leaders. I have heard many faculty in colleges of education decry the largely evidence-free professional development thatâs offered to teachers, but complaining about the competition is not a strategy.
We can fix the sorry state of professional development today. As we move into a new era of education policy, academic researchers should work closely with practicing teachers to develop meaningful professional development centered around questions educators struggle with in the classroom. Letâs build a useful evidence base â and drive out the snake oil.
Related Stories:


Blog
Learning by Scientific Design: Strong instructional decision-making is rooted in the science of learning
Since our inception, Deans for Impact has advocated for aspiring teachers to explore our best
April 02, 2021

Blog
âShifting the lift:â how one district is using cognitive science to transform teaching and learning
We worked with Delta County School District to apply learning science to teaching practice.
September 27, 2023