This originally appeared on Teaching Times.

The over-blown claims for new technology in education never materialise. Even in the Pandemic it is old style technology that is being most deployed. Professor Justin Reich of MIT ask why?

Over the last ten years, education technology evangelists have made remarkable claims about how new technologies will transform educational systems. In 2009, Clay Christensen of the Harvard Business School predicted that half of all secondary school courses in the US would be online by 2019, and that they’d cost 1/3 of a traditional course and provide better outcomes. Sal Khan of Khan Academy proposed in a TED talk that he could use short videos to reinvent education.

Sebastian Thrun of Udacity said that in 50 years we’d have only 10 institutions of higher education in the world after massive open online courses colonized the field. As the winner of the TED Prize, Sugata Mitra claimed that students didn’t even need schools or teachers, and that groups of children with access to the internet could teach themselves anything.

A disaster

And then in 2020, the world was blighted by a terrible pandemic. Schools serving over 1.6 billion learners shut down. It was a moment that technologists had promised for years could be transformative, but for most learners and families, remote online learning has been a disaster.

As educators face the challenge of spooling up new online and hybrid schools to serve vast numbers of students, they can choose from three kinds of technologies that support learning at scale. In classifying education technology, the first question to ask is “who controls the learning experience for students?”

There are three answers to this question: instructor-guided online courses, algorithm-guided adaptive tools, and peer-guided networked learning communities. Each of these genres has strengths and limitations; each is good for some subjects, but not others; for some students, but not others.

Very successful students do well

For older students, there are many different forms of instructor-guided, self-paced online courses available: free offerings from Khan Academy or FutureLearn along with subscription virtual courseware from a variety of companies. These cover many topics in the curriculum, but they only serve a subset of students well. Research shows that the learners who are most successful in self-paced courses are those who are already very successful in school—self-motivated and academically well-prepared.

Virtually none of our youngest learners meet these criteria. For auto-didacts, the options are limitless, but for the vast majority of us who need contact with human teachers to help us learn, these kinds of offerings are not very helpful

There are a variety of algorithm-guided adaptive tools that are engaging and beneficial to student learning. Part of the appeal of these tools is that they use a variety of different kinds of automated assessments to determine student understanding, and then they provide personalized learning pathways to students based on their performance.

Theoretically compelling

Theoretically, it’s a compelling model, where each student gets the instruction, the assessment, the feedback, and the experience that they need. But, the model depends on having good automated assessments, which only exist in three domains: mathematics, computer science, and early language acquisition (such as learning to read a native language in primary school, or learning the introductory parts of a foreign language).

Even within these domains, the assessments are only partially useful—in computer science we can automatically assess whether a student has met a well-defined engineering challenge, but not if they’ve made an aesthetically pleasing home page for a new website. Automated math tutoring software can be a useful part of a school systems’ remote learning plans, but there simply isn’t good automated tutoring software for studying literature, science, social studies, or most of the rest of the school curriculum.

While instructor-guided and algorithm-guided technologies attempt to use computers to directly teach students, some technologists have built learning environments where peers teach each other. The Scratch community, where young people learn computational creativity and share their programs, tutorials, and resources with one another, is probably the best example of a peer learning network that has been adopted in schools.

Outside of formal schooling, virtually everyone in the networked world participates in some way in these learning networks, when they peruse and comment on makeup tutorials, or read up on video game wikis, or participate in networks for hobbies and crafts. The kind of learning that people do in these networks is rich and deep, but it depends tremendously on internal motivation. People learn amazing things in online learning networks that tap into personal interests, but they tend not to be useful for teaching and learning about mandatory school subjects.

To sum up: instructor-guided self-paced online courses are great for autodidacts, but not particularly useful for most students; adaptive tutors work great for many learners, but only in a few subject areas; peer networks work well for learning about personal passions, but not so well for mandatory school curriculum.

Sweet spots for distance learning

For two decades, education technology entrepreneurs have promised a disruptive transformation of the learning landscape, but in reality, the field has produced a limited set of tools that only work for some students, in some subjects, in some contexts. Within those sweet spots, learning technologies can be incredibly powerful. But those sweet spots only cover a fraction of all of the learning that typical school systems try to provide for all of their students.

As a result of these limitations, during the pandemic, the vast majority of school systems—both for primary and secondary students and for higher education, have primarily turned not to emerging tools but towards two of our very oldest learning technologies: learning management systems and video telephony.

Learning management systems, like Google Classroom, Schoology, Canvas, or Moodle are digital spaces for sharing, distributing, and collecting online documents. These systems were theorized in the scholarly literature in the 1960s and 1970s, made commercially available in the 1990s, and available in open source in the 2000s. They let teachers assign and collect digital worksheets.

Video telephony was the 1930s name for what we now call video conferencing, services like Zoom or Microsoft Teams that let people see and hear each other online. These two technologies let systems recreate traditional models of schools—teacher lecture, student recitations, individual student practice on worksheets. This hasn’t worked particularly well, but there really aren’t examples of where new technologies are offering much better outcomes to students.

Four dilemmas

What would it look like to have a more robust set of large-scale learning technologies for the next pandemic? To create and implement technologies that work better at large scales, edtech designers and researchers will have to find new ways to overcome four dilemmas that have consistently hindered efforts to transform education with technology.

The first dilemma is what I call the Curse of the Familiar. When technologists create novel and innovative new tools for teaching and learning, educators and students often find them confusing and hard to adopt. If you make something very different from traditional school practice, it won’t fit into schools very well. But on the other side, if you build a technology that digitizes existing school practices—if you make digital flashcards or digital worksheets—they tend to not be that much better for learning than existing practices. The only solution to this dilemma is to recognize that new technology adoptions require substantial professional development efforts.

Second dilemma

The second dilemmas is the EdTech Matthew Effect. As we have seen in tragic ways throughout the pandemic, learning technologies tend to be most useful for affluent students with the financial, social, and technical resources to take advantage of new innovations. New technologies typically widen educational disparities rather than closing them.

Third dilemma

A third issue is that learning requires feedback, but technology designers are only good at evaluating human performance in domains where correct answers are highly structured. Computers can identify the correct answer to a math problem or even a correctly-pronounced word, but they cannot identify whether an essay shows a student effectively reasoning from evidence. The Trap of Routine Assessment observes that many education technologies rely on automated assessments, but computers can’t assess many of the most important things that our students learning.

Fourth dilemma

The Toxic Power of Data and Experiment highlights how new technologies are powerful platforms for research and A/B testing which can be used to dramatically improve computational systems, but only if communities are willing to tolerate risks to privacy and a growing surveillance over education.

Fundamentally, the dilemmas recognize that technology alone can’t transform schools. At best, technology can play a role in helping educators and communities build better learning systems.

These limits are not, in themselves, cause for despair. Improving teaching and learning is immensely hard. Education technology can’t solve all of the challenges of remote learning, but it can effectively address the needs of some students in some subjects.

For communities facing school closures, teaching young children math will prove incredibly difficult through video conferencing. By good fortune, some of our best learning technologies are adaptive tutors in elementary math subjects. That doesn’t solve every problem that primary head teachers face, but it helps with one of them.

For those who are hoping that education technology can transform our existing systems, that’s probably a disappointment. But if you see human development as a slow, painstaking process of gradual improvement, than those kinds of incremental steps are as good as it gets.

Justin Reich is the Mitsui Career Development Professor at MIT, director of the MIT Teaching Systems Lab, and the author of the forthcoming Failure to Disrupt: Why Technology Alone Can’t Transform Education from Harvard University Press.