OPINION: Schools need more ways of knowing if AI and ed tech tools are working

As schools gear up for the new academic year, buzz around artificial intelligence-powered educational tools is reaching new heights. There’s also a strong undercurrent of skepticism, as evidenced by debates about whether cell phones should be banned in classrooms altogether.

With schools grappling with tighter budgets, packed schedules, stubborn achievement gaps and critical youth mental health challenges, educators face a critical question: Which part of valuable instructional time should be dedicated to digital and AI learning?

To answer this question, schools need more ways to know if children’s technology is genuinely effective. They need guidance on which incentives, standards and policies are needed to ensure that harmful technologies are kept out of classrooms.

Related: Become a lifelong learner. Subscribe to our free weekly newsletter to receive our comprehensive reporting directly in your inbox. 

Since generative AI disrupted the growing mix of educational solutions, academic researchers and transparency experts have emphasized the urgent need for governments to fund systems that independently assess education technology tools for their safety, educational quality and potential to drive equitable student outcomes. Unfortunately, we’ve seen a plethora of low-quality, ineffective technology marketed to children and schools.

We believe there is a timely opportunity to raise awareness about the pervasive problems with the quality of ed tech and to offer long-term solutions. The absence of assessment systems recently gave rise to the AllHere debacle in Los Angeles. AllHere’s AI chatbot made waves in the second-largest school district in the U.S. in March, but the parent company collapsed just three months later after failing to deliver on its expensive promises.

Millions of taxpayers’ dollars were wasted, valuable instruction time squandered and students’ futures compromised. We cannot let this happen again. Every school leader can avoid the trap of powerful tech marketing firms delivering false promises and hype.

Here are some ways:

1. Penalize False Claims and Refine Uneven Evidence Standards

Ed tech product managers and procurement experts need to take a “chill pill” on the marketing bluster. Products sold to schools often claim to be “evidence-based” without any independent validation. Ed tech companies make bold claims on their websites, showcasing results from weak “studies” that do not pass Every Student Succeeds Act standards and have not undergone academic peer review. The hype needs to be shut down by states, districts and charter network operators.

Furthermore, the U.S. Department of Education must fine-tune its ESSA evidence tiers. There are four tiers, or four levels of rigor, in its system. The lowest tier, which says a product “demonstrates a rationale,” is too permissive, as it only demands that companies have shown some kind of logic behind their solutions (without checking the quality of the logic) and that the companies have connected research literature to their design. Products assigned to the other end of the ESSA spectrum have had randomized controlled trials (RCTs), which are too expensive (and not always appropriate) for some products.

Many people — from educators to entrepreneurs — have criticized ESSA’s overemphasis on RCT studies. We propose to add an additional tier, modeled after Digital Promise’s inclusive approach, that prioritizes collaborative research and design with teachers. Without facilitating and studying real-time teacher input across diverse contexts, we will not know what works.

2. Fix the Messy World of Ed Tech Certifications

The U.S. issues more certifications for ed tech than any other nation. These certificates and badges confirm that an ed tech product meets specific standards covering various quality aspects, including educational equity for historically underserved students and safety. The certifications listed on the EdTech Index show the key evaluation and validation criteria used by seven certification providers. However, because the certifications have not been consolidated, a company may be labeled “usable” by one certification body but deemed “pedagogically unsound” by another.

To help schools and districts sort through this “digital Wild West,” certifications need to be presented within a new, highly transparent and consolidated framework that integrates certifications from all providers.

An organization independent of U.S. certification providers should convene to examine the specific indicators, address overlaps and assign rankings based on the strength of evaluation and certification procedures.

Education decision-makers and practitioners need a system that provides a quality score for each ed tech tool, consolidating various criteria such as safety, data use, productivity/time savings, equity, efficacy, cost-effectiveness and teaching value.

The assessment or the assigning of quality scores should be facilitated by an independent body connected to districts, states and charter authorities, not by certification providers or ed tech companies themselves.

Related: OPINION: Some warning flags for those embracing personalized learning powered by education technology

The AllHere case and the recent influx of AI tools in the classroom are a wake-up call: We need to urgently overhaul systemic evaluation and certification processes.

If we do not act now, tens of millions of students will be denied the real promise of emerging digital tools to help transform learning. By developing and implementing a robust, independent verification system, we can help ensure that ed tech products deliver evidence-based and equitable solutions for all students.

Natalia I. Kucirkova is a research professor and director of the International Centre for EdTech Impact that connects ed tech academia and industry. Michael H Levine has led innovative social impact initiatives at Sesame Workshop, Nickelodeon, Asia Society and Carnegie Corporation and is a globally recognized leader in the early learning, educational media and digital technology fields. Both have helped design products for meaningful learning impact.

Contact the opinion editor at opinion@hechingerreport.org.

This story about ed tech verification was produced by The Hechinger Report, a nonprofit, independent news organization focused on inequality and innovation in education. Sign up for Hechinger’s weekly newsletter.

Related articles