The Federal Trade Commission has accused a company that makes AI-powered security screening systems for some 800 schools across 40 states of promoting false claims about its ability to detect weapons and keep kids safe. 

Evolv Technology, which sells AI-powered 鈥渨eapons detection鈥� systems to schools and other businesses, made deceptive claims to customers about its ability to detect all weapons accurately and efficiently, the commission alleged in . 

Schools make up half of Evolv鈥檚 business, according to the complaint, even though the publicly traded company may be best known as a security staple at stadiums and for a pilot program in New York City鈥檚 subways earlier this year that yielded dismal results.

鈥淭he FTC has been clear that claims about technology 鈥� including artificial intelligence 鈥� need to be backed up, and that is especially important when these claims involve the safety of children,鈥� Samuel Levine, director of the commission鈥檚 bureau of consumer protection, said in a media release. 鈥淚f you make those claims without adequate support, you can expect to hear from the FTC.鈥�

Evolv鈥檚 marketing materials promote its scanners as high-tech alternatives to metal detectors, but the complaint argues the company made inaccurate assurances about the product鈥檚 ability to reduce false alarms, cut labor costs, eliminate the need for people to remove innocuous items from their pockets 鈥� and its capability to detect all weapons.

The company it had reached a settlement with the commission that did not involve any admission of wrongdoing or monetary penalties but would give certain K-12 customers a 60-day window to cancel the remainder of their current contracts.

The eligible districts account for 8% of all Evolv customers, according to a media release, and deploy 4% of its scanners. Cancelling those contracts could impact $3.9 million of its annual revenue.

鈥淭his resolution allows us to focus on a small segment of our school customers to ensure they remain satisfied with鈥� Evolv鈥檚 scanners 鈥渁nd allows us to move forward without distraction,鈥� Mike Ellenbogen, the company鈥檚 interim president and CEO, said in the statement.

Evolv has that it uses AI to scan for the unique 鈥渟ignatures鈥� of tens of thousands of weapons, allowing it to distinguish 鈥渁ll the guns, all the bombs and all the large tactical knives鈥� out there from everyday items like keys and laptops. 

In its release, Evolv framed the FTC inquiry as focusing on past marketing materials and not the efficacy of its AI technology, but it also said the company would 鈥渞efine the way it markets its technology, highlighting capabilities and limitations.”

School safety consultant Kenneth Trump, president of National School Safety and Security Services, told 社区黑料 that school leaders have increasingly turned to weapons detection systems to signal to parents that they鈥檙e taking proactive steps to keep students safe. 

鈥淪ome may have unknowingly created the very result they hoped to prevent: A quandary, as they may have to explain to their school communities why they bought technology that may not be delivering what was implied or promised to them and their school community,鈥� he said. 

The 鈥� including its 鈥� has faced pushback for several years, particularly by IPVM, an independent security and surveillance industry research group that tests and evaluates products. Some of the schools the group researched had false alarm rates of up to 60%. 

Last year, a high school student in Utica, New York, after he was stabbed in a campus hallway by a classmate who brought a knife past an Evolv scanner without detection. The school district had spent $3.7 million on the scanners from Evolv, a Massachusetts-based company backed by big-name investors including Bill Gates and Peyton Manning. The company boasts its artificial intelligence-equipped devices can screen up to 1,000 students in 15 minutes 鈥� 10 times faster than traditional metal detectors.聽

New York City Mayor Eric Adams of Evolv鈥檚 scanners inside some New York City subway stations this year, which was met with opposition from civil rights groups who argued it was unconstitutional and impractical to screen millions of transit users daily. Over the course of a month, the scanners across 20 subway stations had 118 false positives and recovered 12 knives. They didn鈥檛 detect a single firearm. 

The company on Tuesday pointed to two incidents last month where Evolv scanners detected guns that students were attempting to bring into their and high schools.

As AI becomes a buzzword in education technology, the FTC in February the prowess of their artificial intelligence offerings, adding that 鈥渇alse or unsubstantiated claims about a product鈥檚 efficacy are our bread and butter.鈥�&苍产蝉辫;

Publicly traded Evolv Technology acknowledged that the Federal Trade Commission had 鈥渞equested information about certain aspects of its marketing practices鈥� in last week, of its technology in promotions that could give customers, including schools, a false sense of security. 

Citing two anonymous sources, is the subject of an FTC investigation into whether its scanners 鈥� essentially next-generation metal detectors with a 鈥� employ artificial intelligence to identify weapons in the ways that it claims.

It鈥檚 unclear whether Massachusetts-based Evolv鈥檚 sales pitches to the education sector are part of the federal probe. An FTC spokesperson declined to comment Tuesday. In its Oct. 12 disclosure form with the Securities and Exchange Commission, and in a statement this week to 社区黑料, Evolv said the company was 鈥減leased to answer鈥� regulators鈥� questions. 

鈥淲hen Evolv receives inquiries from regulators, our approach is to be cooperative and educate them about our company,鈥� the statement continued. 鈥淭he company stands behind its technology鈥檚 capabilities and performance track record.鈥�

The company has that it uses AI to scan for the unique 鈥渟ignatures鈥� of tens of thousands of weapons, allowing it to distinguish 鈥渁ll the guns, all the bombs and all the large tactical knives鈥� out there from everyday items like keys and laptops. 

Yet the 鈥� including its 鈥� has faced pushback for several years, particularly by IPVM, an independent security and surveillance industry research group that tests and evaluates products. Conor Healy, the group鈥檚 director of government research, said that false and misleading marketing claims have been 鈥渁 pattern with the company鈥� for years. Among the inaccurate assertions, he said, is that the tool 鈥渆liminates the friction鈥� that students experience when they pass through security everyday. 

鈥淭hat has been shown to be just simply not true at all,鈥� Healy told 社区黑料 this week. 鈥淭here鈥檚 quite a lot of friction. The schools that we鈥檝e looked at have .鈥�&苍产蝉辫;

Districts have increasingly turned to 鈥渨eapons detection鈥� systems from Evolv and competing security vendors in response to fears of school shootings 鈥� anxiety that the company says 鈥渒eeps both students and staff from doing their best work.鈥�

Evolv 鈥渃ombines powerful sensor technology with proven artificial intelligence鈥� to identify threats like guns in hundreds of U.S. schools. Capable of scanning more than 4,000 people an hour, Evolv says its devices are 鈥�10X faster than metal detectors,鈥� and 鈥渉elp reduce opportunities for bias鈥� by decreasing secondary screenings by humans.

Evolv extols the benefits of its scanners well beyond schools鈥� physical safety. While frequent false alarms by traditional metal detectors lead to 鈥渟ecurity anxiety鈥� and 鈥渋nconvenient delays,鈥� according to the company鈥檚 website, Evolv scanners offer 鈥渁 more effective and dignified solution, fostering a safer, more inclusive environment that bolsters academic achievement and staff retention.鈥�&苍产蝉辫;

IPVM has accused the company of is 10 times faster than traditional metal detectors, and found the scanners . Meanwhile, IPVM has documented instances where false alarms were by water bottles, binders and laptops.

In a statement to Pennsylvania-based IPVM last month, Evolv said 鈥渨e understand if any of our past statements appeared to generalize our capabilities,鈥� which may violate an FTC rule that requires company claims to be evidence-backed. 

With AI a constant, if little understood, buzzword across many sectors right now, the FTC in February the capabilities of their artificial intelligence offerings, adding that 鈥渇alse or unsubstantiated claims about a product鈥檚 efficacy are our bread and butter.鈥�&苍产蝉辫;

鈥淭he minute you hear the word AI in marketing, alarm bells should go off in your head,鈥� said Healy, whose group has also done and the routinely installed in schools. 

鈥淎s far as [Evolv鈥檚] artificial intelligence goes, it does not appear to be very intelligent,鈥� he said, because it routinely fails to differentiate everyday school supplies like Chromebooks from weapons like guns. 鈥淲hat AI is actually in the system? That is something that Evolv has not told us very much about.鈥�&苍产蝉辫;

Evolv has resisted calls to disclose additional information about the ways its scanners function. While scanners鈥� sensitivity settings can alter their performance, a company spokesperson previously told 社区黑料 that publicly sharing information about those settings 鈥渋s irresponsible and puts people at greater risk.鈥�&苍产蝉辫;

鈥淲e must assume any published information regarding details of a physical screening system will be studied and leveraged by a bad actor seeking to do harm,鈥� the statement continued. The company declined to comment on the false alarm rates reported by its customer districts, which include ,, and

 鈥淥ur systems are designed to detect many types of weapons and components of weapons, but there is no perfect solution that will stop 100% of threats, including ours, which is why security must include a layered approach that involves people, process and technology.鈥�&苍产蝉辫;

Knives became a point of conflict last year after the school district in Utica, New York, spent nearly $4 million to install Evolv scanners across 13 of its campuses. The scanners were ultimately removed after a student was stabbed multiple times with a knife during a fight in a high school hallway. The knife-wielding student had passed through an Evolv scanner with the blade in his backpack, a later investigation revealed. 

While the detectors had false alarms, including on a student鈥檚 lunch box, an Evolv scanner failed to alarm when an off-duty police officer accidentally brought a service revolver to a Utica district open house.

Meanwhile, in Buffalo, New York, Evolv scanners were credited for keeping a high school safe. Earlier this month, an to a criminal weapons possession charge after he was caught trying to bring a handgun into a high school. A school security officer reportedly found the disassembled 鈥済host gun鈥� in the teenager鈥檚 backpack as he passed through a weapons detector. Buffalo schools earlier this year. A Buffalo schools spokesperson declined to comment.

As companies increasingly market products with artificial intelligence capabilities to schools, school security consultant Kenneth Trump predicts 鈥� or at least hopes 鈥� that regulation is imminent. He pointed to new rules in . The ban was adopted after an upstate school district鈥檚 decision to install surveillance cameras with facial recognition capabilities prompted an outcry. 

鈥淭he marketing claims are so off the charts by many vendors that there鈥檚 really no chance for the average school administrator to know what鈥檚 true, what鈥檚 false and really the gaps and the limitations that these products have,鈥� said Trump, president of Cleveland-based National School Safety and Security Services. Though he expects regulators to soon reign in security companies, 鈥渦p until that happens, how many school districts are going to fall victim to questionable marketing and grandiose ideas that don鈥檛 come to fruition?鈥�

Since that disclosure in New York City schools, the scope of the breach has only grown, with districts in six states announcing that some had become victims. Illuminate has never disclosed the full extent of the blunder, even as critics decry significant harm to kids and security experts question why the company is being handed awards instead of getting slapped with sanctions. 

Amid demands that Illuminate be held accountable for the breach 鈥� and for allegations that it misrepresented its security safeguards 鈥� the company could soon face unprecedented discipline for violating , a self-regulatory effort by Big Tech to police shady business practices. In response to inquiries by 社区黑料, the Future of Privacy Forum, a think tank and co-creator of the pledge, disclosed Tuesday that Illuminate could soon get the boot.

Forum CEO Jules Polonetsky said his group will decide within a month whether to revoke Illuminate鈥檚 status as a pledge signatory and refer the matter to state and federal regulators, including the Federal Trade Commission, for possible sanctions. 

鈥淲e have been reviewing the deeply concerning circumstances of the breach and apparent violations of Illuminate Education鈥檚 pledge commitments,鈥� Polonetsky said in a statement to 社区黑料. 

Illuminate did not respond to interview requests. 

In a twist, the pledge was co-created by the Software and Information Industry Association, the trade group that last month as being  among 鈥渢he best of the best鈥� in education technology. The pledge, created nearly a decade ago, is designed to ensure that education technology vendors are ethical stewards of kids鈥� most sensitive data. Its staunchest critics have assailed the pledge as being toothless 鈥� if not an outright effort to thwart meaningful government regulation. Now, they are questioning whether its response to the massive Illuminate breach will be any different. 

鈥淚 have never seen anybody get anything more than a slap on the wrist from the actual people controlling the pledge,鈥� said Bill FItzgerald, an independent privacy researcher. Taking action against Illuminate, he said, 鈥渨ould break the pledge鈥檚 pretty perfect record for not actually enforcing any kind of sanctions against bad actors.鈥�

Through the voluntary pledge, launched in 2014, hundreds of education technology companies have agreed to a slate of safety measures to protect students鈥� online privacy. Pledge signatories, , they will not sell student data to third parties or use the information for targeted advertising. Companies that sign the commitment also agree to 鈥渕aintain a comprehensive security program鈥� to protect students鈥� personal information from data breaches. 

The privacy forum, which is , has long maintained that the and offers assurances to school districts as they shop for new technology. In the absence of a federal consumer privacy law, the forum argues the pledge grants 鈥渁n important and unique means for privacy enforcement,鈥� giving the Federal Trade Commission and state attorneys general an outlet to hold education technology companies accountable via consumer protection rules that prohibit unfair and deceptive business practices. 

For years, critics of providing educators and parents false assurances that a given product is safe, than a pinky promise. Meanwhile, schools and technology companies have become increasingly entangled 鈥� particularly during the pandemic. As districts across the globe rushed to create digital classrooms, few governments checked to make sure the tech products officials endorsed were safe for children, by the Human Rights Watch. Shoddy student data practices by leading tech vendors, the group found, were rampant. Of the 164 tools analyzed, 89 percent 鈥渆ngaged in data practices that put children鈥檚 rights at risk,鈥� with a majority giving student records to advertisers.

As companies suck up a mind-boggling amount of student information, a lack of meaningful enforcement has let tech companies off the hook for violating students鈥� privacy rights, said Hye Jung Han, a Human Rights Watch researcher focused on children. As a result, she said, students whose schools require them to use certain digital tools are being forced to 鈥済ive up their privacy in order to learn.鈥� Paired with large-scale data breaches, like the one at illuminate, she said students鈥� sensitive records could be misused for years. 

鈥淐hildren, as we know, are more susceptible to manipulation based on what they see online,鈥� she said. 鈥淪o suddenly the information that鈥檚 collected about them in the classroom is being used to determine the kinds of content and the kinds of advertising that they see elsewhere on the internet. It can absolutely start influencing their worldviews.鈥�

But the regulatory environment under the Biden administration may be entering a new, more aggressive era. The Federal Trade Commission announced in May that it would scale up enforcement on education technology companies that sell student data for targeted advertising and that 鈥渋llegally surveil children when they go online to learn.鈥� Even absent a data breach like the one at Illuminate, the commission wrote in a policy statement, education technology providers violate the if they lack reasonable systems 鈥渢o maintain the confidentiality, security and integrity of children鈥檚 personal information.鈥�&苍产蝉辫;

The FTC  declined to comment for this article. Jeff Joseph, president of the Software and Information Industry Association, said its recent awards were based on narrow criteria and judges 鈥渨ould not be expected to be aware of the breach unless the company disclosed it during the demos.鈥� News of the breach was . 

The trade group 鈥渢akes the privacy and security of student data seriously,鈥� Joseph said in a statement, adding that the Future of Privacy Forum 鈥渕aintains the day-to-day management of the pledge.鈥�&苍产蝉辫;

鈥楢bsolutely concerning鈥�

Concerns of a data breach at California-based Illuminate in January when several of the privately held company鈥檚 popular digital tools, including programs used in New York City to track students鈥� grades and attendance, went dark. 

Yet it that city leaders announced that the personal data of some 820,000 current and former students 鈥� including their eligibility for special education services and for free or reduced-price lunches 鈥� had been compromised in a data breach. In disclosing the breach, city education officials of misrepresenting its security safeguards. The Department of Education, which over the last three years, to stop using the company鈥檚 tools. 

A month later, officials at the New York State Education Department launched an investigation into whether the company鈥檚 data security practices ran afoul of state law, department officials said. Under the law, education vendors are required to maintain 鈥渞easonable鈥� data security safeguards and must notify schools about data breaches 鈥渋n the most expedient way possible and without unreasonable delay.鈥�&苍产蝉辫;

Outside New York City, state officials said the breach affected about 174,000 additional students across the state.

Doug Levin, the national director of The K12 Security Information eXchange, said the state should issue 鈥渁 significant fine鈥� to Illuminate for misrepresenting its security protocols to educators. Sanctions, he said, would 鈥渟end a strong and very important signal that not only must you ensure that you have reasonable security in place, but if you say you do and you don’t, you will be penalized.鈥�&苍产蝉辫;

Meanwhile, Illuminate has since become the subject of two federal class-action lawsuits in New York and California, including one that alleges that students鈥� sensitive information 鈥渋s now an open book in the hands of unknown crooks鈥� and is likely being sold on the dark web 鈥渇or nefarious and mischievous ends.鈥�&苍产蝉辫;

Plaintiff attorney Gary Graifman said that litigation is crucial for consumers because state attorneys general are often too busy to hold companies accountable. 

鈥淭here鈥檚 got to be some avenue of interdiction that occurs so that companies adhere to policies that guarantee people their private information will be secured,鈥� he said. 鈥淥bviously if there is strong federal legislation that occurs in the future, maybe that would be helpful, but right now that is not the case.鈥�

School districts in California, Colorado, Connecticut, Oklahoma and Washington have since disclosed to current and former students that their personal information had been compromised in the breach. But the full extent remains unknown because 鈥淚lluminate has been the opposite of forthcoming about what has occurred,鈥� Levin said. 

companies to disclose data breaches to the public. Some 5,000 schools serving 17 million students use Illuminate tools, according to the company, which was founded in 2009.

鈥淲e now know that millions of students have been affected by this incident, from coast to coast in some of the largest school districts in the nation,鈥� including in New York City and Los Angeles, Levin said. 鈥淭hat is absolutely concerning, and I think it shines a light on the role of school vendors,鈥� who are a significant source of education data breaches. 

Nobody, , can guarantee that their cybersecurity infrastructure will hold up against motivated hackers, Levin said, but Illuminate鈥檚 failure to disclose the extent of the breach raises a major red flag. 

鈥淭he longer that Illuminate does not come clean with what鈥檚 happened, the worse it looks,鈥� he said. 鈥淚t suggests that this was maybe leaning on the side of negligence versus them being an unfortunate victim.鈥�

鈥楢 public relations tool鈥�

When six years ago, it acknowledged the importance of protecting students鈥� data and said it offered a 鈥渟ecure online environment with data privacy securely in place.鈥� , Illuminate touts an 鈥渦nwavering commitment to student data privacy,鈥� and offers a link to the pledge. 

鈥淏y signing this pledge,鈥� the company wrote in a 2016 blog post, 鈥渨e are making a commitment to continue doing what we have already been doing from the beginning 鈥� promoting that student data be safeguarded and used for encouraging student and educator success.鈥�&苍产蝉辫;

Some pledge critics have accused tech companies of using it as a marketing tool. In 2018, argued that pledge noncompliance was rampant and accused it of being 鈥渁 mirage鈥� that offered comfort to consumers 鈥渨hile providing little actual benefit.鈥�&苍产蝉辫;

鈥淭he pledge may be more valuable as a public relations tool than as a means of actually effecting 鈥� or reflecting 鈥� industry improvements,鈥� according to the report. Gaps between the pledge鈥檚 public declarations and companies business practices, it concluded, 鈥渋s likely to mislead consumers.鈥�&苍产蝉辫;

In 2015, a software researcher found a large share of pledge signatories infrastructure to guard student data from hackers. Three years later, The New York Times published , a nonprofit that administers the widely used SAT college admissions exam. College Board, the report exposed, was selling student data to third parties in violation of the privacy pledge. In response, the College Board鈥檚 status as a pledge signatory had been placed 鈥渦nder review,鈥� but as an active signatory a year later. The College Board, it said in a press release, had committed to changing its business practices. 

Still, in 2020 found the College Board was sending student data to major digital advertising platforms, including those operated by Microsoft and Google. The College Board, . 

The nonprofit is 鈥渞esolute in protecting student data privacy,鈥� a spokesperson said in a statement. 鈥淥rganizations that receive data from College Board, such as high schools, districts, colleges, universities, and scholarship organizations, must adhere to strict guidelines when using that data.鈥�

Some critics have argued the College Board should have been removed from the pledge, but the Future of Privacy Forum has held that taking such action against signatories could do more harm than good. When the forum becomes aware of a complaint against a pledge signatory, it typically works with the company to resolve issues and ensure compliance, . The think tank argued it鈥檚 best to work with noncompliant companies to improve their business practices rather than exile them from the pledge outright. Removing companies 鈥渃ould result in fewer privacy protections for users, as a former signatory would not be bound by the Pledge鈥檚 promises for future activities.鈥�&苍产蝉辫;

Attorney Amelia Vance, a former privacy forum employee and the founder and president of Public Interest Privacy Consulting, said the pledge has nudged education technology companies to change their business practices to ensure they鈥檙e following its provisions. 

鈥淚 almost always thought of it as a way to make companies better and more aware of student privacy than something to be enforced with specific teeth,鈥� said Vance, who declined to comment on whether Illuminate should be removed. 鈥淎fter all, the Federal Trade Commission and state [attorneys general] are the ones who really have the enforcement powers here.鈥�

But self-policing efforts, like the pledge, are 鈥渙nly as effective as the enforcement,鈥� said Levin, the school security expert. Otherwise, it can only serve as 鈥渁 nice window dressing鈥� for Big Tech efforts to fend off stricter state and federal regulations 鈥� provisions he said must be strengthened. 

At a minimum, he said the privacy forum should disclose companies that have been credibly accused of violating the pledge and to conduct investigations. If they find a company out of compliance, he said 鈥渋t鈥檚 not clear to me that they should be allowed to re-sign the pledge.鈥�

鈥淚f I were another signatory of the pledge, I would be quite concerned about whether or not the value of that pledge is being diminished鈥� by including companies that violate its provisions, he said. 鈥淚f it鈥檚 going to serve its purpose, there needs to be some policing.鈥�

But to Fitzgerald, the privacy researcher, the forum鈥檚 failure to take action against bad actors has long rendered the pledge useless. 

鈥淚t鈥檚 not like the pledge finally doing what the pledge should have been doing five years ago would make a difference,鈥� he said. 鈥淚t鈥檚 never too late to start鈥� removing companies that violate its provisions, he said, but 鈥渢he fact that it hasn鈥檛 happened yet seems to indicate that it鈥檚 not going to happen.鈥�&苍产蝉辫;

Disclosure: The Bill & Melinda Gates Foundation and the Chan Zuckerberg Initiative provide financial support to the Future of Privacy Forum and 社区黑料