Welcome back to the series.
In Part 1, I introduced the concept of AI as a useful idiot: a brilliant, powerful, and fundamentally clueless accomplice to a malicious actor who understands its blind spots. And with AI being so closely coupled to many advances in sustainable development, this can mean dangerous consequences in domains ranging from emergency management to clean energy development.
Today, I’ll highlight some recent news on one of the ways that AI commonly gets played.
A Flood of Misinformation
When deadly flash floods hit central Texas several weeks ago, people desperate for answers turned to Elon Musk’s AI chatbot, Grok. What they got was a masterclass in digital confusion. As a recent article laid out, Grok first confidently blamed the tragedy on budget cuts made by President Trump, providing specific (and incorrect) numbers about funding and staff reductions.
Then, facing backlash, the chatbot did a complete 180. “That screenshot is a fabrication—I never posted it,” the AI claimed, before offering a new, slightly different set of facts.
This single incident is a perfect microcosm of our core problem. It’s not just that the AI was wrong; it's how it was wrong and what it reveals about the machine's fundamental nature.
What’s the So What?
So, a chatbot in Texas gave a couple of bad answers. Why does this matter beyond a headline? Because the Grok incident isn't the actual disease; it's a symptom of a much deeper pathology.
Let’s connect some dots. In the LA fires earlier this year, we saw AI used for direct injection attacks. Malicious actors commanded generative AI to create specific falsehoods: the burning Hollywood sign, the fake evacuation maps. This was a tactical, offensive use of the useful idiot to create panic in real-time. It’s the digital equivalent of airdropping propaganda leaflets over a city.
Countermeasures: What can be done?
So, how do we stop AI from being so easily played by the crowd? The platforms where these lies fester and spread have a massive role to play. Now, clearly, I don’t have any more insight into the inner workings of Meta or Twitter censoring practices than anyone else outside of these organizations (basically, we have what they’ve openly published). From what we can tell, they rely on very reactive, post-by-post fact-checking that can’t keep up with the sheer volume and velocity of AI-generated junk.
If we want to get serious about this as a people, we need to stop treating the symptoms and start re-engineering the system itself. But that’s unlikely to happen, especially in this environment. Here are some overall recommendations, going from the practical stuff (some of which it seems the platforms are already doing) to the radical overhaul needed to truly fix the problem. Let’s take a second to imagine a world where the following gets taken seriously.
Level 1: The Practical (The Digital Triage)
This is the low-hanging fruit, some of which has been done by some social media companies. It’s about treating an information crisis with the same urgency as a physical one.
Implement "Informational Circuit Breakers": When a crisis keyword (e.g., "Texas floods," "LA wildfire") starts trending, the platform should automatically trigger a state of emergency. During this state, any unverified account or post that starts going viral on that topic is automatically rate-limited. You don't delete it, you simply throttle its reach. This slows the spread of the lie, giving official sources like FEMA or the National Weather Service a crucial head start.
Shift from Content to Behavior Detection: Don’t try to figure out if every single post is true. Instead, use classifiers to spot the behavior of a misinformation campaign. Look for the technical fingerprints: a sudden spike in activity from new accounts, coordinated link sharing, or the use of identical phrases across hundreds of posts. There are heuristics that can identify a propaganda campaign in progress and allow for its systematic neutralization, rather than going after individual posts.
Authority-Weighted Search & Discovery: In a declared crisis, algorithm parameters need to change. Search results and "For You" feeds related to the crisis should be automatically and heavily weighted to favor verified, official sources. A post from CISA or the LA Fire Department should have a "gravitational pull" a thousand times stronger than a random account with a checkmark. Make the truth the path of least resistance.
Level 2: The Ambitious (Rewiring the Incentives)
This level requires admitting the core business model is the problem. Outrage drives engagement, and engagement drives revenue. You have to break that cycle.
Mandate Content Provenance: This is already happening with the rise of C2PA standards. Platforms should make it mandatory. Any AI-generated image or video uploaded must have these embedded "Content Credentials." If it doesn't, it gets automatically flagged as "Unverified Synthetic Media." This creates transparency by default. The useful idiot has to announce itself.
Introduce a "TrustRank" for Accounts: Go beyond simple verification. Every account should have an internal, dynamic "trust score." Is the account new? Does it have a history of being debunked by fact-checkers? Does it primarily share links from known propaganda mills? Accounts with low trust scores don't get banned, but their content is systematically deprioritized by the recommendation algorithm. Make being a habitual liar bad for the business of getting reach. The recommendation algorithm systematically deprioritizes their content.
Deprecate Virality as the Primary Metric: This is the big one for the product managers. The goal of the algorithm can no longer be "what will keep this user scrolling?" It needs to become, "what is the most credible information I can show this user on this topic?" This means fundamentally changing the KPIs for success from pure engagement to a weighted score of engagement plus source credibility. It’s less profitable in the short term, but it’s essential for long-term platform health.
Level 3: "If I Ruled the World" (The Public Health Model)
This is where we stop treating these platforms as just businesses and start treating them as critical public infrastructure.
Mandatory "Glass Box" Audits: No more "the algorithm is a secret sauce" for the platforms. Regulators and vetted academic researchers should get audited, sandboxed access to the live recommendation and moderation algorithms. Develop techniques to do this without revealing too much about intellectual property; it's about transparency and showing your homework. We need to be able to see how the machine is making its decisions, especially in high-stakes environments like elections and public health crises.
A Fiduciary Duty of Care: Legally reframe the relationship between a platform and its users. Argue that a platform operating at this scale has a fiduciary duty not to actively harm its users' well-being by knowingly amplifying dangerous and false content for profit. This would make the health of the information ecosystem a core legal obligation, just as a financial advisor has a duty to act in their client's best interest. It would force a cultural and legal reckoning inside these companies, making them custodians of the public discourse, not just exploiters of it.
Finally, governments and civil society should treat climate misinformation as a security threat — standing up “eco-war rooms” that bring together platforms, emergency services, scientists, and community leaders to coordinate real-time responses when false narratives flare up. The recent London Climate Week 2025 workshop underscored a need for a “rapid, coordinated response” to climate misinformation, along with public education and transparent AI use.
In the long run, boosting media and information literacy is our best defense: an informed public is far less likely to be duped by an AI-manufactured hoax. UNESCO’s new course on navigating climate information is a great example, equipping thousands of people with skills to spot deepfakes and verify claims. We need more education like this to solve the problem. Because unchecked misinformation campaigns on AI-driven social media and news platforms are now as real a threat as climate change itself.