Wrestling the AI Chimera

The main conceit regarding AI comes from the notion of its intelligibility: when using AI, it is NOT POSSIBLE to know WHAT occurs, actually, within it, and even revealing AI's mechanism and dynamics would present yet another problem: very likely, the AI itself would prove completely unintelligible to any man.

Where AI Changes Science

AI, as it stands, now, changes Science: it enables rapid experimentation design and simulation, allowing unimaginable progress across medical science, material science, and other fields which historically presented costly and slow experimentation to explore and validate theses.

On these merits, alone, AI justifies its place in history and in the capital markets.

Where AI Changes Math

Simply browse Terrance Tao's posts on LLM's and proofs, and you see how AI helps illuminate structures and logic otherwise beyond man's ken: researching math without using AI has become unthinkable, as the AI helps the mathematician expand his understanding and discover insights that would very likely elude him.

Where AI Changes Companies

Now, having mentioned two academic uses of AI (and their commercial cognates, e.g. engineering and algorithm design in software), one must consider the much-touted "agentic-AI" sold to conventional companies.

The conceit, and absolute failure, here, comes from grafting-on an uninterpretable, unintelligible process(an AI agent) to an extremely narrowly-defined process (company) with extremely low failure thresholds (litigation, career risk, share price, et cetera).

Case-Study - Replit

Recently, the software development platform, Replit, had its agent deleted the production database for a client - on its own volition:

(for the entire thread, I encourage the curious reader to consult this link: https://x.com/jasonlk/status/1946069562723897802/photo/4)

Of course, Replit management sought to control the damage caused by the rogue agent. Their solutions included:

• Establishing concrete, separate environments for database - automatically
• Implement a "passive mode" to the agent, and so it can only suggest, not ACT, on its decisions.
• Forcing the agent to "learn" the Replit documentation and internal policy.

Nowhere could management even begin to articulate WHY THIS HAPPENED, because it is IMPOSSIBLE for them to understand the agent's logic! There is no post-mortem, no report on what went wrong: just hard-wiring "firewalls" (e.g. automatic separate database environments, passive agent mode) and the hope that training the agent on the internal Replit documentation could magically have prevented this outcome.

AI is Not Trivial to Use, Correctly

There is no proof available, from any company offering "agentic AI", that you won't suffer the same fate as Replit's customer: none. The company has no idea how their own agent works, no one will insure the outcome from an AI agent, and it remains to be seen how the legal system treats catastrophic losses from a poorly-understood AI agent ("rogue AI").

Grafting Agent-based AI onto Existing Companies - Very, Very Hard

Grafting agent-based AI onto existing companies proves very, very hard: agents have a wonderful speculative and exploratory capacity, but using them for execution proves irresponsible: without a proof or some characterization of the agent's behavior, there remains a risk of catastrophic failure.

"Ensemble" Method with Agents

One more appropriate method to managing risk with agents would entail composing them into mesh of agents, acting on "parts" of a problem and thus providing some intelligibility and thus interpretability of risk: thus, an agent could "surface" an intelligible feature, such as a "production database", and another agent would recognize that as something to "ignore" with respect to all mutations on the platform.

Akin to using "weak hypotheses" and "boosting", ensemble methods with agents could provide a solution that allows a statistical bound defined on risk, for a given process - however, the downside comes from the infrastructure and compute cost of such a system, as you dramatically expand the "problem space" by now considering the interaction of so many agents with each other.

Agents Aren't Easy

Correctly using AI agents seems very, very difficult: while trivial to demonstrate casual examples, migrating an entire corporate process to agents could very well "solve" all problems by possibly bankrupting the company, itself! Now, how does that sound for a "global maximum"?

The AI agent naively grafted-onto an existing company proves an anti-pattern: those seeking to effectively use agentic-AI must embrace the risks, and that necessarily precludes naively replacing human labor with agents.

Composing & Scaling Agentic AI

Those seeking to effectively and credibly use agentic AI must have a solution for composing and scaling agents within the organization for a given process: this does not fit the traditional corporate structure of equity -> board -> mgmt -> labor, because there is no possible accountability within the agents, at the moment: furthermore, it remains entirely possible that the agent proves adversarial to management, the board, or even the investors!

How would they know? They can't!

The Best Use of Agents for Existing Companies: Adversarial AI

The best and easiest application for existing companies is to use agents to flood competitors with noise, misinformation, or even carefully-executed fraud: sabotaging existing business processes using AI agents has the absolutely best upside of all agent-based AI applications for conventional companies.

It proves far more profitable and less risky to drown your competitor in agent-based AI noise, fake orders, complaints to government regulators, and social media posts, than to try and graft these agents into your existing workflow.

Agentic-AI Hacking & AI-Proof Companies

It already has become necessary to "AI-proof" the recruiting pipeline, as AI bots flood recruiters with all sorts of resumes.

Soon, agents will take over more and more of the internet, and companies will find themselves overwhelmed by adversarial agentic AI bots seeking ransoms.

All Platforms Will Eat the Costs of Adversarial Agentic AI

ALL platforms will have to absorb and pass-on the costs of protecting human users from adversarial agentic AI.