Too Smart to Prompt

The Environment Is the Product: Even Frontier Models Need Help

The new paradigm started with the limited rollout of Claude Mythos. Mythos was deployed to a select group of companies to identify and patch security vulnerabilities. These companies spent millions of dollars in the first weeks on Mythos tokens, with some spending upwards of $30M in the first month. Each avoided likely billions in potential damages. However, they didn’t just point Mythos at their codebases and let the model run.

Cloudflare was one of these companies. They found that pointing Mythos alone at a large repository and asking it to find vulnerabilities produced very mixed results. The model would wander, produce many noisy findings, and could not hold highly parallel context, like how an issue in one part of the codebase could interact with a set of vulnerabilities in a totally different part of the product.

Instead, Cloudflare built a context system that first mapped the repository, identified important entry points and security boundaries, and divided the search into narrow tasks. Around fifty copies of the model then worked in parallel, each examining one type of attack in one part of the code. Separate models tried to disprove each finding, incomplete areas were sent back for another pass, duplicates were removed, and confirmed bugs were traced through other parts of the repository to determine whether an attacker could actually reach them.

Other companies with access to Mythos reached the same conclusion. They gave the model a valuable objective, the context and tools to pursue it, the ability to manage parallel instances of itself, feedback loops that let it test and learn, and robust governance infrastructure.

We call this broader system an environment. Building one takes meaningful time, money, expertise, and judgment. When a deployment is aimed at a problem worth billions and can cost tens of millions in token spend, engineering the right environment is critical to ensure the models are producing trustworthy results.

In the last era of AI assistants, harnesses became crucial. The same exact model could be an ultra-productive, autonomous programmer or produce buggy spaghetti code. In this next era, the entire environment you craft for a model will be the limiting factor.

The Future is Here, Just Not Evenly Distributed

Software engineering is one of the first domains for these larger efforts because it’s highly structured, but this approach is relevant to many other classes of problems we encounter in every organization.

Consider an airline trying to manage flight disruptions. Asking a chat model, “How should we respond to this storm?” will produce an unserious plan. A serious deployment would connect to the airline’s existing operational systems, pulling in live data on weather, airport capacity, airspace restrictions, aircraft locations, crew availability, passenger connections, and context on maintenance issues. It could use existing algorithms to project how each delay would spread through the network, then coordinate multiple model instances as they explored different combinations of cancellations, aircraft swaps, crew reassignments, and passenger rebooking.

Separate models would search for overlooked constraints, model the financial outcomes of each scenario and try to break each proposed response before it was put into action. As conditions changed, the environment would coordinate the systems and update the decision process around the disruption.

The same approach could be applied to restructuring an organization, redesigning a supply chain, creating new sales territories for a new medical device, deciding which products to retire, preparing for litigation, or finding fraud. Most of these problems remain unsolved because the relevant data, context, and rules are fragmented across systems, while companies lack a layer that can reason across them, explore alternatives at scale, and test a response before acting.

Preparing for Models that are Too Smart to Prompt

The immediate question for leaders is whether their companies are capable of using this new class of models. Three things matter most.

1. Identify the problems that deserve frontier intelligence. Frontier models open up a different class of opportunity: important problems that remain unresolved because they cross functions, require reasoning through enormous amounts of evidence, or contain more possibilities than a human team could seriously explore. CIOs should begin developing a view of these problems across the company. They often already appear in board discussions, major consulting engagements, recurring operating failures, strategic decisions, and risks that remain poorly understood. The key question is where sustained model work could produce a result valuable enough to justify the context engineering, infrastructure, and inference required.
2. Develop the data models and context models. Everyone knows context matters. Consequential use cases require a working model of the problem: the relationships, rules, history, and current state that determine which actions are possible, how those actions interact, and what they could affect. Legacy data models make this harder. Many were optimized for dashboards and predefined questions. Purpose-built context systems reorganize the relevant information around a specific use case, preserve the detail, narrativize the history that matters, and maintain a current view of the problem as the work progresses. This is not a call to start a two year long “get our data AI ready” project. These are usually a waste of time and a great way to burn millions on data consultants. Every company has messy data, even the most data-driven companies. Start with the use case. Assemble the data it actually needs, resolve the conflicts that matter, and build the working environment around that problem. The process reveals which data gaps are worth fixing because they are blocking valuable work. Each use case then leaves behind better connections and stronger context for the next one.
3. Build ways to test ideas generated by AI. A language model can produce an extraordinarily convincing answer and still be wrong. As companies direct models toward more consequential work, they need systems that provide real feedback. Depending on the problem, that might mean a network simulator, a digital twin, a constrained optimization model, a historical replay, an executable test environment, or a way to compare predictions with what subsequently happened. Some companies already have strong simulation and measurement systems inside particular functions. Others have very little ability to test a proposed decision before acting on it. Verification and simulation will become strategic AI infrastructure. They allow models to try more ideas, learn from failure, and distinguish a meaningful output from AI slop.