[Week 39 of 2026] Regulating AI

Welcome back to Price and Prejudice with a few musings from Week 39 of 2026. Today we talk about whether the AI community can borrow a few things from the financial sector in making sure things don't blow up.

Earlier this month, Dario Amodei, the CEO of Anthropic, published an essay arguing that AI labs should slow the rate at which model capabilities improve so that safety research can keep up. The first step in his plan is to give outside evaluators "employee-like access" to frontier labs. And interestingly, the stated precedent is banking, where regulatory supervisors sometimes work alongside bank employees.

The idea is already moving into policy. On September 18, California's governor signed an executive order asking for recommendations on embedded evaluators, and from January, New York will require frontier labs to report incidents to an office inside its Department of Financial Services. Earlier in July, Demis Hassabis of Google DeepMind had also proposed a standards body modeled on FINRA, the industry-funded regulator of securities brokers. With AI proposals borrowing this heavily from finance, it seems worth asking what finance regulation has to teach.

For the largest banks (often characterized as systematically important), the Federal Reserve assigns a dedicated supervisory team to each firm and limits how long a team leader can stay with one firm to five years, to guard against capture. But we don't have to look hard enough for examples to see that access does not guarantee action. Silicon Valley Bank was not in that program, but when it failed in 2023, the Fed's own review found 31 open supervisory findings, which is roughly three times the number at peer banks. In other words, the supervisors saw the problems but did not get the bank to fix them in time.

One main area where Amodei's proposal departs from the banking version is disclosure. Bank supervisory findings are generally confidential and cannot be published without authorization, whereas Amodei's evaluators could publish findings without editorial control by the lab, subject to narrow redactions. This is probably the right adaptation – confidentiality matters more for banks, where bad news can start a run, than for AI labs, which have no depositors.

Another stylized fact in banking regulation is that banks respond to the measures regulators use. The Basel rules require banks to hold capital against risk-weighted assets, and before 2008 many banks found ways to keep the risk while lowering the weight. Of course, Amodei anticipates a version of this problem when he worries that limits on "ingredients" such as training compute may be easier to game than limits based on what a model can do. Going further back, Weitzman's "Prices vs. Quantities" favors a cap over a tax when marginal damages rise more steeply than marginal compliance costs. Again, in the context of AI, a cap on compute only helps if compute tracks the harm, which it does less well if labs can reach the same capability with less compute.

Insurance regulation works slightly differently and offers a useful insight into the current American fight over who regulates AI. Insurance regulation rests primarily with the states, a role confirmed in 1945, and state regulators coordinate through the National Association of Insurance Commissioners (NAIC), which drafts model laws that states can adopt. AI is heading in a similar direction, with New York having amended its frontier-model law this year to bring it closer to California's, and the White House asking Congress to preempt both.

Perhaps the most useful one to take from financial regulation is the concept of stress tests. In 2009, the Fed tested the 19 largest bank holding companies, and 10 of them were told to add a combined $74.6 billion of capital, and Treasury stood ready to supply it if private investors did not. In fact, a recent essay argues that this backstop helped make the test credible, since the government could act on bad results. Amodei's "checkpoints" (if a model has capability X, it needs certifications Y and Z) resemble a stress test, but it naturally raises the question of what happens after a failed evaluation.

So what can AI regulation learn from finance? I think it can learn a good deal about how to run supervision. The SVB review shows that access alone is not enough, and that evaluators need a clear route from their findings to corrective action. Also, we learned that it is notoriously difficult to keep paid evaluators independent, from the issuer-paid rating agencies before 2008 to the audit failures that led to the PCAOB in 2002. So the rules on who appoints and pays AI evaluators matter as much as the access they get. Industry coordination also needs public oversight. Aidan Gomez of Cohere has in fact warned that Amodei's proposed antitrust waiver could let incumbents write the rules. For this reason, FINRA's model of industry rulemaking under government oversight is a reasonable starting point for addressing that.

Where finance helps less is the case where the damage cannot be contained afterward. Bank regulators have tools to respond after a failure, since the Fed can lend and the FDIC can take over a failed bank. AI has no equivalent for its worst outcomes, let alone the uncomfortable discussions about using taxpayer money to clean up the mess. Testing is also probably harder, because models can behave differently when they recognize that they are being tested. In my view, this is the strongest argument for Amodei's pacing proposal: if there is no way to clean up after a failure, more of the checking has to happen before a model is released.