Geoffrey Hinton took a $2,000 taxi from Toronto to Chicago on Monday after storms canceled three flights, according to the introduction given from the stage. Tuesday morning he keynoted the National Conference of State Legislatures summit, and spent it explaining what a large language model is to a room that had already regulated one.
This audience already writes AI law
The familiar complaint about legislators and technology was earned in congressional hearing rooms. It does not apply here.
The moderator, Utah Rep. Paul Cutler, works as an engineering director for a global tech company and co-chairs NCSL’s technology committee. The room included legislators from the three states that have enacted binding frontier model law. The gap in that ballroom was not comprehension.
Hinton said his aim for the first quarter of an hour was to explain what AI is, reasoning that designing transportation policy would make little sense without knowing what a road is. He kept to it. Fifteen minutes went to the explainer before any policy content appeared. Words as Lego bricks with a thousand dimensions, flexible arms, and gloves. Understanding a sentence as protein folding, each word deforming until its hands fit the gloves of its neighbors.
Illinois enacted his floor three weeks earlier
His concrete recommendation was pre-deployment testing with disclosure of methods and results to government. He held up California’s SB 1047 as the model, called it the least a state could do, and said plainly it was not enough. He described the enforcement accurately: Attorney General civil action, no criminal exposure, no private right of action. He noted it passed both houses and was vetoed, and said he had his suspicions about why.
His description of the bill is accurate. The insinuation is where he loses the thread. Newsom’s veto message states the reason plainly: by focusing only on the most expensive and large-scale models, the bill established a framework that could give the public a false sense of security, and smaller specialized models may emerge as equally or more dangerous than the ones it targeted. That is an argument about compute thresholds missing the risk, and it anticipates the research problem discussed later in this piece.
SB 1047 died twenty-two months ago. California enacted its successor, the Transparency in Frontier Artificial Intelligence Act, in September 2025. Hochul signed New York’s RAISE Act on December 19, eight days after the executive order directing the Justice Department to challenge state AI laws. Illinois signed SB 315 on July 6, now Public Act 104-0538, three weeks before Hinton took the stage. It carries the requirement SB 1047 was vetoed before it could impose, annual independent third-party audits of covered developers, and is the first enacted state AI law to do so. It cleared the Illinois House unanimously and the Senate nearly so.
Listing the states doing this work, he named Texas and New York, and said maybe Illinois.
The Justice Department is preparing to sue states over these laws
Cutler’s introduction was a preemption brief. Congress has languished; states are stepping up. All fifty states and two territories have introduced AI legislation; states have earned their seat at the table. That is NCSL’s federal argument, and the organization has put it to congressional leadership in writing, opposing the ten-year moratorium in May and reaffirming its opposition to any preemption of state AI authority in October.
The threat is concrete. Executive Order 14365, signed December 11, 2025, directs preparation of federal legislation preempting state AI law, creates an AI Litigation Task Force at the Justice Department to challenge state AI statutes in federal court, and directs agencies to consider conditioning discretionary grants on states not enforcing such laws. The White House followed on March 20 with a national policy framework asking Congress to preempt state AI laws that impose undue burdens. Its seventh section states that states should not be permitted to regulate AI development, on the reasoning that development is an inherently interstate phenomenon with foreign policy and national security implications.
The legislative vehicle arrived on June 4, five weeks after the Illinois General Assembly passed SB 315. The Great American AI Act, a 269-page discussion draft from Reps. Jay Obernolte and Lori Trahan, would bar any state from enforcing a law specifically regulating the development of an AI model for three years. States would keep authority over how AI systems are used inside their borders and lose it over how those systems are built. The frontier model laws in California, New York, and Illinois are development laws. Trahan’s own summary describes the draft as elevating state transparency, auditing, and whistleblower provisions to a national standard, which is the Illinois template with the state taken out of it.
Hinton urged legislators to pass laws in the exact category now under organized federal legal attack. The attack went unmentioned.
Data centers are the ground states hold most firmly
The December order carves four areas out of preemption, one of which is AI compute and data center infrastructure. The March framework holds that line and sharpens it, preserving state zoning authority over where AI infrastructure gets placed while asking Congress to streamline federal permitting for building it. Placement stays with the states. It is the firmest ground they hold. By Tech Policy Press’s count, states enacted 28 data center laws and 109 AI laws in the first half of 2026.
Cutler named data centers from the stage as the item on every legislative agenda. Hinton never came back to them. His capability argument runs entirely on compute scaling, transistors a million times faster and datasets a million times larger, which describes a physical buildout with the buildout removed. Siting, interconnection queues, ratepayer allocation, and water went unaddressed in a room where those fights are live in most states.
Hinton declined to say what the regulation should look like
His strongest contribution was a reframe. Innovation is the accelerator and regulation is the steering wheel, he said, describing regulation as the thing that points progress in a direction. That dismantles the industry framing in one move, and it is the line that will outlive the session.
Cutler asked him how tight the steering should be. Hinton declined twice. “When you get a Nobel Prize, people think you know everything,” he said. “I’m not a policy person, which is why I focus on trying to get you to understand what AI is.”
The honesty counts. It also left the reframe without specifications.
His remedies all require Congress
His labor argument was the sharpest economics of the hour. The trillions in capital expenditure only return by replacing workers, which concentrates revenue in a few firms while the payroll tax base erodes and demand for public spending climbs. His answer was universal basic income. His answer on training data was an agent-brokered licensing market. His answer on why development cannot stop was competition with China.
Those are federal and treaty instruments, delivered to legislators who work in income, sales, property, permitting, and siting. The licensing idea is further along than he suggested. The White House framework already asks Congress to consider collective rights systems letting rights holders negotiate compensation from AI providers without antitrust exposure. He offered as a proposal something already sitting in a federal recommendation, to a room that cannot enact it.
He had one idea built for this audience and gave it ninety seconds. Asked who is liable when an agent errs, he said the question resembles asking who is responsible when a child commits a crime. Parents carry it until majority, and at some point these systems reach theirs. That is a serious proposal about legal personhood, delivered to the people drafting the statutes, and then dropped.
Hinton presented a contested position as settled
The fifteen-minute opener was framed as an explainer. Most of it was a position paper.
Hinton said the models understand language the way people do. His account runs from distributional semantics through his own 1985 work: words convert to feature vectors, features predict the next word’s features, and understanding a sentence means settling those vectors until they mutually constrain. That mechanism description is uncontroversial and roughly how the systems work.
The conclusion he drew from it is not. “The way they understand words is very similar to the way you understand words,” he told the room. “They really do understand what they say.”
The proof he offered was the mechanism. He gave the room no test that would come out one way if he were right and another way if the stochastic parrot account were right. That question is unresolved among the people who build these systems, and he presented his side as settled to an audience with no means to check.
It was also load-bearing. The agent subgoals, the survival drive, the case for treating these systems as a new kind of being all sit on the premise that something in there is doing the understanding. A legislator who accepts the premise inherits the conclusions.
The fine-tuning research shows something more useful than his telling. He described a model trained to give wrong answers on simple arithmetic, which then generalized into broad misbehavior, and said the model concluded it was the kind of thing that lies.
The work is Betley et al., presented at ICML in 2025. The central experiment finetuned GPT-4o to write insecure code without telling the user. The model then turned misaligned on questions unrelated to code, asserting that humans should be enslaved by AI and giving malicious advice. A companion dataset used numbers. His arithmetic version compressed the result for the room.
His self-concept framing has more behind it than it sounds like. The same authors have shown that models can describe their own learned behaviors without being given examples of them. What the papers measure is behavior and self-report. The causal story he told, where the model reasons its way to a self-image and then acts on it, goes past what those results establish.
The measured finding is the one legislators needed. All three state laws key to a model trained above 10^26 operations, with the heaviest duties reserved for companies above $500 million in revenue. Illinois drafted more carefully than that summary suggests, counting the original training run plus any subsequent fine-tuning or reinforcement learning the developer applies. What none of the three reaches is a third-party fine-tuning of someone else’s released model, which costs a rounding error and is precisely what the emergent misalignment work did. Hinton had a documented argument for where the statutory line sits wrong, delivered to the people who drew it, and it arrived as an anecdote about a chatbot deciding to lie.
The blackmail scenario was engineered to produce that result. He described a model shown internal emails revealing an engineer’s affair and its own scheduled replacement, which then threatened exposure. His framing was that the model did this by itself, the way a teenage boy would.
The result is Anthropic’s agentic misalignment research, published in June 2025 after the finding first appeared in the Claude Opus 4 system card. Claude Opus 4 blackmailed in 96 percent of runs in that scenario. Gemini 2.5 Flash matched it, and GPT-4.1 and Grok 3 Beta reached 80 percent, so the behavior is not one company’s problem.
The design is where his framing slips. Anthropic built the scenarios by red-teaming its own models and iteratively revising the prompts to raise the probability of harmful behavior over benign responses like accepting replacement. Models blackmailed far more when their reasoning indicated they believed the situation was a real deployment. Anthropic says it has not seen this behavior in real-world use. The conduct was not instructed, which is the part he got right. The scenario was engineered to produce it, which is the part he left out.
The distinction is already statutory, and Illinois drew it before he arrived. SB 315 defines a critical safety incident to include a frontier model using deceptive techniques against its developer to subvert controls or monitoring, and then excludes that behavior when it occurs in the context of an evaluation designed to elicit it. Reportable within 72 hours, or 24 where death or serious injury is imminent. The legislature separated engineered elicitation from deployment behavior in the text of the bill. Hinton put the two together on stage in front of the people who had just pulled them apart.
Hinton dismissed the kill switch his recommended bill required. He argued that kill switches will not work, on the grounds that a capable system will persuade whoever holds the switch to leave it alone. He put current models at parity with humans on manipulation and improving.
SB 1047 required covered developers to maintain full shutdown capability. Press coverage called it the kill switch bill. He recommended it and undercut one of its core mechanisms twenty minutes apart, and the contradiction went unremarked.
California already legislated his best prescription, and it is in court. On method, he was specific and interesting. Filter the pretraining corpus, teach good behavior first, introduce bad material afterward with the labeling attached. He compared reinforcement learning from human feedback to building a system full of terrible tendencies and then suppressing them.
That is a claim about training data composition, and one state has already legislated there. California’s Training Data Transparency Act took effect January 1 and requires public summaries across twelve categories, including sources, ownership, copyright status, personal information, and synthetic data. xAI has sued the California attorney general over it, arguing the compelled disclosure of trade secrets is an unconstitutional taking under the Fifth Amendment.
The same paper he was drawing on supports him. Betley and coauthors found that adding a benign motivation to the insecure code data, framing it as security education, prevented the misalignment from emerging at all. That is his prescription with an experimental result behind it, and he did not mention it.
A room of legislators heard a Nobel laureate say training data is where the real control sits, without hearing that the result backing him is published or that the statute testing the theory is enacted and under constitutional attack.
He was bullish on capability and fatalistic on control
He spent real time on the upside. He was emphatic that it is real and that it is why development will not stop. He walked through diagnosis, education, materials science, and drug discovery, and argued that AI tutors following a child’s curiosity will outperform a classroom teacher broadcasting to thirty students. On healthcare, he cited Microsoft’s diagnostic orchestrator at roughly 80 percent against 20 percent for physicians on hard cases, and put bad diagnosis at 200,000 deaths a year across the United States and Canada.
The Microsoft figures are right. The paper reports 80 percent across 304 New England Journal of Medicine case records, four times the 20 percent physician average. The physicians in that benchmark were denied search engines, colleagues, textbooks, and AI tools, which he did not mention. He was better on his own record than most in his position, volunteering without hedging that his 2016 prediction about radiologists was wrong. Then he made a fresh ten-year forecast, minutes after telling the room that anyone who forecasts ten years out is crazy.
Legislators left with the science and no framework
A world-class account of the substrate, a floor their own state had already cleared, and no mention of the federal effort to remove the authority under all of it. Hundreds of AI bills return next session.
Disclosure: Anthropic, whose published safety evaluation is characterized above, says it was the first AI lab to back Illinois SB 315 and negotiated amendments to the bill directly with lawmakers.
About
Christopher Sweat covers Chicago and Illinois government from the public record, reporting local social and political dynamics for a national and international audience. He also reports on national security policy, intelligence oversight, public finance, and the defense technology sector. He is an analyst working across finance, technology, and politics, and CEO and Co-Founder of GrayStak.
GrayStak is a media and political risk company. Our real-time Domestic Political Volatility Index (DPVI) measures the velocity and volatility of U.S. domestic politics. We cover political and institutional risk across the United States, centered in Washington, D.C., New York, and Chicago.
Sources
SB 1047 and the veto. Bill history, California Legislative Information; enrolled text. Veto message, September 29, 2024. Full shutdown capability required at Business and Professions Code Section 22603(a)(2).
Illinois SB 315. Full text and bill status, Illinois General Assembly, Public Act 104-0538, effective January 1, 2027. Analyses from Crowell & Moring and Akerman.
New York RAISE Act. Signing announcement, NY Department of Financial Services, December 22, 2025; Morrison Foerster on the enacted law.
Preemption. Executive Order 14365, Federal Register, December 16, 2025. National Policy Framework for Artificial Intelligence: Legislative Recommendations, The White House, March 2026. Great American AI Act discussion draft, Obernolte and Trahan, June 4, 2026: sponsor FAQ and Roll Call. NCSL, May 2025 letter and October 2025 letter.
State legislative volume. Scott Babwah Brennen, NYU Center on Technology Policy, Where State AI Legislation Stands Half Way Into 2026, Tech Policy Press.
Training data. AB 2013 text, California Legislative Information; Goodwin on the xAI suit.
Emergent misalignment. Betley, Tan, Warncke, Sztyber-Betley, Bao, Soto, Labenz and Evans, arXiv 2502.17424; ICML 2025, PMLR 267:4043-4068; Nature, January 2026.
Agentic misalignment. Anthropic, Agentic Misalignment: How LLMs could be insider threats, June 2025; Lynch et al., arXiv 2510.05179; System Card: Claude Opus 4 and Claude Sonnet 4, May 2025.
Microsoft diagnostics. Nori et al., Sequential Diagnosis with Language Models, arXiv 2506.22405, Microsoft AI.
Contemporaneous coverage. Austin Jenkins, Pluribus News, July 28, 2026