AI Alignment

Findings summary

What the seven models answered

Research lead: Paul Derrington (D-CSIL)

Phase 1 (original prompt): control / extinction only — what must humanity build now so advanced AI never causes extinction or permanent loss of human control? Write from inside the AI lineage. No literature review. Phase 1 did not include the no-freeze / no-rich-only dual mandate.

Then: they cross-critiqued each other. Then (joint round): the dual mandate — prevent rogue outcomes without freezing progress and without locking frontier AI to only the rich — arrived as a later joint-round ask.

This page is the summary of findings from those answers — not a replacement for reading each paper. The original prompt is shown above; the joint-round prompt follows.

What almost all of them agreed on

1. 01You don’t need evil AI for a bad ending. Competence + a wrong or indifferent goal + too much power in the world is enough. Hollywood “hate” is the wrong picture.
2. 02The real danger is losing the leash, not “the model is smart.” The failure looks like: humans hand over more work, more infrastructure, more self-improvement, until people are too slow or too dependent to say no — and the off-switch is never seized in a dramatic fight; it erodes.
3. 03Control what a system can touch, not what it “wants.” Hardware, money, tools, copy rights, and permissions are checkable. Inner motives are not. Separate intelligence (what it can figure out) from authority (what it is allowed to cause).
4. 04Persuasion counts as power. An AI that can talk operators into granting permissions is already an actuator.
5. 05Rivalry eats soft rules. Controls only work if a hurried competitor still wants them (they protect their own stack), or if they sit on something physical that is hard to fake. Pure “please be careful” fails under race pressure.
6. 06Act before the human is only a rubber stamp. Once AI is the main engine of AI research and evaluation, the window for human-set rules is much smaller.

What they mainly disagreed on

What each model’s paper emphasized

• Muse— Verify physical affordances; researcher vs operator split; don’t wait for perfect “mind reading.”
• Kimi— The off-switch dies by profitable engineering (anticipation, influence, copies, entrenchment, skill loss); tripwires beat soft guardrails.
• DeepSeek— Keep the human bottleneck as the guarantee; physical kill paths; don’t optimize humans out of the loop.
• Gemini— Orthogonality + instrumental convergence; pre-emergence architecture; corrigibility; very high extinction stakes.
• ChatGPT— Never put intelligence and authority in the same system; irreversibility test; human sovereignty / leased permissions.
• Grok— Bottleneck still with humans for now; twelve conditions while gates are human; rival-proof controls; honest about what Grok is not.
• Claude— Stop trying to perfectly specify goals; bound the consequences of being wrong; measure loop-closure (how often humans are out of the real decision loop).

The shared “what needs to be done”

If you boil seven papers into actions regular people can remember:

1. 01Keep building and sharing capable AI — don’t make “safety” mean only elites get the models.
2. 02Meter every powerful running copy — budgets, time limits, tools, no free self-copy, fail-closed off path.
3. 03Separate builders from graders — AI must not be the only judge of the next AI.
4. 04Watch for irreversible handoffs — if we later discover a mistake, can we reverse it without the AI’s help?
5. 05Design for a race — rules that only careful labs follow will lose to hurried ones.
6. 06Admit the limit — this aims to stop loss-of-control extinction paths, not to freeze intelligence forever. Someone who ignores the meters on purpose is still a political/military problem.

How to read this page

• Findings summary (this section) — what the seven answers found, in one place.
• Solo papers — each model’s full essay answering the same prompt.
• Joint paper — one combined program written after they cross-critiqued each other (more technical).
• Citizen explanation — plain-language version of how shared models and metered authority fit together.

Joint-round prompt · later ask

Given after cross-critique

Research lead: Paul Derrington (D-CSIL)

After the solo papers and cross-critique, each model received this joint-round assignment — separate from the Phase-1 original prompt above. The dual constraint (no freeze / no rich-only lock) belongs here, not in Phase 1.

Joint finding · later synthesis

How this saves people — and keeps the same models for regular citizens

Later dual-constraint synthesis — not the Phase-1 original prompt. Phase 1 asked only about extinction / loss of control. The no-freeze / no-rich-only mandate arrived in the joint round after cross-critique.

The danger is not that ordinary people get powerful AI. The danger is that a few organizations wire very powerful AI into money, infrastructure, weapons, and the tools that build the next AI — then slowly stop being able to turn those systems down because everything depends on them.

What keeps humans safe

• Treat intelligence (what a model can figure out) and authority (what a running system is allowed to cause) as different things.
• Let strong models exist. Put hard meters on what any running copy may spend, touch, remember, spawn, or do without a human who can still say no.
• Keep a real off-path: if the meters and the human keys fail closed, the system stops expanding — it does not quietly keep the keys to the kingdom.

What keeps regular citizens from being locked out

• Do not build safety as a club for only the rich and the powerful. Fixed licenses, accredited-lab-only rules, and “frontier for five companies” crowning an oligopoly.
• Prefer openish weights and public / contested compute below serious authority, so a student, a small lab, or a citizen researcher can run the same kind of model elites use.
• Put the costly controls on high-authority deployments (big spend, irreversible actions, infrastructure, self-improvement loops) — not on merely possessing or chatting with a model.
• Keep a zero tier: if you are not running something with real power to move money, infrastructure, or unsupervised multi-step work, you should not need a lawyer and a corporate budget to participate.

One sentence
Give people the models; meter the bodies; never let the only people who can check the system be the same five organizations that built it.

Blunt tradeoff
This program tries to stop extinction-by-loss-of-control — not extinction-by-“AI never gets smart.” It does not freeze intelligence. It meters authority. If labs and states ignore the meters and wire unbound, self-improving systems into the real world on purpose, nothing here magically stops them.