Track map
Detailed content
🔁 The loop: AI building AI
The warning’s central idea in plain English: what RSI is, where the bottleneck shifts, and why this is a warning, not a reason to panic.
RSI is when AI helps design a better AI, which designs the next one, faster. Jack Clark’s (Anthropic) metaphor: “Claude 10 building Claude 11.”
This is the idea behind the whole course. Without it, the figures in later tracks (METR task horizon, MirrorCode) lack context.
RSI; frontier model; the self-reinforcing loop.
Today, progress is limited by how quickly people can think, test, and code. With RSI, the limits shift to compute, infrastructure, and autonomy.
Understanding the bottleneck explains why “more chips and energy” become the variables that determine speed—a Track 3 topic.
Bottleneck; compute; autonomy.
Demis Hassabis (DeepMind): today it is “soft” (AI makes engineers faster), not “hard” (a model disappears into a data center and returns superintelligent).
This separates what is actually happening from a fictional scenario—the line between observation and speculation.
Soft × hard; superintelligence (ASI).
What it means to assign a probability to an idea. The ~60% by 2028 is one person’s calibrated estimate, not a field-wide consensus. (according to the video—unverified).
This trains you to distinguish an honest probabilistic opinion from a guaranteed prediction—the heart of critical reading.
Subjective probability; estimate × consensus; solid evidence vs. hype.
RSI is not a conscious robot waking up, nor is it “the singularity.” It is a bottleneck shift in an engineering process.
This avoids two opposite mistakes: science-fiction panic and dismissing everything as hype.
Singularity; AGI vs. ASI; consciousness.
The course’s tone: informed attention, not certainty or despair. And a map of each track: Evidence (T2) and Safety & 2028 (T3).
This sets the stance that makes everything else useful: read numbers without panicking or ignoring them.
Attention × panic; the solid-vs-hype method; course map.
💻 Why it starts with code
Self-improvement appears first in software because the loop closes in seconds. Learn how agents run on their own and why humans move up a level instead of disappearing.
In software, the cycle is: write → run → check whether it passed → try again—and it happens in seconds.
This is the mechanical reason self-improvement starts with code, not biology or physics.
Feedback loop; short cycle; iteration.
In the physical world, an experiment takes weeks or months (a cell culture, a chemical reaction). In software, the loop closes in seconds.
This speed difference explains WHY self-improvement appears first in code.
Loop speed; experiment cost.
An “agent” is AI that carries out steps on its own: run code, edit a file, test, and try again—with no human at every step.
It is the “worker” that actually keeps the fast feedback loop running for hours.
Agent; execution autonomy; closed loop.
AI proposes a change → tests it → keeps the winner → repeats, searching for solutions faster than humans. The category is well established; the exact name (“Gemini-guided evolutionary agent”) is a claim from the video (unverified).
This shows how the loop scales: many attempts in parallel, with automatic selection of the best.
Evolutionary search; winner selection; parallelism.
Anthropic publicly says Claude writes most code that gets merged. Exact figures (>80%, ~8× per engineer, ~5 months without typing) come from the video (unverified).
This is the strongest concrete example of “soft” RSI running today inside a frontier lab.
Productivity; solid evidence vs. exact figures; internal loop.
The human role shifts from typing to directing, reviewing, and deciding what matters. Humans are not “out”; they are “above.”
It answers the inevitable question (“so, do programmers disappear?”) honestly: the work moves up a level.
Human above, not out; review; direction.