This is definitely in line with what I've been thinking. I keep seeing Musk with this. I wouldn't be surprised if he already has something close with all the compute he's been throwing at it. Deployed at swinging elections, as we saw he wasn't afraid to do in '24. Isn't he the one who said empathy is a weakness?
Now that everybody knows its possible, everybody will have it as soon as they can scale. The outcome I see is a magnifying glass on the current bad behavior - the exceptionally wealthy able to control the complete narrative and skew all legislation to give them even more wealth/power with no concern for (empathy again) the rest of the population.
This is a sharp piece, and your core observation that the bottleneck has shifted from insight to capital is spot on. I’d push it one level deeper.
The reason there’s no Tony Stark isn’t a personnel problem. It’s a selection problem.
Once the persistence condition for these labs becomes “acquire and deploy compute faster than competitors,” the system has a single basin of attraction: maximize throughput, minimize self-imposed friction. Everyone converges toward the same operational profile because anything else gets outcompeted.
That’s what makes this structurally intractable. It’s not that bad actors take over. It’s that the environment filters out any strategy that can’t keep up with unconstrained scaling. Caution and interpretability don’t disappear because people abandon them. They disappear because they don’t contribute to survival under the current conditions.
So the trajectory doesn’t change by finding a better actor inside the game. It only changes if the conditions of survival change.
Fully agree. I don’t think there’s a “Tony” out there that could save the situation at this point. the system is more or less on rails. Someone could, at least in theory, do some things to even the playing field at BEST, but even that would come with an absolutely massive level of chaos that I’m not sure would be better. It WOULD be be interesting than letting a handful of rich asshats decide the shape of the world’s economy in the next… couple years? ish? But… it wouldn’t suddenly fix things if someone with the resources decided they wanted to be the world’s Tony Stark. Which is it’s own sort of depressing… no clean answers on this one I think. just “bad” and “worse”. Which… is pretty rough.
This is definitely in line with what I've been thinking. I keep seeing Musk with this. I wouldn't be surprised if he already has something close with all the compute he's been throwing at it. Deployed at swinging elections, as we saw he wasn't afraid to do in '24. Isn't he the one who said empathy is a weakness?
Now that everybody knows its possible, everybody will have it as soon as they can scale. The outcome I see is a magnifying glass on the current bad behavior - the exceptionally wealthy able to control the complete narrative and skew all legislation to give them even more wealth/power with no concern for (empathy again) the rest of the population.
This is a sharp piece, and your core observation that the bottleneck has shifted from insight to capital is spot on. I’d push it one level deeper.
The reason there’s no Tony Stark isn’t a personnel problem. It’s a selection problem.
Once the persistence condition for these labs becomes “acquire and deploy compute faster than competitors,” the system has a single basin of attraction: maximize throughput, minimize self-imposed friction. Everyone converges toward the same operational profile because anything else gets outcompeted.
That’s what makes this structurally intractable. It’s not that bad actors take over. It’s that the environment filters out any strategy that can’t keep up with unconstrained scaling. Caution and interpretability don’t disappear because people abandon them. They disappear because they don’t contribute to survival under the current conditions.
So the trajectory doesn’t change by finding a better actor inside the game. It only changes if the conditions of survival change.
Fully agree. I don’t think there’s a “Tony” out there that could save the situation at this point. the system is more or less on rails. Someone could, at least in theory, do some things to even the playing field at BEST, but even that would come with an absolutely massive level of chaos that I’m not sure would be better. It WOULD be be interesting than letting a handful of rich asshats decide the shape of the world’s economy in the next… couple years? ish? But… it wouldn’t suddenly fix things if someone with the resources decided they wanted to be the world’s Tony Stark. Which is it’s own sort of depressing… no clean answers on this one I think. just “bad” and “worse”. Which… is pretty rough.