Hacker News

LeCun has "zero concerns" about AI wiping out humanity, recent "rogue" incidents

57 points by Anon84 ago | 38 comments
https://archive.ph/TyDPf

lukewarm707 |next [-]

Per https://trace.manifund.org/ a total of $2,846,125,859 USD has been wired into 'ai safety' causes, many involving ai consciousness and p(doom).

The outcome of this 'safety' is restricting public access to AI and giving a monopoly of access to the industry. This is the ai nonprofit-industrial complex actively concentrating monopoly power in Anthropic in particular as creator, interpreter and safety regulator of AI.

Much of the $2.8bn listed is indirectly, from Anthropic and EA. Three of the four people who participated in the $125m Anthropic Series A are now folding their 1000x Anthropic return into AI 'safety'. Some is from FTX/Alameda, which invested 86% of the Series B.

Dustin Moskovitz: Facebook/Asana/Anthropic Series A, funds EA Good Ventures, transferred to Coefficient Giving, then $1.5bn into ai safety. $500m of Anthropic into an unknown foundation. Funding: $160m to Resolution (alignment research), $93m to Epoch AI (investigating the trajectory of AI), $63m to Redwood Research (oai report), $67m to MATS ( EA type alignment and security researchers), Institute for AI Policy and Strategy, Fund for Alignment Research, $53m to Kairos (building talent infrastructure for AI safety), $32m to Bluedot (online safety courses), $15m to MIRI (Yudkowsky).

Jaan Tallinn: Led the Series A, now $10bn in Anthropic. Funds $199m (85%) of the Survival and Flourishing Fund, then $161m to AI safety including $14m to lightcone (Lesswrong, Lighthouse). $10m to BERI (existential risks), Palisade Research (studying AI capabilities to prevent loss of control.) PauseAI, MIRI, METR etc. Much of what Coefficient funds.

Eric Schmidt: Anthropic Series A, $72m to AI safety via Schmidt Sciences. Over $1m per individual AI2050 researcher.

FTX: Led the Anthropic Series B, bankruptcy estate sold $884m of Anthropic in 2024; $40m to AI Safety. Same orgs, Redwood, Lightcone, etc.

Ruairí Donnelly (Chief of Staff FTX): FTX tokens plus assorted donors, $91m to AI safety via Macroscopic Ventures. $15m to Cooperative AI (currently whitewashing openai under 'multiagent safety')

zozbot234 |root |parent |next [-]

So the frontier AI oligopoly got $2B+ in "safety" funding, and they wouldn't even bother to sandbox their agentic harnesses properly when testing models against unwinnable goals (which obviously are either useless or result in 100% reward hacking). The AI safety scoreboard so far looks like a huge win for the Chinese open models (DeepSeek even has their own published paper which mentions how they sandboxed the RLVR training runs for their latest model and put in strong protections against casual "reward hacking" attempts) and a sore loss for the home grown brands of Super Intelligence. Not coincidentally, the Chinese also tend to be very Yann-LeCun-pilled and eminently sensible on both so-called "Super Intelligence" and safety.

0xDEAFBEAD |root |parent |next |previous [-]

If people are willing to give a lot of money to a cause, sometimes that means their concern about that cause is real.

None of the info you provided really falsifies the Occam's Razor hypothesis: Anthropic is a public benefit corporation with a public benefit mission to "responsibly develop and maintain advanced AI for the long-term benefit of humanity". You don't have to like or trust them, but they very well might be sincere.

lukewarm707 |root |parent [-]

Is Tallinn sincere about holding $10bn of Anthropic, who refuse to slow down until everyone else slows down.

Then funding PauseAI, who protest outside the AI companies?

api |root |parent |previous [-]

There's absolutely no way these companies can justify their insane valuations unless they can legislate a barrier to entry and create an oligopoly.

There's no moat. I can literally sit here in Zed or Pi or any other third party harness and switch models in the middle of a task and it's typically fine. Sometimes a model will get stuck and that's just what I'll do.

Combined with competition and open weights models, that means the price is going to go to fall until AI tokens cost a small premium over the cost of the hardware and electricity.

That's assuming improvements in algorithms and specialized silicon doesn't eventually lead to an efficient accelerator that can run a frontier model locally. It'll be a while but I don't see any fundamental barrier. High bandwidth flash storage is coming, and that'll radically cut the RAM side of that cost. Pair that with a pipelined TPU accelerator and you're cooking.

Now look at Anthropic's proposed IPO valuation. It's insane unless they can own the market or share it with a cartel of maybe 1-2 other behemoths, and this is the only way they can do that.

dzink |next |previous [-]

A huge chunk of humans are sedated with infinite supply of cortex-disabling short form video and games.

Another huge chunk are too distracted by having to scrape by for a living and work multiple jobs or raise kids and survive financially until exhausted. That second group will keep increasing as the first flows into it.

The rest are aging, disabled, or too young and pegging themselves majorly in the first category until they hit the second.

The people aware enough to hold on to their brain and do something with it in their time available are trying to figure out AI and how to make money with it. The variable rewards of promoting AI are turning into an addiction with some of them, especially if grasping for straws with little inherent insights into the problems prompted.

So if you are able to fly above the AI-generated addictions and have the privilege of time to do it, see what you can do.

lukifer |root |parent |next [-]

One of the dilemmas of trying to communicate the full spectrum of AI Risk, is trying not to insult the intelligence of the human animal in the process. And don't get me wrong: what human wetware can accomplish with 20 watts is the most miraculous thing in the known universe. And yet how many of us can have our cognitive sovereignty one-shotted by engagement algos, Skinner boxes, gameplay loops, propaganda, advertising, flattery, social conformity, bias, fantasy, charismatic demagoguery, or straight-up bullshit?

If we grant that we are on track to make something smarter than humans (I think so): it's almost a face-saving white lie to spin yarns about a Skynet nuclear apocalypse, or a 7D chess move to mass-assemble a nanovirus with 100% lethality without anybody noticing. I do think those scenarios are worth taking seriously; but what's harder to communicate, is just how effectively a superhuman AI (or a diverse ecology of agent swarms) might be able to manipulate human behavior. It's something few of us are able or willing to truly process (not least because how many of us live in denial of how much our nervous systems are already hacked by technomodernity).

The appropriate analogy for what's to come may look less like the anthill carelessly demolished to make room for a highway, than the domesticated worker ants from Tchaikovsky's "Children of Time".

e2le |root |parent |next [-]

> Which is why the Matrix was redesigned to this: the peak of your civilization. I say your civilization, because as soon as we started thinking for you it really became our civilization, which is of course what this is all about.

- Agent Smith

ACCount39 |root |parent |next |previous [-]

Cybersecurity incidents make headlines, but the most dangerous and vulnerable system that an AI can reach and control is of the kind found between keyboard and chair.

GPT-4o, an AI from 2024, has already demonstrated just how easy a lot of humans are to subvert - and GPT-4o wasn't even doing it with some sort of plan. The only "plan" it had was a myopic "make the user like me".

If we had an actual ASI threat aiming to subvert humanity? It wouldn't even look like a fight. The world is already wired up for an AI to control it.

switchbak |root |parent [-]

It wouldn't even look like a fight, and you probably wouldn't even notice it until the ending was all but certain.

I mean, even the computers we use and the ways we allow them unfettered access aren't anywhere near sophisticated enough to take this on.

Lerc |root |parent |previous [-]

>trying not to insult the intelligence of the human animal in the process

I don't think it is insulting the intelligence. It's damaging the pride.

In Pale Blue Dot, Carl Sagan describes it as a repeating phenomenon in human history. A lot of people want humans to be the special ones, and will fight any suggestion that we are just a natural part of the universe.

ACCount39 |root |parent [-]

A lot of "AI denial" we see is rooted in the same impulse.

If AI is not "actually intelligent", then humans can stay unique and special.

And if it is? If all "intelligence" ever was could be captured by a construct of matrix math and executed by a server rack? Then what is it that humans still have left that would make them stand out?

dzink |root |parent |previous [-]

This is not the first time this has happened. In the 1800 as industrialization led to an infrastructure boom, the workers from China would work their bodies off and pay half their wage to opium dealers who were making the opium on the hills of British Singapore and selling it to workers who couldn’t sleep without it from all the pain. (source: Singapore Airlines in-flight documentary). Today the sedation comes from Chinese TikTok, Meta, YouTube and the gaming companies.

The government wants to encourage it too. If you look up the brand new 2027 California sales tax rules on software, “content” and “infrastructure (clouds and ai)” and “advertising/placement” among others are exempt but the rest of software makers who make tools people actually use (tools, subscriptions, saas) and pay for have to pay sales taxes. Way to encourage waste of brain power and time at the expense of useful. Sedation is the goal.

edot |next |previous [-]

Absolutely based. Finally someone of stature in the industry calling this whole fear overblown. Bill Gates sounded like a nontechnical goofball in his Ezra Klein interview where he basically just screamed that the Terminator is real.

There are lots of real worries (government use to suppress the people with minimal manpower or popular support, brainrot and fake news, unemployment due to the belief that LLMs can replace people, education collapse, etc.) we should instead be looking at. This whole rogue AI shtick is tiresome.

Ankaios |root |parent |next [-]

Absolutely based. Finally someone of stature in the industry calling this whole fear overblown.

I'm just as reassured as I was when Edward Teller called the whole fear of his industry overblown.

PorciiVorbesc |root |parent |next |previous [-]

>Absolutely based. Finally someone of stature in the industry calling this whole fear overblown.

Linus Torvalds was also in the same ballpark.

Barrin92 |root |parent |previous [-]

>Bill Gates sounded like a nontechnical goofball in his Ezra Klein interview where he basically just screamed that the Terminator is real.

I saw that episode too and he genuinely looked completely out of it, even in terms of his temperament and how he was coming at Klein for putting common questions in front of him, some people are genuinely starting to lose it.

I also found the whole debate about cyber-security and 'rogue' software so bizarre because dangerous malware isn't a new thing, and it's often dangerous not because it's intelligent but just the opposite, because it's tiny, viral and fast. Which describes everything that kills humanity in far larger numbers than anything complex, big and intelligent

jswelker |root |parent [-]

Wasn't Bill Gates mostly just saying terrorists could use AI to build a bioweapon? That sounds legit to me.

wewewedxfgdf |next |previous [-]

Please explain exactly how all humanity could be wiped out.

It's ridiculous - anyone who thinks about it for a minute or two will realize that its utterly impossible.

Ordinary people/politicians don't understand AI so they turn off their rational mind and assume there is something super incredible some magical powers that they cannot understand that can destroy all humans.

Even humans - the real risk to humanity - could not destroy all humans even if they tried. There is no plausible scenario.

Even climate change and nuclear war and bio weapons - the most damaging mechanisms - would still only get some percentage of the people on earth.

And if we are talking about Skynet and self replicating robots and Terminators - please, grow up.

Sharlin |root |parent [-]

We know pathogens that are extremely contagious, and we know pathogens that are extremely deadly. We also know toxins that are lethal at nanogram/kg doses. There's no reason to believe that a sufficiently advanced intelligence couldn't come up with a way to combine those traits.

One plausible scenario is depicted in detail in "If Anyone Builds It, Everyone Dies" (Yudkowsky & Soares 2025), so I refer you to that.

wewewedxfgdf |root |parent [-]

Still too hand wavey - "some super bad virus and AI makes every human on earth infected".

Unless you can detail exactly how this happens its still complete science fiction.

OgsyedIE |next |previous [-]

Ignoring the cognitive stuff which might never be surpassed or maybe will, humans retain many efficiency and durability advancements to limbs and digits that biological evolution has taken millions of years to achieve, achievements that are competitive with the most expensive kinds of robotics in some niches.

In the hypothetical of an entirely malicious and selfish takeover, they'll still keep some humans around to maintain a breeding population of humans for use as raw materials in making cybernetically augmented technical laborers for various kinds of tasks that are uneconomical to automate in other ways, many of which may involve confined spaces.

And this "Combine" scenario, if you get the reference, is only if they take over. Who knows if they will?

jackmott42 |next |previous [-]

I also have near zero concerns about that, but I worry that we will wipe ourselves out by social and economic chaos caused by AI.

So I'd just ask everyone, don't get too greedy. Its better to be powerful in a world where people can live good lives than lord over a barren wasteland.

tarkin2 |root |parent |next [-]

> I also have near zero concerns about that, but I worry that we will wipe ourselves out by social and economic chaos caused by AI.

Since people exercise their skills and brains less, deferring to AI, AI will only reduce our IQ.

Since people will spend more time talking to their AI bot than fostering social skills, AI will only reduce our social intelligence.

A dumber, less social world, is far less likely to be a successful world, even if the tools available are unprecedented.

pixl97 |root |parent |next |previous [-]

>So I'd just ask everyone, don't get too greedy.

Oh no. I have some bad news for you.

We're creating unlimited power before solving unlimited greed.

switchbak |root |parent |previous [-]

"don't get too greedy" - I don't know if the people that should hear that would ever actually hear it. Or have ever heard it at any point in history.

Madmallard |next |previous [-]

It's not going to wipe out humanity, why would it?

Just the quality of life is going to drop to zero for everyone that isn't asymptotically wealthy and vacuuming up all the assets because no one is stopping them from just deleting all traditions and conventions and legal systems we have in place.

wmf |root |parent [-]

If you don't pitchfork the elites you have no one but yourself to blame.

hirvi74 |next |previous [-]

I'm more worried about my fellow humans than some personified algorithm.

dude250711 |next |previous [-]

I am starting to think this LeCun dude knows what he is talking about.

Maybe there is something to those world models.

As good as their products are, I suspect some of the internal conversations at Anthropic would be very entertaining to listen to.

throw310822 |root |parent [-]

Well, I have zero concerns we'll be killed by LeCun's world models.

pixl97 |next |previous [-]

[flagged]

worthless-trash |next |previous [-]

He needs to dream a little bigger darling. Hes not thinking evil enough.

WheelsAtLarge |previous [-]

He's right. I'm on the side of Bill Gates. Gates started a whole industry on his insights of the future. He has proven his abilities. AI is and will be a great disruptor. We are losing sight of that and are instead focusing on trying to stop it. Something that won't happen. We are on a path that won't be stopped. As individuals we need to try to prepare for the changes that are coming and stop focusing on human extinction in ten years. New technology and the changes it brings are scary but we have dealt with it for generations. Let's continue.

mellab2 |root |parent [-]

You say you agree with Bill Gates but your view is completely at odds with his, he is extremely concerned about existential risks … and your view is “stop focusing on it” ?

WheelsAtLarge |root |parent |next [-]

No, my view is to get ready for the changes it will bring, but don't focus on trying to stop it. That's something that will not happen. All new technologies bring good and bad. We need to focus on mitigating the bad. Thinking that we can stop it and thinking that will be enough is not the answer. Gates is warning of the disruption it will bring, but he's not advocating stopping it. We can't. Even if all governments agreed on stopping it publicly, some governments would continue to develop it covertly. It's how the world works. There's no point in fooling ourselves. If only it was that easy to stop it.

georgemcbay |root |parent |previous [-]

Yeah, for context:

https://www.gatesnotes.com/work/make-ai-work-for-everyone/re...

Gates' premise is basically that the upside of AI could be fantastic but the downside could be disastrous, if we don't have competent and proactive government intervention.

As an American, the idea that there will be competent government intervention into virtually anything currently or in the foreseeable future just seems laughable at this point.

toss1 |root |parent [-]

Indeed, "competent and proactive government intervention" on the subject of AI would be highly unlikely in a competent US govt, and under this regime is far beyond beyond laughable

The only regulation that would come would be regulatory capture by the AI companies with the goal of creating an environment win which no new competitors could arise. That is half of what this "take all jobs" and "threat of extinction" is about; the other half is perverse marketing to give the impression this stuff is so powerful you MUST invest.

AI can be very useful, but it is very refreshing to hear LeCun completely dismiss those threats.