Hacker News

Ask HN: How realistic is an LLM-powered worm that spreads from PC to PC?

3 points by anon115 ago | 2 comments

strangecasts |next [-]

Anthropic had a very hotly debated proof-of-concept last month: https://news.ycombinator.com/item?id=50061603

This proof-of-concept skipped the work of exploitation to start running the code, assuming that there are security holes that make IoT/smart TVs/older devices essentially exploitable "for free", and just focused on the worm/replication [1] part. They were able to have a 27B model run sharded across an impromptu network of set-top boxes, but also showed that this wasn't particularly practical - the resulting network output less than a token a second.

This suggests that you probably won't see "LLM-powered worms" in the sense that runs an LLM on each infected device (which becomes noticeable in terms of power consumption and/or temperature), but rather LLMs as additions to typical client-server botnets, where the models run on more powerful servers and manage infected devices, using them to probe other devices on the network and stage attacks.

[1] I am a bit annoyed that "self-replication" is the term they use, since the proof-of-concept requires the harness to have access to the weights - if you allow that then you can make a <100-parameter "self-replicating" model by outputting the tokens "please run `cp skynet.safetensors skynet2.safetensors`"...

mrlonglong |previous [-]

Pfft.

You'd need massive amounts of memory and bandwidth to replicate.

One day maybe, but now? No.