if you have a single commodity machine, the cited solutions are useful for deploying small models. If you have several commodity machines, you can’t combine them efficiently with the cited solutions to deploy a large model, and even if you could, it would require a team to manage and maintain the system.

You’re wrong and OP is right. Llama.cpp has the ability to do exactly what “Anyway System” claim to do without the bullshit. Like this claim of " even if you could, without a team to manage it" is so stupid.

There are several frameworks that allow this beside Llama.cpp that are open source, that have been for a while, is extremely maintained.

Also, this seems like a stunt to get VC money. Good on the founders I guess. But the solution from what I read is a nothing burger. My money on llama.cpp or ktransformers or ik_llama.cpp

For me the worst part is that someone developed the functionality to monitor and track, until the signal is lost, and if so, kill. It’s really crazy how daring this is.

Mandatory respect for that guy. He left too soon.