Ask HN: Cheapest way to run local LLMs?
3 by explosion-s | 0 comments on Hacker News.
Not super knowledgeable about all the different specs of the different Orange PI and Rasberry PI models. I'm looking for something relatively cheap that can connect to WiFi and USB. I want to be able to run at least 13b models at a a decent tok / s. Also open to other solutions. I have a Mac M1 (8gb RAM) and upgrading the computer itself would be cost prohibitive for me.

Post a Comment

Previous Post Next Post