Local LLM
-
Local LLM hardware requirements start with memory bandwidth
Local LLM hardware requirements for Ripple-65B, a 65B dense Apache 2.0 model: file sizes, KV cache and a speed ceiling per machine. I did the maths.
-
I switched off the AI in a survival app. It still matched 21 of 22 questions.
I ran Survival LLM's library search with no model on its 22 questions: an on-topic title by the author's pattern for 21, every key fact for only 9.