Local LLM Fit and Benchmark Kit
by Hung Le
$5
Check whether a local LLM fits your GPU and how fast it runs. Three standard library Python scripts estimate memory, benchmark Ollama or vLLM, and write a scored report that ranks your models.
1
0
I am a UC Berkeley junior studying data science and applied math. I spend a lot of time running models locally with Ollama, vLLM and SGLang, and I build small tools for the parts that are hard to guess, like whether a model fits on your GPU and how fast it really runs. My