GitHub - magnitudedev/magnitude: Open source inference engine that tunes its kernels on your …

via RustTrending (@RustTrending@yuri.garden): magnitudedev / magnitude: Open source inference engine that tunes its kernels on your hardware to find the fastest way to run each model. 2x faster than Ollama. Works on Apple Silicon, NVIDIA, AMD, Intel, or just a CPU. ★7588 https://github.com/magnitudedev/magnitude

Magnitude is an open-source local inference engine that auto-tunes its kernels to whatever hardware you’re running — Apple Silicon, NVIDIA, AMD, Intel, or plain CPU — and claims roughly 2x the throughput of Ollama. If you run models locally, it’s worth benchmarking against your current setup.

GitHub - magnitudedev/magnitude: Open source inference engine that tunes its kernels on your …