Hardware performance running LLMs from www.ollama.ai

CPU cores GPU RAM VRAM operating system prompt eval rate eval rate model name reporter report
Intel(R) Core(TM) i7-6500U CPU @ 2.50GHz Intel Corporation Skylake GT2 [HD Graphics 520] (rev 07) 16 GB 4 GB 2.09 1.69 llama2 VeselinBitPie
Intel(R) Core(TM) i7-6500U CPU @ 2.50GHz Intel Corporation Skylake GT2 [HD Graphics 520] (rev 07) 16 GB 4 GB 1.75 1.2 llama2 VeselinBitPie
Intel(R) Core(TM) i7-6500U CPU @ 2.50GHz Intel Corporation Skylake GT2 [HD Graphics 520] (rev 07) 16 GB 4 GB 1.63 0.83 llama2 VeselinAdmin
11th Gen Intel 4-Core i3-1125G4 2 Ghz Intel UHD Graphics 32 GB 2.22 1.05 mixtral vincecate
CPU 12th Gen Intel(R) Core(TM) i9-12900K Nvidia RTX A6000 128 GB 48 GB 91.85 35.37 mixtral:latest vincecate
12th Gen Intel(R) Core(TM) i9-12900K Nvidia RTX A6000 128 GB 48 GB 277.06 7.54 llama2:70b vincecate
12th Gen Intel(R) Core(TM) i9-12900K Nvidia RTX A6000 128 GB 48 GB 24.18 0.4 falcon:180b-chat vincecate
12th Gen Intel(R) Core(TM) i9-12900K Nvidia RTX A6000 128 GB 48 GB 12.86 5.79 llama2:7b vincecate
12th Gen Intel(R) Core(TM) i9-12900K Nvidia RTX A6000 128 GB 48 GB 1828.01 38.31 mistral:7b-ins16 vincecate
12th Gen Intel(R) Core(TM) i9-12900K Nvidia RTX A6000 128 GB 48 GB 559.88 25.32 vicuna:33b vincecate
12th Gen Intel(R) Core(TM) i9-12900K Nvidia RTX A6000 128 GB 48 GB 1594.66 22.49 orca2:13b-fp16 vincecate
12th Gen Intel(R) Core(TM) i9-12900K Nvidia RTX A6000 128 GB 48 GB WSL/Ubuntu 0.97 0.39 megadolphin vincecate
12th Gen Intel(R) Core(TM) i9-12900K Nvidia RTX A6000 128 GB 48 GB WSL/Ubuntu 427.21 56.07 openchat:7b-v3.5-q8_0 vincecate
12th Gen Intel(R) Core(TM) i9-12900K Nvidia RTX A6000 128 GB 48 GB 7.93 2.18 mixtral:8x7b-instruct-v0.1-q8_0 vincecate
12th Gen Intel(R) Core(TM) i9-12900K Nvidia RTX A6000 128 GB 48 GB WSL/Ubuntu 76.51 29.78 mixtral:8x7b-instruct-v0.1-q4_1 vincecate
12th Gen Intel(R) Core(TM) i9-12900K Nvidia RTX A6000 128 GB 48 GB WSL/Ubuntu 91.62 34.49 nous-hermes2-mixtral:latest vincecate
AMD Threadripper 64 Nvidia RTX A6000 768 GB 48 GB Ubuntu 31.27 1.17 --verbose vincecate