Hardware

Resources on deciding what hardware to use for powering your local LLMs.

Relatively maintained resources:

• Tim Dettmers keeps a relatively up-to-date guide of recommendations: Which GPU(s) to Get for Deep Learning: My Experience and Advice for Using GPUs in Deep Learning
• LLM Utils is a documentation resource similar to llm-tracker that has a dedicated subsite GPU Utils that has GPU recommendations and requirements for various use-cases. 
• r/LocalLLaMA Wiki Models has a list of memory requirements for different model types, parameters, quantizes

Possibly out of date articles:

• 2023-07-06 LLaMa 65B GPU benchmarks - comparison of 3090, 4090, A6000, A600 Ada w/ exllama and llama.cpp
• 2023-06-30 MosaicML compares training with MI250X vs A100 and H100
• 2023-06-15 r/LocalLLaMA: Free GPU options for LlaMA model experimentation