Large Language Models (LLM) are a type of generative AI that power chatbot systems like ChatGPT.

You can try many of these for free if you’ve never tried one (although most of this site is aimed at those with more familiarity with these types of systems).

All of these have free access, although many may require user registration, and remember, all data is being sent online to third parties so don’t say anything you’d want to keep very private.

Hosted Services

Proprietary Model Chat

  • OpenAI ChatGPT - the free version uses 3.5, and is not as smart, but good for dipping your toe in.
  • Anthropic Claude - this is an interesting alternative, has super long context window (short-term memory) so can interact w/ very long files for summarization, etc.
  • Google Bard - now running Gemini Pro, which is about ChatGPT 3.5 level
  • Perplexity.ai - the best free service w/ web search capabilities
  • You.com - another search and chat provider
  • Phind - Proprietary coding model (fine-tuned off of Code Llama 34B)

Open Model Chat

Open Model Testing

This site tracks quality, throughput, price of various API hosting providers:

https://openrouter.ai/ https://www.anyscale.com/endpoints https://replicate.com/ https://www.together.ai/ https://www.fireworks.ai/ https://octo.ai/pricing/

GPU Rental Pricing

H100’s

You can also run LLMs locally on most modern computers (although larger models require strong GPUs).

The easiest (virtually one-click, no command line futzing) way to test out some models is with LM Studio (Linux, Mac, Windows). Other alternatives include:

If you are more technical:

Other chat clients: https://news.ycombinator.com/item?id=39532367

Most of the guides in this HOWTO section will assume:

Global Recommendations:

Install Mambaforge and create a new conda environment anytime you are installing a package which have many dependencies. eg, create a separate exllama and autogptq or lm-eval environment.

Other Resources

https://github.com/LykosAI/StabilityMatrix https://github.com/comfyanonymous/ComfyUI