Trainers
- https://github.com/InternLM/xtuner
- Unsloth
- Llama Factory?
7B Fine Tune: https://huggingface.co/deepseek-ai/deepseek-llm-7b-base
67B Fine Tune: https://huggingface.co/deepseek-ai/deepseek-llm-67b-base https://github.com/deepseek-ai/DeepSeek-LLM
vs Mixtral
DeepSeek 67B vs Mixtral 8x7B - chatntq qlora
KTO https://contextual.ai/better-cheaper-faster-llm-alignment-with-kto/ https://github.com/ContextualAI/HALOs/blob/main/assets/report.pdf
MergeKit https://github.com/cg123/mergekit/blob/mixtral/moe.md
Test https://huggingface.co/openchat/openchat-3.5-1210 https://github.com/imoneoi/openchat https://openchat.team/
Magicoder on DeepSeek-Coder 33B, CodeLlama 70B
Base Models
- https://huggingface.co/deepseek-ai/deepseek-coder-33b-instruct
- https://huggingface.co/codellama/CodeLlama-70b-hf Info:
- https://github.com/ise-uiuc/magicoder/
- https://huggingface.co/ise-uiuc/Magicoder-S-DS-6.7B
https://github.com/bigcode-project/starcoder2 https://huggingface.co/blog/starcoder2 https://huggingface.co/m-a-p/OpenCodeInterpreter-DS-33B https://huggingface.co/m-a-p/OpenCodeInterpreter-SC2-7B
Search w/ Lepton https://github.com/leptonai/search_with_lepton
Voice Clone Grimlock LLM to Toy
Brave Search