FP16
--- Scores for Model: shisa-ai/shisa-v2-llama3.1-405b ---
Category gpt-4-0613 gpt-4-turbo gpt-4.1-2025-04-14 gpt-4.1-mini-2025-04-14 gpt-4o
----------- ------------ ------------- -------------------- ------------------------- --------
coding 9.35 9.75 9.1 9.05 8.9
extraction 9.85 10 9.65 9 9.45
humanities 9.82 9.8 9.35 9.3 9.05
math 8 8.35 8.9 8.85 8.55
reasoning 8.8 8.15 8.15 8.4 8.25
roleplay 9.65 9.75 9.55 9.1 9.1
stem 9.8 9.85 9.1 9 8.9
writing 9.65 9.8 9.25 9 9.15
Overall 9.37 9.43 9.13 8.96 8.92
Overall xCM 9.6 9.56 9.18 8.97 8.98
IQ2_XXS
--- Scores for Model: shisa-v2-llama3.1-405b-IQ2_XXS ---
Category gpt-4-turbo gpt-4.1-2025-04-14 gpt-4.1-mini-2025-04-14 gpt-4o
----------- ------------- -------------------- ------------------------- --------
coding 7.85 7.3 7.35 7.5
extraction 9.95 9.05 8.4 9.05
humanities 9.6 8.8 8.85 9
math 7.05 7.55 7.5 6.95
reasoning 6.25 5.15 5.8 5.3
roleplay 8.4 6.8 7.3 7.5
stem 9.4 7.65 8.3 8.6
writing 9.3 7.5 7.7 8.4
Overall 8.47 7.48 7.65 7.79
Overall xCM 8.82 7.49 7.73 7.98