Miqu, a leaked model from Mistral, is a Llama-2 70B fine-tune that has generated significant attention. It has scored high on an emotional IQ benchmark, surpassing every other LLM in the world except GPT-4. However, it does not perform well on standard benchmark scores like GSM8K and Winograde. Despite the hype, there are concerns about its coding abilities. Miqu has the same MMLU score as Mistral Medium, indicating a close relationship between the two models.
Miqu Has The EXACT Same MMLU Score As Mistral Medium!! 🤯🤯 Miqu is a leaked model from Mistral. They have said that this model is basically a Llama-2 70B fine-tune Here is the crazy part. - Mistral Medium has a published MMLU score of 75.3%. - Our internal eval just… https://t.co/7oqDyuPXyL
Miqu is a Llama-2-70b fine-tune and DOES NOT have good standard benchmark scores. Lots of chatter about Miqu 1, which is a leaked llama-70 fine-tune. While it scores high on an emotional IQ benchmark, it doesn't score very well on the other evals. GSM8K - 70.43 Winograde -… https://t.co/sJN87RXa1o
Miqu is a llama-70b fine-tune and DOES NOT have good standard benchmark scores. Lots of chatter about Miqu 1, which is a leaked llama-70 fine-tune. While it scores high on an emotional IQ benchmark, it doesn't score very well on the other evals. GSM8K - 70.43 Winograde - 79.87… https://t.co/ki8Twr8Fbo
the mistral team is too good at drumming up hype. the miqu leaker might as well be someone on their marketing team https://t.co/BRlfsq0MUS
Alright so miqu-leak just a big llama-2 finetune but on same data as mistral medium. It was still bad at coding( i.e would not respond with c++ part like mixtral and medium would) even after others retested my prompt with the right [INST] prefix. Again, it’s NOT mistral-medium. https://t.co/5e8OKayUGS
Whatever Miqu is, it has some sort of special sauce. It gets an 83.5 on EQ-Bench (evaluated locally), surpassing *every other LLM in the world except GPT-4*. EQ-Bench has a 0.97 correlation w/ MMLU, and a 0.94 correlation w/ Arena Elo. It *beats* Mistral Medium - at Q4_K_M. I… https://t.co/0gOOPjxjPD