The OpenChat team has developed the World's First Gemma fine-tune model based on openchat-3.5-0106 data and method (C-RLFT). Despite similarities in performance to the Mistral-based version, Gemma's base model doesn't outperform Mistral, possibly due to its instruct data affecting entropy. Additionally, Gemma 7B instruction fine-tuned on Tamil alpaca dataset is receiving positive attention. In a new LM benchmark, GPT-4 and Gemini Pro show similar performance with around 30% accuracy.
Very cool new LM benchmark! Surprisingly, GPT-4 and Gemini Pro are pretty much matched in performance on this. They get roughly 30% accuracy. https://t.co/wY5BM3SX4O
Wow! Looking nice Gemma 7B instruction fine tuned on Tamil alpaca dataset! π https://t.co/ZHBFvvdhP4
Nice!! The first Gemma fine-tune that is actually useful by the OpenChat team, Interesting to see that it doesn't outperform mistral even though the base model is outperforming mistral's base. I suspect Gemma's instruct data in it's base model takes away from it's entropy. https://t.co/Q8kFhgDAII
π The World's First Gemma fine-tune based on openchat-3.5-0106 data and method (C-RLFT). Almost the same performance as the Mistral-based version. 6T tokens = secret recipe? HuggingFace: https://t.co/X5WIZyxAlr
6T tokens = secret recipe? π The World's First Gemma fine-tune based on openchat-3.5-0106 data and method (C-RLFT). Almost the same performance as the Mistral-based version. HuggingFace: https://t.co/X5WIZyxAlr
6T tokens = secret recipe? π The World's First Gemma fine-tune based on openchat-3.5-0106 data and method (C-RLFT). Almost the same performance as the Mistral-based version. HuggingFace: https://t.co/X5WIZyxAlr GitHub: https://t.co/zLSCQU4lNQ