Erhartford has announced the release of Dolphin-2.6-mistral-7b-dpo and Dolphin-2.7-mixtral-8x7b models, with plans for a 3-epoch+DPO+LASER version. Pragmaticml praises a model from MosaicML that trains for less than $100 and is an encoder. FernandoNetoAi and Erhartford announce the first successful LASER model at Huggingface, showing superior benchmarks over the latest DPO version of Dolphin finetune over Mistral AI's Mistral 7b.
Me and @erhartford are pleased to announce [maybe] the first successful LASER model @ @huggingface. Our model showed superior benchmarks over our latest DPO version of Dolphin finetune over Mistral AI's Mistral 7b. Pt 1
Refreshing to see work from @MosaicML on a model that: 1) Trains from scratch for <$100 2) Is an encoder (BERT-like) rather than a decoder Encoder-only models are still workhorses and I'm happy to see recent LLM advances being ported back to BERT. https://t.co/xZX1Vi6f0x
I published Dolphin-2.7-mixtral-8x7b. This is an "apples to apples" release at 1.5 epochs to compare evals with 2.5 and 2.6 (which were also published at 1.5 epochs) In 2-3 days, I will release a 3-epoch+DPO+LASER version that will be the best. After that I'll leave mixtral…
I published Dolphin-2.7-mixtral-8x7b. This is an "apples to apples" release at 1.5 epochs to compare evals with 2.5 and 2.6. In 2-3 days, I will release a 3-epoch+DPO+LASER version that will be the best. After that I'll leave mixtral alone until dolphin-3.0.…
announcing Dolphin-2.6-mistral-7b-dpo My first DPO training, using argilla/ultrafeedback-binarized-preferences-cleaned and unalignment/toxic-dpo-v0.1 datasets, with axolotl. This is really a very fine dolphin. Next I will LASER it. https://t.co/juhbXWiqfS
Here is the first working version of the LLaVA-3b model based on fine-tuned Phi-2 from @erhartford. The model successfully passes the vibe check. Sometime later, I will check it on formal benchmarks. https://t.co/FPbimTIuYA