Allen Institute for AI (AI2) has released OLMo, a fully open 7B large language model (LLM), including model weights, pretraining data, training code, and comprehensive documentation. This release, praised for its openness, includes the OLMo model and Dolma dataset, detailed in two papers. The development was supported by Jonathan and the @MosaicML team, described as a 'game-changer,' and utilized their command-line interface for efficient scaling. The release also features the nomic-embed-text-v1 model, and AI2, a @weights_biases customer, has made their training logs public. The initiative, part of a cross-institutional collaboration, leverages the 'foundry model' for AI development, with OLMo 7B now available to run as a Studio.
Huge congrats OLMo team! Super special -- takes a lot of effort and care to put so many research artefacts together! OLMo is entirely open source and a cross-institutional collaboration 🎉🔥 https://t.co/O6ekbcZbGP
By releasing OLMo alongside the pre-training data and training code, we’re giving the industry an opportunity to truly understand what is going on inside AI models. https://t.co/MmdFOIIQhe
Congratulations to the @allen_ai team and Lightning alumn @AnanyaHarsh for an awesome model release! Love the focus on pure open source ⚡⚡ OLMo 7B is available to run now as a Studio! https://t.co/Q5BNRfTTKe
Congratulations to the @allen_ai team and Lightning alumn @AnanyaHarsh for an awesome model release! OLMo 7B is available to run now as a Studio! https://t.co/CkqGvkJlZP
Congrats and thanks to the OLMo team @allen_ai ! I'm proud to say they were able to do this fast and efficiently by building on top of @databricks / @MosaicML's stack. Another proof point in the foundry model for AI...focus on things that matter! https://t.co/7kZOfBYSLb
very cool to release training data scripts and training code!!! @allen_ai is pushing the limits of openness, great work. https://t.co/vY9thKyVGd
Allen AI has been a long-time @weights_biases customer, its great to see them releasing their training logs along with training code, datasets and weights - truly open source! https://t.co/AXDnV9JNrO https://t.co/Bse8wROGVv
Congratulations to our friends at @allen_ai on joining (along with EleutherAI and @llm360) the tiny club of organizations that have trained a large language model with: 1. Public training data 2. Partially trained checkpoints 3. Open source licensing on model weights https://t.co/v2pOSFOpob
It seems February started with a fully open source AI renaissance 🌟 LLM - @allen_ai's OLMo-7B 🧠 Embedding - @nomic_ai nomic-embed-text-v1 📚 models with open dataset, training code, weights ✅ https://t.co/SMHAX8sxmj
Impressive release of an open everything LLM by @allen_ai: weights, code, data and documentation with two highly detailed papers: *Olmo model: https://t.co/wWfNNMxOkv *Dolma dataset: https://t.co/flXd2dujcI This is the foundations we need to build LLM as a commons. https://t.co/2m3cjFIITp
THIS is the way to open-source models! Not just weights, but pre-training data and code as well. https://t.co/J84C7YrWPO
Thank you, Jonathan! We could not have done this without you and the @databricks @MosaicML team — "Mosaic was a game-changer for developing OLMo. Their platform allowed us to effortlessly scale up training and ablations when needed, while their command-line interface lets us… https://t.co/MOnKXkWeTE
It’s finally here! Amazing work by @soldni and AI2 friends! OLMa a truly open 7B LLM https://t.co/CFmJj0gg8x
So here we go, the first fully open model released, which includes the pretraining data as well. https://t.co/uLwlEDiVAV https://t.co/gxWEuXay43