Apple has recently introduced a new series of open-source language models known as OpenELM. These models are designed to be more efficient, requiring 2x fewer pre-training tokens and fewer parameters compared to existing models like OLMo, with which they perform on par. OpenELM also features an open-source training and inference framework, and is capable of layer-wise scaling to efficiently allocate parameters in its transformer model. Additionally, the series includes OpenELM Instruct, which has been described as solid and well-aligned. Apple's advancements in AI are highlighted by their ability to support MLX out of the box and the release of these models as part of their ongoing AI development.
[CL] OpenELM: An Efficient Language Model Family with Open-source Training and Inference Framework S Mehta, M H Sekhavat, Q Cao, M Horton… [Apple] (2024) https://t.co/BtWUAA74wR - OpenELM uses layer-wise scaling to efficiently allocate parameters in the transformer model,… https://t.co/2ab3idHwAc
Apple releases new family of Open-source Efficient Language Models as AI work progresses https://t.co/BgKrzrunNe by @ChanceHMiller
Apple just released 4 open source AI models: Meet OpenELM Instruct - an apple Collection. I am testing now, thus far solid model but very “aligned”. More soon. Article: https://t.co/anfRLAWfKw
Cool new work from some colleagues at Apple: more accurate LLMs with fewer parameters and fewer pre-training tokens. Also has MLX support out of the box! Code here: https://t.co/823UXDlHyP https://t.co/kcIjvdzTgi
Apple presents OpenELM - An efficient LM family with open-source training and inference framework - Performs on par with OLMo while requiring 2x fewer pre-training tokens repo: https://t.co/1RJg9gGKtt hf: https://t.co/ThdDPUuD21 abs: https://t.co/xK9O66hAsL https://t.co/T4aXHIfhrC
QLoRA fine-tuning 4-bit Gemma 2B on iPhone 15 Pro with MLX Swift. A nice size for fine-tuning on device, getting 70-100 toks/sec depending on the batch. Guide here: https://t.co/daK5rxdFxf https://t.co/bakPpPRL4b
[CL] A Survey on Self-Evolution of Large Language Models https://t.co/Lwhh6TqAvV - Self-evolving LLMs autonomously acquire, refine, and learn from their own generated experiences without intensive human supervision. This mimics human experiential learning and enables… https://t.co/up3MDJYlhf