Groq Inc. has launched the Llama 3 AI model, achieving unprecedented processing speeds. The Llama 3 model, available in various configurations including 70B and 8B, is noted for its high throughput, reaching up to 290 tokens per second (T/s) for the 70B model and 876 T/s for the 8B model. This performance significantly surpasses that of other models in its class, such as the Sonnet, which operates at 63 T/s. The Llama 3's capabilities are enhanced by Groq's custom LPU inference engine, which allows it to operate at speeds comparable to, or faster than, other leading models like GPT-4. Notably, the Llama 3 70B achieves 222 T/s and the 8B version reaches 850 T/s.
** Llama 3 70b can be run at 280+ tokens/sec via @GroqInc. Everything else in the frontier is literally 5-10x slower. https://t.co/hCwBnNkNCe
The inference speed of Llama 3 on @GroqInc is a complete game changer for realtime use cases.
Llama3 70b is clearly at GPT-4 level and can run at 873 Tokens per seconds with @GroqInc, what a time to be alive! https://t.co/TQCeRluA7y
Llama 3 70B at 222 tokens per second at @GroqInc , crazy speed and response times. and a wild 850 t/s for Llama 3-8b. https://t.co/9H2AxZN1A1
llama3 on @groq is absurdly fast. https://t.co/j8JaNxpYVp
whisper + llama 3 on @GroqInc🤭 https://t.co/4iYIW3Amyi
OMG LLaMA 3 70b on @GroqInc https://t.co/cUJ0btYRWs
Llama 3 on @GroqInc 🤝 @e2b_dev code interpreter It's fast! 🏎️ https://t.co/XBkx7AX4g1
Llama 3 running at 825t/sec on Groq. 👀 https://t.co/Niq7vdL6KP
First throughput tokens/s benchmarks of Llama 3 🐎 @GroqInc is coming fast out of the gate with 284 tokens/s on Llama 3 70B and 876 tokens/s on Llama 3 8B. Once-again showing the step-change speed benefits of their custom LPU chips. Congratulations to other providers too,… https://t.co/tot6TfzeiQ
Perhaps the fastest inference of Llama 3 70b on Groq! Video not sped up! https://t.co/pNALAR4Iw4 https://t.co/unzO6PDRNo
Llama3 is live on @GroqInc! https://t.co/nG1ApI7sHX
Llama 3 400b is gonna be running on groq faster than GPT-4-turbo and at the same level of quality 😳 https://t.co/gPRSEGMWLl
Llama 3 8b running at upwards of 800 T/s on groq 😲 https://t.co/W3I8Or3Uqb
Llama on Speed! 🦙💨 @GroqInc just started running @AIatMeta Llama3-70b; 🚀 wicked fast via the LPU inference engine. Great job 👏 @sundeep. A must try, visit https://t.co/mNqGIVdHnB https://t.co/PhMzjb6dsL
Llama 3 is on @GroqInc now 😄💥 https://t.co/Z8FkmKUKdd
Llama 3 in @GroqInc soon? Look the token speed/s 😳 https://t.co/w9wegzNmOl
Friendly reminder that 70B LLama 3 could run at 250-290 T/s on @GroqInc That is an insane speed for a Sonnet class model which itself is only at 63 T/s according to @ArtificialAnlys https://t.co/gXQZyGthB3
Yes, Llama3 on @GroqInc coming.