Google's Gemini, a new AI model, has been compared to OpenAI's GPT models in an impartial study by Carnegie Mellon University (CMU). The study found that Gemini Pro, comparable in size to GPT-3.5 Turbo, generally achieves slightly lower accuracy than GPT-3.5 Turbo and is much worse than GPT-4. The study also mentioned that Gemini Ultra was not covered. Some researchers and analysts have expressed skepticism about Gemini's performance and its potential impact on the AI market. However, some believe that Gemini may be a real alternative to GPT-3.5 as an LLM evaluator. Google's marketing of Gemini as a competitor to GPT-3.5 and GPT-4 has sparked a debate about the significance of the results and the future of generative AI models.
With its recent release, Gemini, @Google claims to have introduced a new era for generative AI, which can outperform human experts in various intelligence tests. ♊ Join #DeepFest24 to get the latest on generative #AI development. https://t.co/HdKTNcTt7K #generativeAI https://t.co/ecXX9JVz14
Google's Gemini is here to shake up the #AI world! Get the scoop on this groundbreaking OpenAI ChatGPT challenger - https://t.co/tUUaWB0uK9 #Google #Gemini #ChatGPT
What is Gemini? Google's AI model and GPT-4 alternative explained https://t.co/0Ntw2SClis https://t.co/8PPIj1li98
Introduction to Google’s Most Powerful Multimodal Model Gemini, From a Technical Perspective https://t.co/6D5IwG5p3q
Using Gemini AI in Android Apps with the New Google AI SDK https://t.co/BQLPh6hNwI by Sergio De Simone
An In-depth Look at Gemini's Language Abilities Key point - Similar to GPT 3.5 Turbo on average, or maybe a slightly less. Much worse than GPT 4. Google has a long way to go. ----- Gemini's Pro model achieved comparable but slightly inferior accuracy compared to the current… https://t.co/LBSvvFW5Hr
*New video* Google's Gemini: Bringing 'Her' to Life With Multimodal AI? Google recently unveiled Gemini, an AI model that can seamlessly reason across text, images, video, and audio. This model is "multimodal from the ground-up" and could enable a Jarvis-style AI assistant that… https://t.co/Ky3ov5rZlP
Exciting news! A study by Carnegie Mellon University compared Gemini, GPT, and Mixtral. Gemini Pro showed slightly lower accuracy but performed well. Looking forward to testing Gemini Ultra! 🚀 #ai #geminiultra #artificialintelligence https://t.co/usUNxotglm
AI Power Struggle: Google's Gemini challenges GPT-3.5 Turbo! Who will win? Read our analysis & weigh in! #AI #GPT3 #Gemini #TechNews https://t.co/PdqhqZiIzz via @LinkedIn
Google drops Gemini to end ChatGPT's reign. Who will win this AI-war? Google's Gemini AI just changed the game! Outperforming ChatGPT, Gemini is ready to lead the AI revolution. With Ultra, Pro, and Nano models, it's not just an upgrade – it's a total overhaul.… https://t.co/oYos7DH4xn
Google’s Gemini models were marketed as the first legitimate competitor to OpenAI’s ChatGPT and GPT-4. But, in an age where LLM evals are becoming more of a marketing opportunity in big tech, it’s hard to know whether these results are substantial. Here are my honest (itemized)… https://t.co/7eRuarp6aL
Discover Google's Gemini models: AI that understands text, images, audio, and video with precision. Leading the way in benchmarks and applications, these models are reshaping what's possible in AI: https://t.co/47aLiK2XM1
Our colleague @_nerdai_ benchmarked Gemini Pro against GPT-3.5 using our own LlamaDataSets from LlamaHub and Gemini fared pretty well! Gemini may be a real alternative to GPT-3.5 as an LLM evaluator. Read our post for a full description with code showing how we evaluated their… https://t.co/qBXkD7utjz
Some benchmarks coming in for Gemini pro - the scores are well below GPT3.5 paper title "An In-depth Look at Gemini’s Language Abilities" https://t.co/dGFCUXiseZ https://t.co/3KuZGVumic
Google Gemini is not even as good as GPT-3.5 Turbo, researchers find https://t.co/kmxJ5c4VWM Visit https://t.co/l8fNQzV9nN for more AI news. #AI #artificialintelligence #models #gemini #google #openai
[CL] An In-depth Look at Gemini's Language Abilities S N Akter, Z Yu, A Muhamed, T Ou, A Bäuerle, Á A Cabrera, K Dholakia, C Xiong, G Neubig [CMU] (2023) https://t.co/rVX6AwH2YE - Gemini Pro, comparable in size to GPT-3.5 Turbo, generally achieves slightly lower accuracy than… https://t.co/c4KqgQRYJi
Google Gemini is not even as good as GPT-3.5 Turbo, researchers find https://t.co/Y9G36QQ3ni
😵 Gemini Pro Vs ChatGPT is out! Verdict - Gemino Pro is not as good as GPT3.5-turbo 🤐 Paper - An In-depth Look at Gemini's Language Abilities Datasets * Knowledge-based QA (MMLU) * Reasoning (BIG-Bench Hard) * Math (GSM8k, SVAMP, ASDIV, MAWPS) * Code Gen (HumanEval, ODEX) *…
The main table from their arXiv paper comparing Gemini Pro, GPT 3.5 Turbo, and GPT 4 Turbo. Ignore the Mixtral results since they are based on a 3rd party model (see below). https://t.co/Ht61YhsCT2 https://t.co/o0nOWTdk9K
The main table from their arXiv paper comparing Gemini Pro, GPT 3.5 Turbo, GPT 4 Turbo, and Mixtral https://t.co/WFpLECIjw5 https://t.co/o0nOWTdk9K
This is an interesting misinformation tactic — Qualify the GPT version, but speak of Gemini generally to reduce all of Gemini to 3.5. Fixed TLDR: Gemini Pro is similar to GPT3.5* (The paper doesn't cover Gemini Ultra) *Even that's not entirely accurate if we read the paper,… https://t.co/7gUF5dVKNb
An independent study by CMU compares Gemini Pro with GPT 3.5 and shows that GPT 3.5 comes ahead. I recommend waiting until Gemini-ultra is launched before considering switching to Google. They also attempted to compare it to Mistral-MoE but used the wrong model, so the numbers… https://t.co/eFo1Th2nQS
This paper confirms that Google’s new Gemini Pro is the equivalent OpenAI’s free ChatGPT 3.5. For most individual cases, you want to use the best AI & that is clearly still GPT-4, accessible with ChatGPT Plus or Bing. (At least until Gemini Ultra is released in the new year) https://t.co/eYo3dCHphb
An In-depth Look at Gemini's Language Abilities paper page: https://t.co/9cGC9nMXtF The recently released Google Gemini class of models are the first to comprehensively report results that rival the OpenAI GPT series across a wide variety of tasks. In this paper, we do an… https://t.co/KnYMIMGfGs
Google’s Gemini recently made waves as a major competitor to OpenAI’s GPT. Exciting! But we wondered: How good is Gemini really? At CMU, we performed an impartial, in-depth, and reproducible study comparing Gemini, GPT, and Mixtral. Paper: https://t.co/S3T7ediQLa 🧵 https://t.co/NmEOeDd8pI