NVIDIA has introduced NV-Embed, a generalist embedding model that has achieved the top position on the Massive Text Embedding Benchmark (MTEB) leaderboard. NV-Embed, which utilizes decoder-only large language models (LLMs), has outperformed BERT and T5-based models in general-purpose text embedding tasks. The model incorporates several architectural improvements, including latent attention pooling and a two-stage contrastive instruction-tuning. NV-Embed scored 59.36 on 15 retrieval tasks within the MTEB benchmark, using only publicly available data. This development is expected to enhance the performance of decoder-only LLMs like Mistral-7B, maintaining simplicity and reproducibility. The model was developed by a team including C Lee, R Roy, M Xu, and J Raiman.
Nvidia just released the weights of NV-Embed-v1, the current leader on MTEB. Some additional info below: Base model: Mistral-7B-v0.1 Pooling type: Latent-Attention Embedding dimension: 4096 Max input tokens: 32k https://t.co/mTH5wwxtjO
NV-Embed: NVIDIA’s Groundbreaking Embedding Model Dominates MTEB Benchmarks #DL #AI #ML #DeepLearning #ArtificialIntelligence #MachineLearning #ComputerVision #AutonomousVehicles #NeuroMorphic #Robotics https://t.co/E5SYh5KgJD
🙋🏻♂️@NVIDIAAI just released a cool EMBEDDING model , that's "instruction" based ! that means you can use it with @llama_index instructor embeddings function, or try it out on @huggingface directly with my little @Gradio demo : https://t.co/2FU3MwLwU2 this is the future !
Really interesting line of work on using 🪆in # tokens for encoder-decoder multi-modal models!! As good with results with much fewer image tokens in LLaVa! Nice work @MuCai7 and co! https://t.co/px5aOnU18y
[CL] NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models C Lee, R Roy, M Xu, J Raiman... [NVIDIA] (2024) https://t.co/a14j1olhK0 - Proposes NV-Embed, a generalist embedding model based on decoder-only LLMs (e.g. Mistral 7B) that significantly… https://t.co/fvhNqWwg7G
"NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models" This paper will enhance the performance of decoder-only LLMs like Mistral-7B as versatile embedding models, while maintaining simplicity and reproducibility. 📌 Decoder-only architecture: NV-Embed… https://t.co/xVG6BUGMJA
NV-Embed: enhanced LLMs as embedding models paper by @nvidia for improved model on text embedding tasks, achieving top scores on the Massive Text Embedding Benchmark (MTEB)
NVIDIA's NV-Embed Tops MTEB Leaderboard with 59.36 on Retrieval Tasks https://t.co/kXBwoldF64
Introducing NV-Embed, a generalist embedding model that ranks No. 1 on the MTEB Benchmark, which includes 56 diverse tasks, using only publicly available data. Notably, our model also achieves the highest score of 59.36 on 15 retrieval tasks within this benchmark. NV-Embed…
Introducing NV-Embed, a generalist embedding model that ranks No. 1 on the MTEB Benchmark, using only publicly available data. Notably, our model also achieves the highest score of 59.36 on retrieval tasks within this benchmark. NV-Embed introduces several new designs, including…
NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models Nvidia introduces an embedding model with architectural improvements like latent attention pooling and a 2-stage contrastive instruction-tuning. 📝https://t.co/1BnwvIDe3D 👨🏽💻https://t.co/QNZjZB0Pt4 https://t.co/cZ85I7OQHI
NV-Embed Improved Techniques for Training LLMs as Generalist Embedding Models Decoder-only large language model (LLM)-based embedding models are beginning to outperform BERT or T5-based embedding models in general-purpose text embedding tasks, including dense https://t.co/Jgw7x2eC79
NVIDIA presents NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models Achieves #1 on the MTEB leaderboard https://t.co/zbih7Gfqyv https://t.co/1Y5BITMd5i
A new, mysterious, NVIDIA embedding model, "NV-Embed" is 1st overall, in retrieval, reranking, & classification on MTEB. cc-by-nc-4.0 licensed, LLM based (huge?), no config, no weights, doesn't even specify output dimensions or model parameter count. @nvidia don't be a tease!!