Transformers News

Overwatch 2, Free-to-Play Hero Shooter, Brings Back Rare Pink Mercy Skin for Charity in Season 11
Authors
6
11 days
Tech
TikTok and HKU Release Depth Anything V2 for Real-Time Monocular Depth Estimation
Authors
5
14 days
AI
Tech
Hybrid State Space Models Samba 3.8B, Mamba-2-Hybrid Outperform Transformers
Authors
10
15 days
AI
Tech
'Transformers One' Animated Film Receives Standing Ovation at Annecy Premiere Screening
Authors
7
17 days
Culture

Is attention all you need? (transformers SOTA in 2027)

Nov 16, 7:02 PMJan 31, 11:30 PM

52.81%chance

12020819

OptionVotes

YES

3775

3306

Will superposition in transformers be mostly solved by 2026?

Mar 16, 7:31 AMDec 31, 11:59 PM

63.63%chance

7915992

OptionVotes

YES

3021

1249

By EOY 2025, will the model with the lowest perplexity on Common Crawl will not be based on transformers?

Jan 5, 4:19 AMDec 31, 11:59 PM

25.2%chance

313460

OptionVotes

YES

1243

877

Will a big transformer LM compose these facts without chain of thought by 2026?

Jan 19, 5:08 AMJan 1, 11:59 PM

85.72%chance

6132248

OptionVotes

YES

3180

1513

Will Transformer based architectures still be SOTA for language modelling by 2026?

Oct 27, 3:05 AMJan 2, 7:59 AM

73.77%chance

493219

OptionVotes

YES

2015

1196

In 2025, will I teach people that Projective Geometric Algebra (PGA)'s "dual" is similar to the Fourier transform?

Sep 20, 7:52 AMSep 21, 10:59 PM

48%chance

191284

OptionVotes

YES

1530

452

Are Mixture of Expert (MoE) transformer models generally more human interpretable than dense transformers?

Sep 15, 11:23 AMDec 31, 6:29 PM

52.11%chance

131161

OptionVotes

YES

994

991

If OpenAI makes a transformer sized advancement in the next 5 years, will they publish an accompanying paper?

Nov 29, 12:12 AMDec 6, 7:59 AM

43.88%chance

18788

OptionVotes

YES

1091

915

6) An alternative to the transformer architecture will see meaningful adoption.

Dec 25, 1:35 PMDec 31, 10:59 PM

60.91%chance

523225

OptionVotes

YES

1082

803

[Metaculus] Will an International AI regulatory agency for oversight of transformative AI be established before 2030?

Oct 16, 1:24 PMJan 2, 8:00 PM

67.77%chance

10360

OptionVotes

YES

1057

963

LLMs by EOY 2025: Will Retentive Learning Surpass Transformers? (Subsidised 400 M$)

Sep 15, 7:21 PMDec 31, 10:59 PM

14.29%chance

343409

OptionVotes

YES

1948

826

Will a new type of AI model replace transformers as the state-of-the-art artificial intelligence technology by 2024?

Dec 18, 10:58 AMDec 31, 10:59 PM

25.8%chance

10360

OptionVotes

YES

1115

924

Latest stories

Overwatch 2, Free-to-Play Hero Shooter, Brings Back Rare Pink Mercy Skin for Charity in Season 11
Authors
6
11 days
Tech
SAMBA Hybrid Model Introduced with Mamba and Sliding Window Attention
Authors
7
13 days
AI
Tech
TikTok and HKU Release Depth Anything V2 for Real-Time Monocular Depth Estimation
Authors
5
14 days
AI
Tech
Derek Connolly to Write 'Transformers' and 'G.I. Joe' Crossover Movie Starring Chris Hemsworth
Authors
6
15 days
Crime
Culture
Hybrid State Space Models Samba 3.8B, Mamba-2-Hybrid Outperform Transformers
Authors
10
15 days
AI
Tech
'Transformers One' Animated Film Receives Standing Ovation at Annecy Premiere Screening
Authors
7
17 days
Culture
Texas Man Named 'Optimus Prime' Arrested in Austin for Vehicle Theft
Authors
4
18 days
Crime
Culture
JinaAI's Jina-CLIP Outperforms Previous Models; TogetherCompute's Dragonfly Excels in Medical Imaging, Outperforms Med-Gemini
Authors
8
22 days
AI
Tech
Jina AI and Nomic AI Unveil Superior Multimodal Embedding Models for The Met's 250,000 Artworks
Authors
4
23 days
AI
Tech
Introduction of Mamba and Mamba-2 with 50% Faster Training and 8x Larger States Revolutionizes AI Sequence Modeling
Authors
8
25 days
AI
Tech
University of Minnesota Introduces GNN-RAG, Ant Group Proposes MetRag, Outperform GPT-4 in AI
Authors
4
26 days
China
AI
World
Chris Hemsworth in Talks to Star in Paramount's Transformers/G.I. Joe Crossover Movie, Voicing Optimus Prime, in Collaboration with Hasbro
Authors
14
28 days
Culture
Chris Hemsworth in Talks to Star in Paramount's 'Transformers' and 'G.I. Joe' Crossover Movie Produced by Michael Bay
Authors
19
29 days
Culture
University of Maryland Enhances Transformers' Arithmetic Abilities with Abacus Embeddings, Achieving 99% Accuracy on 100-Digit Addition by Encoding Positions
Authors
16
1 month
AI
Tech
HuggingFace Introduces New PaliGemma and YOLOv10 Models for Visual Question Answering and Object Detection
Authors
6
1 month
AI
Tech
Sepp Hochreiter's xLSTM: A New Rival to Transformers Under NXAI Initiative
Authors
16
2 months
AI
Tech
Robosen Launches First Auto-Converting Megatron to Mark Transformers' 40th Anniversary
Authors
10
2 months
Tech
DeProt Emerges as Transformer Alternative in Protein Language Models
Authors
4
2 months
Bio
Science
Paramount Confirms Transformers & G.I. Joe Crossover Movie for 2025/2026, Spielberg Executive Producing, Teasers in 'Transformers: Rise of the Beasts'
Authors
11
3 months
Culture
'Transformers' and 'G.I. Joe' Crossover Film Set for 2025 or 2026
Authors
19
3 months
Culture
Researchers Study Adaptation of LLaMA Decoder to Vision in Vision-Language Models (VLMs)
Authors
4
3 months
AI
Tech
Researchers' DiJiang Frequency Domain Kernelization Enhances Transformers Efficiency
Authors
6
3 months
AI
Tech
Henry Golding Optimistic About Paramount's G.I. Joe Future Amid Snake Eyes Disappointment
Authors
5
3 months
Culture
Hugging Face's Transformers Library Adds Quanto, GaLore, and LLAVa-NEXT Models with MLX Support and OS Vision Language Mode
Authors
4
3 months
AI
Tech
Google Researchers Collaborated on 'Attention Is All You Need' Paper Revolutionizing AI
Authors
4
3 months
AI
Tech
BurstAttention: DMC Improves Speed and Performance with KV Cache for LLaMa 2
Authors
6
4 months
AI
Tech
Hugging Face Releases GPT4All, Supports Quantized Models for AI Apps with Open Source Models & Transformers Library
Authors
5
4 months
AI
Tech
Google's Gemma 2B and 7B Models Achieve 22+ and 475+ tok/sec, Leading Open-Source AI
Authors
12
4 months
AI
Tech
Meta's 'Beyond A*': Transformers Achieve 94% Optimal Plans with 27% Less Search
Authors
5
4 months
AI
Tech
Gemma 2B/7B Models Hit >2500 tok/s, 100x Usability Boost on Windows
Authors
18
4 months
AI
Tech
Ollama Adds Google Gemma Support; Gemma Model Reaches Over 475 tok/sec on iPhone and Android Devices
Authors
5
4 months
AI
Tech
MGK Debuts Entire Upper Body Blackout Tattoos for Spiritual Reasons
Authors
11
4 months
Culture
Google Deepmind and Researchers: Transformers Handle Extended Input Lengths
Authors
6
4 months
AI
Tech
BriaAI's New RMBG v1.4 Surpasses Existing Models, Enables Local Image Processing
Authors
8
5 months
AI
Tech
OpenAI Introduces Truncatable - Matryoshka Embeddings, NEAR Develops Neural Embeddings for Amino Acid Relationships with RWKV and ResNet Models
Authors
4
5 months
Bio
Science
NEAR, DeepGOMeta, and ProtHyena Advance Protein Annotation and Prediction with RWKV Model
Authors
6
5 months
Bio
Science
RWKV-v5 Introduces Eagle-7B, an Attention-Free LLM Trained on 1.1 Trillion Tokens, Outperforming Mistral-7B in Multilingual Tasks, Licensed as Apache 2.0
Authors
9
5 months
AI
Tech
Eagle 7B LLM, 3x Faster than Vanilla Decoding, 2x Faster than Lookahead, 1.6x Faster than Medusa, Competes with Mistral 7B Models
Authors
4
5 months
China
AI
World
Depth Anything: Hugging Face's AI for Depth Estimation in 3 Lines of Code
Authors
5
5 months
AI
Tech
TikTok's 'Depth Anything' Outperforms MiDaS, Now on Hugging Face and Transformers.js
Authors
5
5 months
AI
Tech
Japan Becomes Fifth Nation to Land on Moon, Faces Solar Battery Issue
Authors
26
5 months
Tech
TinyLlama Project Trains 1.1B Llama Model on 3 Trillion Tokens Using 16 A100-40G GPUs, Compatible with 🤗 Transformers.js and Llama 2 Architecture
Authors
7
6 months
AI
Tech
Duke's Wayne Frederick Drafted No. 2 in MLS; Father Tied to Howard University Soccer in Late 1980s
Authors
5
6 months
AI
Tech
New Mathematical Framework Reveals Transformers as Interacting Particle Systems in d-dimensional Real Vector Space
Authors
5
6 months
AI
Tech
TogetherAI Releases StripedHyena 7B Model, Competing with Open-Source Transformers for Faster Performance
Authors
9
7 months
AI
Tech
EMNLP2023 Introduces Pushdown Transformers for Enhanced Model Functions at Language Conference
Authors
5
7 months
AI
Tech
Mamba, a Sequence Model with Fast Inference, 5× Higher Throughput, and Linear Scaling, Challenges Transformers
Authors
6
7 months
AI
Tech
New Model Mamba Outperforms in Performance, Language Modeling, and Fast Inference
Authors
4
7 months
AI
Tech
Researchers J Weston, S Sukhbaatar Propose System 2 Attention to Address Transformer Attention Issues
Authors
5
7 months
AI
Tech
Renowned Actor Shirley Jo Finney, 74, and 'American Dreams' Actor Peter Spellos, 69, Pass Away from Cancer
Authors
7
7 months
Culture
FlashFFTConv Paper Shows 7.93x Speedup Over PyTorch in Exact FFT Convolutions for Long Sequences with Tensor Cores
Authors
6
8 months
AI
Tech
Researchers Explore Methods to Make AI Language Models Forget Specific Data
Authors
4
8 months
AI
Tech
GitHub Copilot, First Generative AI Product to Reach $100M ARR, Despite Microsoft's Losses
Authors
6
9 months
Tech

Transformers News

Top stories

Prediction markets for Transformers

Prediction markets for Transformers

Prediction markets for Transformers

Is attention all you need? (transformers SOTA in 2027)

Will superposition in transformers be mostly solved by 2026?

By EOY 2025, will the model with the lowest perplexity on Common Crawl will not be based on transformers?

Will a big transformer LM compose these facts without chain of thought by 2026?

Will Transformer based architectures still be SOTA for language modelling by 2026?

In 2025, will I teach people that Projective Geometric Algebra (PGA)'s "dual" is similar to the Fourier transform?

Are Mixture of Expert (MoE) transformer models generally more human interpretable than dense transformers?

If OpenAI makes a transformer sized advancement in the next 5 years, will they publish an accompanying paper?

6) An alternative to the transformer architecture will see meaningful adoption.

[Metaculus] Will an International AI regulatory agency for oversight of transformative AI be established before 2030?

LLMs by EOY 2025: Will Retentive Learning Surpass Transformers? (Subsidised 400 M$)

Will a new type of AI model replace transformers as the state-of-the-art artificial intelligence technology by 2024?

Latest stories

Top stories

Prediction markets for Transformers

Prediction markets for Transformers

Prediction markets for Transformers

Is attention all you need? (transformers SOTA in 2027)

Will superposition in transformers be mostly solved by 2026?

By EOY 2025, will the model with the lowest perplexity on Common Crawl will not be based on transformers?

Will a big transformer LM compose these facts without chain of thought by 2026?

Will Transformer based architectures still be SOTA for language modelling by 2026?

In 2025, will I teach people that Projective Geometric Algebra (PGA)'s "dual" is similar to the Fourier transform?

Are Mixture of Expert (MoE) transformer models generally more human interpretable than dense transformers?

If OpenAI makes a transformer sized advancement in the next 5 years, will they publish an accompanying paper?

6) An alternative to the transformer architecture will see meaningful adoption.

[Metaculus] Will an International AI regulatory agency for oversight of transformative AI be established before 2030?

LLMs by EOY 2025: Will Retentive Learning Surpass Transformers? (Subsidised 400 M$)

Will a new type of AI model replace transformers as the state-of-the-art artificial intelligence technology by 2024?

Latest stories