Loading...
A new paper titled 'FlashFFTConv: Efficient Convolutions for Long Sequences with Tensor Cores' has been released, showcasing significant speed improvements in exact FFT convolutions and end-to-end speedup. The paper highlights the ability of FlashFFTConv to achieve up to 7.93x speedup over PyTorch and reduce memory footprint. It emphasizes the importance of convolutions for long sequence modeling and their efficiency compared to Transformers.
[LG] FlashFFTConv: Efficient Convolutions for Long Sequences with Tensor Cores D Y. Fu, H Kumbong, E Nguyen, C Ré [Stanford University] (2023) https://t.co/SGQnUDKQ1W - Convolutions are important for long sequence modeling but lag behind Transformers in efficiency on modern… https://t.co/IS2nap5Drk https://t.co/1irzksfkkL
Excited to share our latest collaboration on optimizing performance for sub-quadratic model architectures: FlashFFTConv. FlashFFTConv speeds up exact FFT convolutions by up to 7.93x over PyTorch, reduces memory footprint, & gets 4.4x speedup end-to-end. https://t.co/KPkNlnv7ya https://t.co/cGcUVuirXr
Announcing FlashFFTConv: Efficient Convolutions for Long Sequences with Tensor Cores! We speed up exact FFT convolutions by up to 7.93x over PyTorch, reduce memory footprint, and get 4.4x speedup end-to-end. Read on for more details: Thanks @arankomatsuzaki and @_akhaliq for… https://t.co/Z8EBeZxYjR https://t.co/OY5uJ8iCXq
Very nice proposal in this paper - "FlashFFTConv: Efficient Convolutions for Long Sequences with Tensor Cores" FlashFFTConv speeds up exact FFT convolutions by up to 7.93× over PyTorch and achieves up to 4.4× speedup end-to-end. Given the same compute budget, FlashFFTConv allows… https://t.co/GaILUvImis https://t.co/hJ4OJXqShc
FlashFFTConv: Efficient Convolutions for Long Sequences with Tensor Cores paper page: https://t.co/cOUXER7Q6K Convolution models with long filters have demonstrated state-of-the-art reasoning abilities in many long-sequence tasks but lag behind the most optimized Transformers… https://t.co/J5hGdI2dLw https://t.co/to8HaMeC7l
FlashFFTConv: Efficient Convolutions for Long Sequences with Tensor Cores Speeds up exact FFT convolutions by up to 7.93x over PyTorch and achieves up to 4.4x speedup end-to-end https://t.co/WaL4rFfK4E https://t.co/1DZcfWIsSR