✕

Beksultan's blog

Deep Learning, papers, career and other topics

Paper Review 7: FlashAttention - Fast and Memory-Efficient Exact Attention with IO-Awareness

IO-aware exact attention algorithm that uses tiling to reduce the number of memory reads/writes
By Beksultan Sagyndyk
Posted on February 6, 2024

VIDEO [Read More]
Tags:
- efficiency
- long input transformers
Paper Review 6: Mixtral of Experts

About the new interesting approach from MistralAi, and why there is no word 'data'
By Beksultan Sagyndyk
Posted on January 30, 2024

[Read More]
Tags:
- LLM
- long input transformers
Paper Review 5: Mistral 7B

Unleashing Mistral 7B: The Game-Changing 7-Billion-Parameter Beast?
By Beksultan Sagyndyk
Posted on January 26, 2024

[Read More]
Tags:
- LLM
- long input transformers
Paper Review 4: Self-attention Does Not Need O(n^2) Memory

How can the memory efficiency of self-attention be increased?
By Beksultan Sagyndyk
Posted on January 22, 2024

[Read More]
Tags:
Paper Review 3: Pix2Struct is an image-encoder-text-decoder based on the Vision Transformer (ViT)

How to improve ViT for specific pixel to text tasks?
By Beksultan Sagyndyk
Posted on January 22, 2024

Pix2Struct is an image-encoder-text-decoder based on the Vision Transformer (ViT) [Read More]
Tags:
- Multimodality
- image2text

Older Posts