Tubidy
    Sparse is Enough in Scaling Transformers (aka Terraformer) | ML Research Paper Explained

    Sparse is Enough in Scaling Transformers (aka Terraformer) | ML Research Paper Explained

    57:06 |
    Download MP3

    Related Videos

    Were RNNs All We Needed? (Paper Explained) 27:48

    Were RNNs All We Needed? (Paper Explained)

    Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters (Paper) 53:02

    Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters (Paper)

    Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools (Paper Explained) 1:11:58

    Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools (Paper Explained)

    xLSTM: Extended Long Short-Term Memory 57:00

    xLSTM: Extended Long Short-Term Memory

    Privacy Backdoors: Stealing Data with Corrupted Pretrained Models (Paper Explained) 1:03:56

    Privacy Backdoors: Stealing Data with Corrupted Pretrained Models (Paper Explained)

    Scalable MatMul-free Language Modeling (Paper Explained) 49:45

    Scalable MatMul-free Language Modeling (Paper Explained)

    [ML News] OpenAI is in hot waters (GPT-4o, Ilya Leaving, Scarlett Johansson legal action) 29:22

    [ML News] OpenAI is in hot waters (GPT-4o, Ilya Leaving, Scarlett Johansson legal action)

    Flow Matching for Generative Modeling (Paper Explained) 56:16

    Flow Matching for Generative Modeling (Paper Explained)

    JEPA - A Path Towards Autonomous Machine Intelligence (Paper Explained) 59:38

    JEPA - A Path Towards Autonomous Machine Intelligence (Paper Explained)

    ORPO: Monolithic Preference Optimization without Reference Model (Paper Explained) 33:26

    ORPO: Monolithic Preference Optimization without Reference Model (Paper Explained)

    Big Bird: Transformers for Longer Sequences (Paper Explained) 34:30

    Big Bird: Transformers for Longer Sequences (Paper Explained)

    Grokking: Generalization beyond Overfitting on small algorithmic datasets (Paper Explained) 29:47

    Grokking: Generalization beyond Overfitting on small algorithmic datasets (Paper Explained)

    Mamba: Linear-Time Sequence Modeling with Selective State Spaces (Paper Explained) 40:40

    Mamba: Linear-Time Sequence Modeling with Selective State Spaces (Paper Explained)

    Attention Is All You Need 27:07

    Attention Is All You Need

    DINO: Emerging Properties in Self-Supervised Vision Transformers (Facebook AI Research Explained) 39:13

    DINO: Emerging Properties in Self-Supervised Vision Transformers (Facebook AI Research Explained)

    V-JEPA: Revisiting Feature Prediction for Learning Visual Representations from Video (Explained) 50:03

    V-JEPA: Revisiting Feature Prediction for Learning Visual Representations from Video (Explained)

    TransformerFAM: Feedback attention is working memory 37:01

    TransformerFAM: Feedback attention is working memory

    Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention 37:17

    Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention

    An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (Paper Explained) 29:56

    An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (Paper Explained)

    No, Anthropic's Claude 3 is NOT sentient 15:12

    No, Anthropic's Claude 3 is NOT sentient