MatPag's activity

• All
• Models
• Datasets
• Spaces
• Papers
• Collections
• Community
• Posts
• Upvotes
• Likes
• Articles

authored a paper 5 months ago

Benchmarking Optimizers for Large Language Model Pretraining

Paper • 2509.01440 • Published Sep 1, 2025 • 25

authored 4 papers over 1 year ago

Unsupervised Learning of Sentence Embeddings using Compositional n-Gram Features

Paper • 1703.02507 • Published Mar 7, 2017

DoGE: Domain Reweighting with Generalization Estimation

Paper • 2310.15393 • Published Oct 23, 2023 • 1

The AdEMAMix Optimizer: Better, Faster, Older

Paper • 2409.03137 • Published Sep 5, 2024 • 6

DenseFormer: Enhancing Information Flow in Transformers via Depth Weighted Averaging

Paper • 2402.02622 • Published Feb 4, 2024 • 3

authored a paper over 2 years ago

Faster Causal Attention Over Large Sequences Through Sparse Flash Attention

Paper • 2306.01160 • Published Jun 1, 2023 • 1