A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicity
Paper
•
2305.13169
•
Published
•
3
data
-
-
A Survey on Data Selection for Language Models
Paper • 2402.16827 • Published • 4 -
HuggingFaceFW/fineweb-edu
Viewer • Updated • 3.5B • 254k • 954 -
allenai/MADLAD-400
Updated • 259k • 155 -
uonlp/CulturaX
Viewer • Updated • 7.18B • 37k • 587 -
Best Practices and Lessons Learned on Synthetic Data for Language Models
Paper • 2404.07503 • Published • 31 -
Scaling Synthetic Data Creation with 1,000,000,000 Personas
Paper • 2406.20094 • Published • 104 -
DDK: Distilling Domain Knowledge for Efficient Large Language Models
Paper • 2407.16154 • Published • 22 -
Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Models
Paper • 2408.02085 • Published • 19 -
Better Alignment with Instruction Back-and-Forth Translation
Paper • 2408.04614 • Published • 15 -
The ShareLM Collection and Plugin: Contributing Human-Model Chats for the Benefit of the Community
Paper • 2408.08291 • Published • 11