Pretraining Context Compressor for Large Language Models with Embedding-Based Memory
YuHong Dai
BroAlanTaps
AI & ML interests
CV(2D-Object Detection), NLP(Infer Accelerate)
Recent Activity
upvoted
a
paper
8 days ago
Do Reasoning Models Enhance Embedding Models?
updated
a collection
18 days ago
PCC-Pretrained
updated
a dataset
18 days ago
BroAlanTaps/Warmup-GPT2-Large-Llama3-8B-Redpajama
Organizations
models
129
BroAlanTaps/wikipedia-visual-en-qwen2.5-2b-instruct-lora-sft-wikipedia-samples
Image-to-Text
•
2B
•
Updated
•
2
BroAlanTaps/Stage2-PCC-Large-16x
Updated
BroAlanTaps/Stage2-PCC-Large-128x
Updated
BroAlanTaps/Stage2-PCC-Large-4x
Updated
BroAlanTaps/PCC-Large-Encoder-Llama3-8B-Instruct
8B
•
Updated
•
1
BroAlanTaps/Stage1-PCC-Large-128x
Updated
BroAlanTaps/Stage2-PCC-Lite-256x
0.8B
•
Updated
BroAlanTaps/Stage2-PCC-Lite-128x
0.8B
•
Updated
•
2
BroAlanTaps/Stage2-PCC-Lite-64x
0.8B
•
Updated
BroAlanTaps/Stage2-PCC-Lite-16x
0.8B
•
Updated
datasets
7
BroAlanTaps/Warmup-GPT2-Large-Llama3-8B-Redpajama
Viewer
•
Updated
•
1.55M
•
21
BroAlanTaps/pretrain-corpus-test
Viewer
•
Updated
•
576
•
1
BroAlanTaps/GPT2-Large-Llama3-8B-fineweb-256-5Btokens
Viewer
•
Updated
•
19.2M
•
29
BroAlanTaps/efficiency_samples_8k
Viewer
•
Updated
•
1k
•
6
BroAlanTaps/Stage2-PCC-Large-SFT-Squad
Viewer
•
Updated
•
92.7k
•
14
BroAlanTaps/Stage2-PCC-SFT-HotpotQA
Viewer
•
Updated
•
90.4k
•
125
BroAlanTaps/Stage2-PCC-Lite-SFT-Squad
Viewer
•
Updated
•
92.7k
•
450