FP8-block, FP8-dynamic, NVFP4, w4a16, w8a8 quantized models of ibm-granite/granite-4.0-h-small and ibm-granite/granite-4.0-h-tiny models
Inference Optimization
community
AI & ML interests
None defined yet.
Recent Activity
View all activity
FP8-dynamic, FP8-block, NVFP4, INT4, versions of nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B
-
inference-optimization/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8
Text Generation • 32B • Updated • 18 -
inference-optimization/NVIDIA-Nemotron-3-Nano-30B-A3B-NVFP4
18B • Updated • 181 -
inference-optimization/NVIDIA-Nemotron-3-Nano-30B-A3B-quantized.w4a16
6B • Updated • 72 -
inference-optimization/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8-dynamic
32B • Updated • 171
FP8-block, FP8-dynamic, NVFP4, w4a16, w8a8 quantized models of ibm-granite/granite-4.0-h-small and ibm-granite/granite-4.0-h-tiny models
FP8-dynamic, FP8-block, NVFP4, INT4, versions of nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B
-
inference-optimization/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8
Text Generation • 32B • Updated • 18 -
inference-optimization/NVIDIA-Nemotron-3-Nano-30B-A3B-NVFP4
18B • Updated • 181 -
inference-optimization/NVIDIA-Nemotron-3-Nano-30B-A3B-quantized.w4a16
6B • Updated • 72 -
inference-optimization/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8-dynamic
32B • Updated • 171
models
80
inference-optimization/Ministral-3-14B-Instruct-2512-FP8-dynamic
Text Generation
•
14B
•
Updated
•
121
inference-optimization/Ministral-3-14B-Instruct-2512-NVFP4
Updated
•
31
inference-optimization/Ministral-3-14B-Instruct-2512.w8a8
Updated
•
19
inference-optimization/Ministral-3-14B-Instruct-2512.w4a16
Updated
•
57
inference-optimization/Meta-Llama-3-8B-Instruct-NVFP4-GPTQ-Quant
5B
•
Updated
•
8
inference-optimization/Meta-Llama-3-8B-Instruct-NVFP4-GPTQ-MSE
5B
•
Updated
•
9
inference-optimization/Meta-Llama-3.1-8B-Instruct-NVFP4-FP8-Dynamic_6.5-bits
7B
•
Updated
•
18
inference-optimization/Meta-Llama-3.1-8B-Instruct-NVFP4-FP8-Dynamic_6.25-bits
6B
•
Updated
•
16
inference-optimization/Meta-Llama-3.1-8B-Instruct-NVFP4-FP8-Dynamic_6.0-bits
6B
•
Updated
•
18
inference-optimization/Meta-Llama-3.1-8B-Instruct-NVFP4-FP8-Dynamic_5.75-bits
6B
•
Updated
•
16
datasets
0
None public yet