KORMo-Team/dclm-baseline-filtered
Preview
•
Updated
•
6.48k
•
1
KORMo pretraining datasets
The pretraining datasets for KORMo-10B were collected from diverse, publicly available source.
-
KORMo-Team/korean-web-collection
Preview • Updated • 1.77k • 1Note Stage 1 Pretraining Datasets English - KORMo-Team/dclm-baseline-filtered (~1000B Tokens) Korean - KORMo-Team/korean-web-collection (~42.5B Tokens)
-
KORMo-Team/UltraFineWeb-filtered
Preview • Updated • 2.03k • 1 -
HuggingFaceTB/stack-edu
Viewer • Updated • 167M • 3.91k • 55 -
HuggingFaceTB/smollm-corpus
Viewer • Updated • 237M • 17.1k • 391 -
HuggingFaceTB/finemath
Viewer • Updated • 48.3M • 5.89k • 338 -
HuggingFaceTB/cosmopedia
Viewer • Updated • 31.1M • 55.4k • 644 -
nvidia/OpenCodeReasoning
Viewer • Updated • 753k • 3.75k • 507
nvidia/OpenMathReasoning
Viewer • Updated • 5.68M • 8.66k • 347Note Stage 2 Pretraining Datasets English - KORMo-Team/UltraFineWeb-english-filtered (~793B Tokens) - HuggingFaceTB/finemath (~37.3B Tokens, sampling) - HuggingFaceTB/stack-edu & HuggingFaceTB/smollm-corpus(python-edu) (~152B Tokens, sampling) - HuggingFaceTB/cosmopedia (~25B Tokens) - nvidia/OpenCodeReasoning (~0.65B Tokens, sampling) - nvidia/OpenMathReasoning (~3.19B Tokens, sampling)
-
KORMo-Team/korean-public-corpus
Preview • Updated • 568 -
KORMo-Team/FineWeb2-ko-synth
Preview • Updated • 1.31k -
KORMo-Team/Cosmopedia-ko-synth
Preview • Updated • 1.12k -
KORMo-Team/UltraFineWeb-ko-synth
Preview • Updated • 2.32k
KORMo-Team/NemoPost-ko-synth
Preview • Updated • 368 • 1Note Stage 2 Pretraining Datasets Korean - KORMo-Team/korean-public-corpus (~5.57B Tokens) - KORMo-Team/FineWeb2-ko-synth (~10.97B Tokens) - KORMo-Team/Cosmopedia-ko-synth (~4.07B Tokens) - KORMo-Team/UltraFineWeb-ko-synth (~41.69B Tokens) - KORMo-Team/NemoPost-ko-synth (~7.05B Tokens)