Data · dataset · 2022
acul3/KoPI-CC
Listed in Hugging Face Datasets
Dataset
Description
Summary
KoPI-CC (Korpus Perayapan Indonesia)-CC is Indonesian only extract from Common Crawl snapshots using ungoliant, each snapshot also filtered using some some deduplicate technique such as exact hash(md5) dedup technique and minhash LSH neardup Preprocessing Each folder name inside snapshots folder denoted preprocessing technique that has been applied . Raw this processed directly from cc snapshot using ungoliant without any addition filter ,you can read it… See the full description on the dataset page: huggingface.co/datasets/acul3/KoPI-CC.
Links
Where it is published
- Hugging Face dataset page huggingface.co/datasets/acul3/KoPI-CC ↗
landing page · from Hugging Face
Documentation and papers
- arXiv:2201.06642 arxiv.org/abs/2201.06642 ↗
publication · from Hugging Face
Catalogue records · 1
- Hub API huggingface.co/api/datasets/acul3/KoPI-CC ↗
metadata API · from Hugging Face
Topics
- Stated by source
- text generation
- From keywords
- Computer Science & AI
Provenance · 1 source records, 9 field assertions
| Source | Key | Last seen | Raw |
|---|---|---|---|
| Hugging Face Datasets | acul3/KoPI-CC | 15 d ago | JSON v1 |
| Field | Assertion | Extractor | Evidence |
|---|---|---|---|
| access_level | source · Hugging Face | connector:huggingface@1.0.0 | /gated |
| concepts[field].local:field:computer-science-ai | mapping · Hugging Face | connector:huggingface@1.0.0 | |
| concepts[task].hf_task:text-generation | source · Hugging Face | connector:huggingface@1.0.0 | /tags[task_categories:*] |
| created_date | source · Hugging Face | connector:huggingface@1.0.0 | |
| description | source · Hugging Face | connector:huggingface@1.0.0 | /description |
| license_text | source · Hugging Face | connector:huggingface@1.0.0 | |
| publication_date | source · Hugging Face | connector:huggingface@1.0.0 | |
| title | source · Hugging Face | connector:huggingface@1.0.0 | /id |
| updated_date | source · Hugging Face | connector:huggingface@1.0.0 |