Data · dataset · 2022
naab-raw (raw version of the naab corpus)
Listed in Hugging Face Datasets
Huge corpora of textual data are always known to be a crucial need for training deep models such as transformer-based ones.
Description
This issue is emerging more in lower resource languages - like Farsi. We propose naab, the biggest cleaned and ready-to-use open-source textual corpus in Farsi.
It contains about 130GB of data, 250 million paragraphs, and 15 billion words. The project name is derived from the Farsi word ناب which means pure and high-grade. This corpus contains the raw (uncleaned) version of it.
Links
Where it is published
- Hugging Face dataset page huggingface.co/datasets/SLPL/naab-raw ↗
landing page · from Hugging Face
Documentation and papers
- arXiv:2208.13486 arxiv.org/abs/2208.13486 ↗
publication · from Hugging Face
Catalogue records · 1
- Hub API huggingface.co/api/datasets/SLPL/naab-raw ↗
metadata API · from Hugging Face
Topics
- Stated by source
- fill mask · text generation
- From keywords
- Computer Science & AI
- Inferred from text
- Artificial intelligence 72%
Provenance · 1 source records, 11 field assertions
| Source | Key | Last seen | Raw |
|---|---|---|---|
| Hugging Face Datasets | SLPL/naab-raw | 15 d ago | JSON v1 |
| Field | Assertion | Extractor | Evidence |
|---|---|---|---|
| access_level | source · Hugging Face | connector:huggingface@1.0.0 | /gated |
| concepts[field].anzsrc:group:4602 | enrichment · Hugging Face | taxonomy-embedding@1.1.0 | title+keywords+description (72%) |
| concepts[field].local:field:computer-science-ai | mapping · Hugging Face | connector:huggingface@1.0.0 | |
| concepts[task].hf_task:fill-mask | source · Hugging Face | connector:huggingface@1.0.0 | /tags[task_categories:*] |
| concepts[task].hf_task:text-generation | source · Hugging Face | connector:huggingface@1.0.0 | /tags[task_categories:*] |
| created_date | source · Hugging Face | connector:huggingface@1.0.0 | |
| description | source · Hugging Face | connector:huggingface@1.0.0 | /description |
| license | source · Hugging Face | connector:huggingface@1.0.0 | /tags[license:*] |
| publication_date | source · Hugging Face | connector:huggingface@1.0.0 | |
| title | source · Hugging Face | connector:huggingface@1.0.0 | /id |
| updated_date | source · Hugging Face | connector:huggingface@1.0.0 |