Text · dataset · 2022
naab (A ready-to-use plug-and-play corpus in Farsi)
Listed in Hugging Face Datasets
Huge corpora of textual data are always known to be a crucial need for training deep models such as transformer-based ones.
Description
This issue is emerging more in lower resource languages - like Farsi. We propose naab, the biggest cleaned and ready-to-use open-source textual corpus in Farsi.
It contains about 130GB of data, 250 million paragraphs, and 15 billion words. The project name is derived from the Farsi word ناب which means pure and high-grade.
Links
Where it is published
- Hugging Face dataset page huggingface.co/datasets/SLPL/naab ↗
landing page · from Hugging Face
Documentation and papers
- arXiv:2208.13486 arxiv.org/abs/2208.13486 ↗
publication · from Hugging Face
Catalogue records · 1
- Hub API huggingface.co/api/datasets/SLPL/naab ↗
metadata API · from Hugging Face
Topics
- Stated by source
- fill mask · text · text generation
- From keywords
- Computer Science & AI
- Inferred from text
- Artificial intelligence 74%
Provenance · 1 source records, 12 field assertions
| Source | Key | Last seen | Raw |
|---|---|---|---|
| Hugging Face Datasets | SLPL/naab | 14 d ago | JSON v1 |
| Field | Assertion | Extractor | Evidence |
|---|---|---|---|
| access_level | source · Hugging Face | connector:huggingface@1.0.0 | /gated |
| concepts[field].anzsrc:group:4602 | enrichment · Hugging Face | taxonomy-embedding@1.1.0 | title+keywords+description (74%) |
| concepts[field].local:field:computer-science-ai | mapping · Hugging Face | connector:huggingface@1.0.0 | |
| concepts[modality].hf_modality:text | source · Hugging Face | connector:huggingface@1.0.0 | |
| concepts[task].hf_task:fill-mask | source · Hugging Face | connector:huggingface@1.0.0 | /tags[task_categories:*] |
| concepts[task].hf_task:text-generation | source · Hugging Face | connector:huggingface@1.0.0 | /tags[task_categories:*] |
| created_date | source · Hugging Face | connector:huggingface@1.0.0 | |
| description | source · Hugging Face | connector:huggingface@1.0.0 | /description |
| license | source · Hugging Face | connector:huggingface@1.0.0 | /tags[license:*] |
| publication_date | source · Hugging Face | connector:huggingface@1.0.0 | |
| title | source · Hugging Face | connector:huggingface@1.0.0 | /id |
| updated_date | source · Hugging Face | connector:huggingface@1.0.0 |