Table · dataset · 2022
OpusParaCrawl
Listed in Hugging Face Datasets
Dataset Card for OpusParaCrawl Dataset
Description
Summary
Parallel corpora from Web Crawls collected in the ParaCrawl project. Tha dataset contains: 42 languages, 43 bitexts total number of files: 59,996 total number of tokens: 56.11G total number of sentence fragments: 3.13G To load a language pair which isn't part of the config, all you need to do is specify the language code as pairs, e.g. dataset = load_dataset("opus_paracrawl", lang1="en", lang2="so") You can find the valid… See the full description on the dataset page: huggingface.co/datasets/Helsinki-NLP/opus_paracrawl.
Links
Where it is published
- Hugging Face dataset page huggingface.co/datasets/Helsinki-NLP/opus_paracrawl ↗
landing page · from Hugging Face
Catalogue records · 1
- Hub API huggingface.co/api/datasets/Helsinki-NLP/opus_paracrawl ↗
metadata API · from Hugging Face
Topics
- Stated by source
- text · translation
- From keywords
- Computer Science & AI
Provenance · 1 source records, 10 field assertions
| Source | Key | Last seen | Raw |
|---|---|---|---|
| Hugging Face Datasets | Helsinki-NLP/opus_paracrawl | 15 d ago | JSON v1 |
| Field | Assertion | Extractor | Evidence |
|---|---|---|---|
| access_level | source · Hugging Face | connector:huggingface@1.0.0 | /gated |
| concepts[field].local:field:computer-science-ai | mapping · Hugging Face | connector:huggingface@1.0.0 | |
| concepts[modality].hf_modality:text | source · Hugging Face | connector:huggingface@1.0.0 | |
| concepts[task].hf_task:translation | source · Hugging Face | connector:huggingface@1.0.0 | /tags[task_categories:*] |
| created_date | source · Hugging Face | connector:huggingface@1.0.0 | |
| description | source · Hugging Face | connector:huggingface@1.0.0 | /description |
| license | source · Hugging Face | connector:huggingface@1.0.0 | /tags[license:*] |
| publication_date | source · Hugging Face | connector:huggingface@1.0.0 | |
| title | source · Hugging Face | connector:huggingface@1.0.0 | /id |
| updated_date | source · Hugging Face | connector:huggingface@1.0.0 |