MBBMD: Media Bias Bias-Mitigated Dataset

It includes percentage scores reflecting the degree of agreement. Level 2: Multilabel bias classificationThis level categorizes bias into five specific types: intentional bias, spin bias, statement bias, coverage bias, and gatekeeping bias. Each type includes binary majority vote annotations and percentage agreement scores from annotators.

To enhance annotation robustness, Counterfactual Data Augmentation (CDA) techniques were applied to a subset of the dataset. These modifications involve outlet swaps, entity swaps, and terminological changes, allowing for the assessment of how these factors influence annotator perceptions of bias. Dataset File Structure and Field Descriptions The Multilevel Bias Detection Dataset for Spanish Media (MBBMD) is organized into two main directories, corresponding to the two annotation phases: Phase 1 (Document-level annotations): Focused on binary and multilabel bias classification at the article level.

Phase 2 (Sentence-level annotations): Provides fine-grained sentence-level bias annotations. Each directory contains three subsets: train: The primary dataset for training models. test: A reserved subset for model evaluation. control: Contains articles modified using Counterfactual Data Augmentation (CDA) to analyze annotation robustness. The dataset files are stored in TSV (Tab-Separated Values) format, encoded in UTF-8, ensuring compatibility with most data processing tools.

Phase 1: Document-Level Annotations Files: phase_1/train_phase1.tsv phase_1/test_phase1.tsv phase_1/control_phase1.tsv Fields in Phase 1 files Each row represents a full news article annotated for bias. topic: The topic of the article (e.g., Pedro Sánchez Investiture, Barcelona Amnesty Protest). text_id: A unique identifier for each article. title: The headline of the article. text: The full body of the news article. is_biased_majority_vote: Binary label (1 = biased, 0 = not biased), determined by the majority vote of annotators. is_biased_%: The percentage of annotators who classified the article as biased.

Phase 2: Multilabel Bias Classification at the Document Level Files: phase_1/train_phase2.tsv phase_1/test_phase2.tsv phase_1/control_phase2.tsv Fields in Phase 2 files Each row corresponds to a news article, but with bias annotations broken down into five specific categories. topic: Topic of the article. text_id: Unique identifier for the article. title: Headline of the article. text: Full text of the article. is_intentional_bias_majority_vote: Binary label (1 = present, 0 = absent) indicating whether intentional bias is detected. is_spin_bias_majority_vote: Binary label indicating the presence of spin bias. is_statement_bias_majority_vote: Binary label indicating statement bias. is_coverage_bias_majority_vote: Binary label indicating coverage bias. is_gatekeeping_bias_majority_vote: Binary label indicating gatekeeping bias. is_intentional_bias_%: Percentage of annotators who identified intentional bias. is_spin_bias_%: Percentage of annotators who identified spin bias. is_statement_bias_%: Percentage of annotators who identified statement bias. is_coverage_bias_%: Percentage of annotators who identified coverage bias. is_gatekeeping_bias_%: Percentage of annotators who identified gatekeeping bias.

These fields enable multilabel classification, allowing researchers to analyze different dimensions of bias simultaneously.