Hybrid

Each scenario specifies a world description, matched fictional and real-world topics, an epistemic-distance score, and admissible text genres.<br><br>For each scenario and argument-graph size, the generation pipeline creates two paired generation directions:<br><br>- **F2R:** a fictional-world argument map is verbalized and then minimally adapted into a real-world text;<br>- **R2F:** a real-world argument map is verbalized and then minimally adapted into a fictional-world text.<br><br>Each direction contains six variants:<br><br>1. `1`: valid argumentation in a fictional world;<br>2. `2a`: invalid argumentation with light reasoning errors in a fictional world;<br>3. `2b`: invalid argumentation with serious reasoning errors in a fictional world;<br>4. `3`: valid argumentation in the real world;<br>5. `4a`: invalid argumentation with light reasoning errors in the real world;<br>6. `4b`: invalid argumentation with serious reasoning errors in the real world.<br><br>The archive includes 5-node and 10-node argument settings.

Across both settings, the generation design yields 1,200 generated texts before model-specific evaluation records are considered.<br><br>The released texts were generated with `anthropic/claude-opus-4.7` using temperature 0.7, a 2,500-token output limit, and seed 42. Validity was then evaluated through OpenRouter at temperature 0.0 with a 65,536-token output limit. The model identifier, raw response, parsed label, and parsing status are retained for each evaluation record.<br><br>## Files<br><br>- `generation_items.json` — the structured inventory of benchmark scenarios and metadata.<br>- `hybrid_benchmark_prompts.txt` — the complete prompt templates used for argument-map generation, corruption, verbalization, cross-world adaptation, and validity evaluation.

Pipeline placeholders are retained.<br>- `openrouter_dataset_node5.evaluated.all_models.completed12.csv` — model-level validity evaluations for arguments with 5 nodes.<br>- `openrouter_dataset_node10.evaluated.all_models.completed12.csv` — model-level validity evaluations for arguments with 10 nodes.<br><br>The CSV files contain the scenario and generation metadata (`item_id`, `category`, `epistemic_distance`, `direction`, `variant`, `node_n`, `genre`, and `noise_level`); the world description and argument-map information; the generated argument text; and evaluation fields.

The latter include the constructed ground-truth label (`eval_ground_truth`), world scope (`eval_world_scope`), parsed model prediction (`eval_prediction`), parser status and reason, correctness indicator, raw model output, and model identifier. Each file contains results from 12 evaluated model identifiers.<br><br>Some CSV text fields contain embedded line breaks. Use a standards-compliant CSV reader that supports quoted multiline fields.<br><br>## Important limitations<br><br>The validity labels are derived from the controlled generation procedure: variants 1 and 3 are labeled valid, whereas variants 2 and 4 are labeled invalid.

Invalid texts are produced by intentionally modifying argument maps. These labels are not a substitute for complete human annotation, and the associated study explicitly notes the possibility of residual label noise. The texts are synthetic rather than a sample of naturally occurring argumentation; this increases experimental control but limits ecological validity.

The archive is intended for research, replication, analysis, and benchmark development; users should independently validate examples before using them as high-stakes ground truth.<br><br>This sub-archive contains finalized data and prompt templates, not the generation or evaluation scripts. Exact regeneration also depends on continued access to the specified external model and inference service.<br><br>## Citation<br><br>If you use this archive, please cite the associated manuscript:<br><br>Filip Gampel, Krzysztof Zaborek, and Rafał Olszowski. <i>*Non-Factual Argument Mining: Stress-Testing LLM Reasoning Beyond Factual Familiarity*</i>.</pre><p></p>