Zhaoming213/baike_qa

数据集介绍 这是社区问答对话数据集，对原始数据集进行了简单清洗，最终变成了类似:{"text":....}对结构，jsonl文件。 清洗代码 import json datas = [] with open("baike_qa_train.json","r",encoding="UTF-8") as f: for line in f: datas.append(json.loads(line)) with open("baikeqa.jsonl","w",encoding="UTF-8") as f: for lines in datas: if len(lines['title']) + len(lines['answer']) <512: sections = {"text":f"{lines['title']}{lines['answer']}"} f.write(json.dumps(sections… See the full description on the dataset page: huggingface.co/datasets/Zhaoming213/baike_qa.