cryoPANDA: A 37-million-particle dataset from over 250 experiments to accelerate data-driven cryo-EM analysis

For each experiment, the dataset includes its particles, per-particle annotations, the corresponding 3D electrostatic potential map, the published Electron Microscopy Data Bank map and Protein Data Bank model, where available.The data are arranged in separate directories, one per cryo-EM experiment. Each experiment directory contains two zipped subdirectories: particles.zip, which stores restacked particles in batches of 1,000, and particles_information.zip, which contains metadata related to restack, 2D classification, and 3D reconstruction jobs.

These jobs are further organized into dedicated subdirectories and include output files from cryoSPARC, such as .png images and metadata files in .cs, .mrc, and .star formats. In addition to these two subdirectories, the main experiment directory contains (i) particles.star, which provides particle annotations required for importing particles into software packages such as cryoSPARC, (ii) extended per-particle annotations in .xlsx format, (iii) the reconstructed electrostatic potential map produced by cryoPANDA in .mrc format, (iv) the corresponding EMDB map in .map format, and, where available, (v) an associated PDB atomic model in .cif format.The cryoPANDA dataset is also provided in HDF5 format, chunked by experiment into 252 separate .h5 files.

Within each HDF5 file, particle images and per-particle annotations can be accessed under the keys 'particles' and 'annotations', respectively. All particle images are resampled to 224x224 pixels, stored as uint8, and normalized per image to the [0–255] range. Combined with the smaller total size (1.8~TB) and the ease of distribution, this format significantly improves sampling efficiency during the training of machine-learning models.