BinFuncGt

It provides function-level ground truth, static feature extraction, candidate retrieval, call-graph-aware structural matching, and evaluation of component and version identification.</p><p dir="ltr">The project, benchmark, and construction utilities are named <b>BinFuncGT</b>. Python modules and filesystem paths use the lowercase spelling <code>binfuncgt</code>.</p><h2 dir="ltr">Features</h2><ul><li><b>Binary analysis:</b> Parse ELF metadata, symbol tables, and DWARF information, and recover functions and call graphs with angr CFGFast.</li><li><b>Function representation:</b> Extract instruction counts, basic-block counts, control-flow edges, call sites, immediate-constant statistics, cyclomatic complexity, and mnemonic histograms.</li><li><b>Candidate retrieval:</b> Generate candidate function pairs using cosine similarity, a top-r selection rule, and a similarity threshold.</li><li><b>Structural matching:</b> Combine local similarity, preserved call relationships, and one-to-one matching constraints in a unified optimization objective.</li><li><b>Reduction and classical solvers:</b> Apply safe reductions and interaction-graph decomposition, with greedy, Hungarian, CP-SAT, simulated annealing, and small-instance exhaustive solvers.</li><li><b>Function-level evaluation:</b> Measure precision, recall, and F1 against raw and strict ground-truth correspondences.</li><li><b>Version identification:</b> Explore constant features, string features, differential weighting, and version-discriminative string scoring.</li><li><b>Statistical analysis:</b> Provide paired comparisons, ranking diagnostics, and Benjamini–Hochberg false discovery rate correction.</li></ul><h2 dir="ltr">Repository Structure</h2><p dir="ltr">The main directories and modules are organized as follows:</p><pre><pre>.<br>├── configs/ # Benchmark and experiment configurations<br>├── data/ # Binary data, intermediate features, and instances<br>├── experiments/<br>│ ├── pilot/ # Matching, baseline, ablation, and ranking results<br>│ │ └── score_cache_v2/ # Per-instance scoring caches<br>│ ├── case_study/ # Debian case-study results and reports<br>│ └── versiongrid_v3/ # Extended version-grid design and build records<br>├── results/ # Aggregate results, statistics, and logs<br>├── scripts/ # Experiment and report-generation entry points<br>├── src/binfuncgt/<br>│ ├── binary/ # ELF parsing, call graphs, and ground truth<br>│ ├── data/ # BinFuncGT schemas and instance I/O<br>│ ├── features/ # Static features and vectorization<br>│ ├── retrieval/ # Candidate function retrieval<br>│ ├── matching/ # Matching instances and objectives<br>│ ├── reduction/ # Safe and heuristic reductions<br>│ ├── solvers/ # Classical optimization methods<br>│ ├── scoring/ # Downstream structural scores<br>│ ├── evaluation/ # Evaluation metrics<br>│ └── feasibility/ # Classical matching and solver statistics<br>└── requirements.txt # Pinned Python dependencies<br></pre></pre><h2 dir="ltr">Configuration</h2><p dir="ltr">The principal benchmark configurations are:</p><table><tr><th><p dir="ltr">Configuration</p></th><th><p dir="ltr">Experiment setting</p></th></tr><tr><td><code>configs/binkit_x86_v2.yaml</code></td><td><p dir="ltr">Same-architecture matching across build variants</p></td></tr><tr><td><code>configs/binkit_xarch_v2.yaml</code></td><td><p dir="ltr">Cross-architecture matching with the same compiler and optimization level</p></td></tr><tr><td><code>configs/pilot_versiongrid_v2.yaml</code></td><td><p dir="ltr">Component and version comparisons on VersionGrid v2</p></td></tr><tr><td><code>configs/pilot_versiongrid_v3.yaml</code></td><td><p dir="ltr">Extended version-grid comparisons</p></td></tr></table><p dir="ltr">The <code>x86</code> configuration name is historical: its pairing rule is <code>arch_relation: same</code>, covering matching architectures represented in the input corpus.</p><p dir="ltr">Common settings include:</p><table><tr><th><p dir="ltr">Parameter</p></th><th><p dir="ltr">Main protocol setting</p></th></tr><tr><td><p dir="ltr">Candidate count per target function</p></td><td><code>r = 5</code></td></tr><tr><td><p dir="ltr">Cosine-similarity threshold</p></td><td><code>0.5</code></td></tr><tr><td><p dir="ltr">Same-architecture feature set</p></td><td><code>full</code></td></tr><tr><td><p dir="ltr">Cross-architecture feature set</p></td><td><code>structural</code></td></tr><tr><td><p dir="ltr">Matching objective weights</p></td><td><code>alpha = beta = 1</code></td></tr><tr><td><p dir="ltr">CP-SAT time budget</p></td><td><p dir="ltr">60 seconds by default in the main scoring runners</p></td></tr><tr><td><p dir="ltr">Structural score weights</p></td><td><code>(0.5, 0.3, 0.2)</code></td></tr></table><h2 dir="ltr">Experiment Scripts</h2><table><tr><th><p dir="ltr">Script</p></th><th><p dir="ltr">Purpose</p></th></tr><tr><td><code>scripts/rq1_function_gt.py</code></td><td><p dir="ltr">Function-level matching evaluation</p></td></tr><tr><td><code>scripts/rq1_function_gt_report.py</code></td><td><p dir="ltr">Function-level result summaries and paired comparisons</p></td></tr><tr><td><code>scripts/run_feasibility.py</code></td><td><p dir="ltr">Structural matching and per-instance feasibility statistics</p></td></tr><tr><td><code>scripts/analyze_results.py</code></td><td><p dir="ltr">Pilot result aggregation</p></td></tr><tr><td><code>scripts/score_v2.py</code></td><td><p dir="ltr">Multi-method benchmark scoring</p></td></tr><tr><td><code>scripts/report_v2.py</code></td><td><p dir="ltr">Scoring-cache summaries and statistical comparisons</p></td></tr><tr><td><code>scripts/run_baselines.py</code></td><td><p dir="ltr">Classical baselines through</p> <code>run</code> <p dir="ltr">and</p> <code>summarize</code> <p dir="ltr">subcommands</p></td></tr><tr><td><code>scripts/rq5_feature_attribution.py</code></td><td><p dir="ltr">Feature attribution analysis</p></td></tr><tr><td><code>scripts/rq6_stripped.py</code></td><td><p dir="ltr">Evaluation with stripped target binaries</p></td></tr><tr><td><code>scripts/rq7_diff_weighted.py</code></td><td><p dir="ltr">Differential feature-weighting experiments</p></td></tr><tr><td><code>scripts/rq7_tie_audit.py</code></td><td><p dir="ltr">Tie and candidate-set diagnostics</p></td></tr><tr><td><code>scripts/rq7_str_scorer.py</code></td><td><p dir="ltr">Version-discriminative string scoring</p></td></tr><tr><td><code>scripts/rq7_stats.py</code></td><td><p dir="ltr">Statistical comparisons of string-based methods</p></td></tr><tr><td><code>scripts/fdr_correction.py</code></td><td><p dir="ltr">Benjamini–Hochberg FDR correction</p></td></tr><tr><td><code>scripts/debian_case_study.py</code></td><td><p dir="ltr">Debian case-study evaluation</p></td></tr><tr><td><code>scripts/debian_str_idf.py</code></td><td><p dir="ltr">String-IDF evaluation on Debian binaries</p></td></tr></table><p></p>