Skip to content

feat(benchmarks): add QFUERZA-recovery results for all 5 TS systems#11

Open
ericchansen wants to merge 1 commit into
mainfrom
feat/qfuerza-recovery-results
Open

feat(benchmarks): add QFUERZA-recovery results for all 5 TS systems#11
ericchansen wants to merge 1 commit into
mainfrom
feat/qfuerza-recovery-results

Conversation

@ericchansen
Copy link
Copy Markdown
Owner

Summary

Adds benchmarks/<system>/from-qfuerza/ artifacts for all 5 TS systems from the QFUERZA-recovery validation experiment. Companion to ericchansen/q2mm#290.

What's in each directory

File Content
validation_results.json Full run record: provenance (q2mm git SHA, JAX device), starting-point audit (which OPT rows were QFUERZA-overwritten vs published-retained), initial/final objective scores, Seminario R², optimizer trajectory, ratio diagnostics
paper_metrics.json Published-paper-comparable summary metrics
<system>_optimized.fld Final optimized force field

Experiment

Each run starts from QFUERZA Hessian-derived bond/angle values (per-molecule Seminario with TS inversion, multi-molecule mean) overlaid on the published OPT topology (frozen partition, vdW, SB, atom-type rows from the published FF). Optimized with SciPy L-BFGS-B + JaxLoss, --ratio-tol -1, run on RTX 5090 (WSL2).

This is not from-scratch FF generation — only 17–33% of active parameters per system are overwritten by QFUERZA. See docs/benchmarks/qfuerza-recovery.md (in the companion PR) for full methodology and interpretation.

Results summary

QFUERZA-start vs published-start final objective:

System Pub. final OF QFUERZA final OF Ratio Improvement %
rh-enamide 2.70 × 10⁵ 2.78 × 10⁵ 1.03× +28.9%
pd-allyl 7.99 × 10⁶ 7.98 × 10⁶ 1.00× +0.41%
pd-conjugate 7.24 × 10⁶ 8.25 × 10⁶ 1.14× −0.20%
rh-conjugate 5.10 × 10⁶ 1.78 × 10⁷ 3.49× +33.4%
heck-relay 1.45 × 10⁶ 1.45 × 10⁸ 100× −7.70%

rh-enamide and pd-allyl converge to essentially the same basin as the published-start runs. The other three land in different (worse) basins, with heck-relay failing due to JaxLoss surrogate divergence at the poor starting FF (bond R² = −247 at the QFUERZA starting point).

File size

~530 KB total, all JSON + small .fld files. No large binary artifacts.

Provenance

  • q2mm git SHA: 29b61f8 (loader commit on feat/qfuerza-from-scratch)
  • q2mm-data git SHA at run time: 72add3d
  • JAX device: cuda:0 (RTX 5090, WSL2)
  • ratio_tol: -1 (bypassed)

Adds benchmarks/<system>/from-qfuerza/ artifacts for rh-enamide,
heck-relay, pd-allyl-amination, pd-1,4-conjugate-addition, and
rh-1,4-conjugate-addition. Each directory contains:

- validation_results.json — full run record with provenance, audit, R²
- paper_metrics.json — published-paper-comparable metrics
- <system>_optimized.fld — final optimized force field

These runs start from QFUERZA Hessian-derived bond/angle values
(overwriting the published OPT scalars) and run the standard SciPy
L-BFGS-B + JaxLoss pipeline. See ericchansen/q2mm#290 for the loader
and CLI code, and docs/benchmarks/qfuerza-recovery.md (in that PR) for
the methodology and interpretation.

Summary (QFUERZA vs published-start final OF ratio):
- rh-enamide: 1.03x (same basin)
- pd-allyl: 1.00x (same basin)
- pd-conjugate: 1.14x (nearby basin)
- rh-conjugate: 3.49x (different basin)
- heck-relay: 100x (JaxLoss surrogate diverged)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant