Comprehensive Benchmarking and Validation Framework Enhancement#307
Comprehensive Benchmarking and Validation Framework Enhancement#307jrfaeder wants to merge 4 commits into
Conversation
- Expand benchmark suite with diverse models (egfr_net, fceri_ji, fceri_fyn, gene_expr, simple_nfsim). - Integrate peak memory (RSS) tracking using a custom Python helper. - Add phase-level timing breakdown (generation vs. simulation). - Add NFsim throughput metrics (events per second). - Integrate optional automated validation with the existing suite via --validate flag. - Improve model discovery logic and overall script robustness. - Update documentation in README.md and USAGE.md. - Compile and integrate NFsim from source, including fixes for modern C++ compilers. - Ensure all 73 standard models pass the validation suite. Co-authored-by: jrfaeder <8277248+jrfaeder@users.noreply.github.com>
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
- Expand benchmark suite with diverse models. - Integrate peak memory (RSS) tracking. - Add phase-level timing breakdown. - Add NFsim throughput metrics and submodule. - Integrate optional automated validation via --validate flag. - Improve model discovery logic and script robustness. - Update documentation. - Fix C++ compatibility issues in NFsim for modern compilers. Co-authored-by: jrfaeder <8277248+jrfaeder@users.noreply.github.com>
- Expand benchmark suite with diverse models. - Integrate peak memory (RSS) tracking. - Add phase-level timing breakdown. - Add NFsim throughput metrics. - Add NFsim submodule pointing to upstream master. - Integrate optional automated validation via --validate flag. - Improve script robustness and update documentation. Co-authored-by: jrfaeder <8277248+jrfaeder@users.noreply.github.com>
- Expand benchmark suite with diverse models. - Integrate peak memory (RSS) tracking. - Add phase-level timing breakdown. - Add NFsim throughput metrics. - Add NFsim submodule pointing to upstream master. - Integrate optional automated validation via --validate flag. - Improve script robustness and update documentation. - Add CI patching logic for NFsim compilation. Co-authored-by: jrfaeder <8277248+jrfaeder@users.noreply.github.com>
1 participant
This PR significantly enhances the BioNetGen benchmarking framework to provide comprehensive performance and validation metrics. Key improvements include tracking peak memory usage, separate timing for network generation and simulation phases, and throughput measurement for NFsim models. The framework now optionally runs the full validation suite for benchmarked models to ensure accuracy alongside performance. Additionally, NFsim has been compiled from source and integrated into the environment. Documentation has been updated to reflect these new capabilities.
PR created automatically by Jules for task 10604326215037883847 started by @jrfaeder