[hipDNN] Add RMSNorm backward CPU reference implementation#7494
Merged
saikubairkota merged 5 commits intoMay 29, 2026
Conversation
Codecov Report❌ Patch coverage is ❌ Your project status has failed because the head coverage (77.83%) is below the target coverage (80.00%). You can increase the head coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## develop #7494 +/- ##
===========================================
+ Coverage 61.93% 61.95% +0.02%
===========================================
Files 2084 2086 +2
Lines 357263 357576 +313
Branches 54006 54043 +37
===========================================
+ Hits 221253 221521 +268
- Misses 117207 117231 +24
- Partials 18803 18824 +21
*This pull request uses carry forward flags. Click here to find out more.
🚀 New features to boost your workflow:
|
3ccd5f5 to
7dc6f79
Compare
7dc6f79 to
66fda72
Compare
adickin-amd
reviewed
May 21, 2026
adickin-amd
reviewed
May 21, 2026
adickin-amd
reviewed
May 21, 2026
adickin-amd
reviewed
May 21, 2026
Merged
1 task
77761aa to
88871d4
Compare
Contributor
|
looked like CI hit some infra errors on your PR. I merged in develop to get it to run again |
adickin-amd
approved these changes
May 27, 2026
Contributor
adickin-amd
left a comment
There was a problem hiding this comment.
LGTM, miopen-provider failure is unrelated to these change. the code coverage issue also looks unrelated. Ive brought up the code coverage issue with hipdnn-core to discuss if we can exlude mocks from coverage.
0e2136d to
2d62162
Compare
The CpuFpReferenceRMSNorm tests are not yet added!
…RMSNorm backward reference implementation
…sorAttr availability check in constructor
…l dbias in RMSNormBwdPlan
2d62162 to
89bafdb
Compare
saikubairkota
added a commit
that referenced
this pull request
May 29, 2026
…channel last support (#7702) **Caution**: This PR should be merged only after [this PR](#7494) is merged. ## Motivation This PR implements the RMSNorm backward kernels and RMSNorm channel-last support for both forward and backward operations in the hip kernel provider. ## Technical Details - Adds the RMSNorm backward kernels and makes relevant changes in `RMSnormBwdPlan` to compile and launch the kernels. - Adds channel last support for both `RMSnormFwd` and `RMSnormBwd` operations. - Adds/updates unit tests and integration tests to test the changes introduced in this PR. ## Test Plan Build the plugin and run the unit and integration tests with `ninja check`. ## Test Result All unit and integration tests pass successfully on an MI210. ## Submission Checklist - [ ] Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
This PR adds the RMSNorm backward CPU reference implementation to the hipDNN test SDK.
Technical Details
CpuFpReferenceRMSNorm::backward()that calculates the gradientsdx,dscale, anddbias.RMSNormBwdPlan,RMSNormBwdPlanBuilder, andRMSNormBwdSignatureKeyfor executing RMSNorm backward graph operations.Test Plan
Build
hipDNNand run the relevant unit tests with./bin/hipdnn_test_sdk_tests --gtest_filter="*RMSNormBwd*".Test Result
All relevant unit tests pass successfully on an MI210.
ninja check- full test suiteSubmission Checklist