Skip to content

[hipDNN] Fix backward activation reference input order#7736

Open
SamuelReeder wants to merge 1 commit into
developfrom
users/sareeder/fix-relu-backward-reference-order
Open

[hipDNN] Fix backward activation reference input order#7736
SamuelReeder wants to merge 1 commit into
developfrom
users/sareeder/fix-relu-backward-reference-order

Conversation

@SamuelReeder
Copy link
Copy Markdown
Contributor

@SamuelReeder SamuelReeder commented May 25, 2026

Summary

This PR fixes hipDNN backward activation reference and graph construction to consume upstream gradient dy before forward input x. It updates direct pointwise, CPU graph executor, flatbuffer graph utility, hip-kernel-provider and MIOpen-provider fused batchnorm handling, and fused BN inference + DReLU + BN backward sample call sites so backward activation input ordering is consistent.

Risk Assessment

Low risk. This is a narrow CPU reference/test SDK/sample/provider ordering fix with targeted local coverage and no public API, schema, or build configuration changes; PR CI is pending.

Testing Summary

  • Built hipdnn_test_sdk_tests to verify affected test SDK sources compile after the ordering change.
  • Built and ran all hipDNN sample tests, including the fused BN inference + DReLU + BN backward sample path updated by this PR.
  • Built hip-kernel-provider unit and integration test targets in a superbuild configuration using this branch's hipDNN sources.
  • Built MIOpen-provider unit and integration test targets in a superbuild configuration using this branch's hipDNN sources.
  • Ran hip-kernel-provider unit tests, including the fused batchnorm backward plan-builder tests that failed in CI.
  • Ran targeted MIOpen-provider fused batchnorm backward unit tests covering plan builder, applicability, and fused parameter binding.
  • Ran targeted backward activation and pointwise signature unit tests covering ReLU, parameterized ReLU, sigmoid, tanh, and mixed dy/x datatypes.
  • Ran formatting and diff whitespace checks on changed files.

Testing Checklist

  • hipDNN test SDK build - cmake --build build --target hipdnn_test_sdk_tests - Status: Passed
  • hipDNN samples build - cmake --build build - Status: Passed
  • hip-kernel-provider superbuild test targets - cmake --build build-hipkernel-pr --target hip_kernel_provider_tests hip_kernel_provider_integration_tests - Status: Passed
  • MIOpen-provider superbuild test targets - cmake --build build-miopen-pr --target miopen_plugin_tests miopen_plugin_integration_tests - Status: Passed
  • hip-kernel-provider unit tests - ./build-hipkernel-pr/bin/hip_kernel_provider_tests - Status: Passed
  • hip-kernel-provider fused ReLU backward unit tests - ./build-hipkernel-pr/bin/hip_kernel_provider_tests --gtest_filter='TestBatchnormPlanBuilder.IsApplicableReturnsTrueForValidThreeNodeGraph:TestBatchnormPlanBuilder.BuildPlanSetsPlanForFusedBackwardGraph:TestBatchnormBwdParams.InitializesFusedActivationWithAllTensors:TestBatchnormValidator.ValidInferenceActivationBackward' - Status: Passed
  • MIOpen-provider fused ReLU backward unit tests - ./build-miopen-pr/bin/miopen_plugin_tests --gtest_filter='*FusedThreeNodeGraph*:*FusedGraph*:*BatchnormBwdParams*Fused*:*FusedMode*:*FusedBackward*:*ActivationBackward*:*ActivationMissingIn1*:*ActivationOutputDoesNotMatchBnBackwardDy*:*BnInferenceOutputDoesNotMatchActivationInput*' - Status: Passed
  • hipDNN targeted backward activation and pointwise signature tests - ./build/bin/hipdnn_test_sdk_tests --gtest_filter='*PointwiseSignatureKey*:*PlanRegistrySignatureKey*:*ReluBackward*:*ParameterizedReluBackward*:*SigmoidBackward*:*TanhBackward*' - Status: Passed
  • hipDNN failing fused DReLU sample test - ctest --test-dir build -R hipdnn_sample_fused_bn_inference_drelu_bn_backward --output-on-failure - Status: Passed
  • hipDNN sample tests - ctest --test-dir build --output-on-failure --parallel 8 - Status: Passed
  • Formatting - clang-format-18 --dry-run -Werror <changed files> - Status: Passed
  • Diff whitespace - git diff --check -- <changed files> - Status: Passed
  • PR CI - GitHub PR checks - Status: Pending

Technical Changes

  • Changes ReluBackward, ParameterizedReluBackward, SigmoidBackward, and TanhBackward functor signatures to take dy before x while preserving downstream gradient calculations.
  • Adds CPU graph executor plan registrations for dy-first backward activation signatures where dy is input type and the forward activation input is float.
  • Updates direct CPU reference tests and CPU graph executor ReLU backward fixtures to pass and fill inputs in dy-first order, preserving expected output semantics.
  • Updates flatbuffer graph test utility, fused BN inference + DReLU + BN backward sample construction, hip-kernel-provider fused batchnorm backward parsing, and MIOpen-provider fused batchnorm backward parsing to use dy-first ReLU backward inputs.

@SamuelReeder SamuelReeder self-assigned this May 25, 2026
@SamuelReeder SamuelReeder force-pushed the users/sareeder/fix-relu-backward-reference-order branch from ece9af8 to 2dc7dd9 Compare May 25, 2026 14:33
@SamuelReeder SamuelReeder marked this pull request as ready for review May 25, 2026 14:35
@SamuelReeder SamuelReeder requested a review from a team as a code owner May 25, 2026 14:35
@SamuelReeder SamuelReeder linked an issue May 25, 2026 that may be closed by this pull request
@SamuelReeder SamuelReeder enabled auto-merge (squash) May 25, 2026 14:57
@SamuelReeder SamuelReeder disabled auto-merge May 25, 2026 15:17
Copy link
Copy Markdown
Contributor

@CMiservaAMD CMiservaAMD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@SamuelReeder SamuelReeder force-pushed the users/sareeder/fix-relu-backward-reference-order branch from 2dc7dd9 to 1d687ab Compare May 25, 2026 15:53
@SamuelReeder SamuelReeder marked this pull request as draft May 25, 2026 15:55
@SamuelReeder SamuelReeder marked this pull request as ready for review May 25, 2026 16:07
@SamuelReeder SamuelReeder force-pushed the users/sareeder/fix-relu-backward-reference-order branch from 1d687ab to 1d8f4aa Compare May 25, 2026 16:35
@SamuelReeder SamuelReeder requested review from a team as code owners May 25, 2026 16:35
@SamuelReeder SamuelReeder marked this pull request as draft May 25, 2026 16:35
@SamuelReeder SamuelReeder marked this pull request as ready for review May 25, 2026 17:01
@SamuelReeder
Copy link
Copy Markdown
Contributor Author

Editing MIOpen provider too...

@SamuelReeder SamuelReeder force-pushed the users/sareeder/fix-relu-backward-reference-order branch from 1d8f4aa to 37a4b6e Compare May 25, 2026 17:27
@SamuelReeder SamuelReeder force-pushed the users/sareeder/fix-relu-backward-reference-order branch 2 times, most recently from f8f89b4 to 3efd97c Compare May 25, 2026 18:21
@SamuelReeder SamuelReeder changed the title [hipDNN] Fix ReLU backward reference input order [hipDNN] Fix backward activation reference input order May 25, 2026
@SamuelReeder SamuelReeder force-pushed the users/sareeder/fix-relu-backward-reference-order branch from 3efd97c to 21d16fe Compare May 25, 2026 20:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[hipDNN] ReluBackward parameter issue

4 participants