Skip to content

NativeEngine: support standalone sampleable depth/stencil textures#1738

Draft
bkaradzic-microsoft wants to merge 2 commits into
BabylonJS:masterfrom
bkaradzic-microsoft:fix/native-depthstencil-framebuffer
Draft

NativeEngine: support standalone sampleable depth/stencil textures#1738
bkaradzic-microsoft wants to merge 2 commits into
BabylonJS:masterfrom
bkaradzic-microsoft:fix/native-depthstencil-framebuffer

Conversation

@bkaradzic-microsoft
Copy link
Copy Markdown
Contributor

What

NativeEngine::CreateFrameBuffer now supports Babylon's request for a standalone, sampleable depth/stencil texture.

thinNativeEngine.pure.ts::_createDepthStencilTexture creates such a texture by calling createFrameBuffer(...) with a freshly-created (and therefore uninitialized) color texture — its bgfx handle is still kInvalidHandle. CreateFrameBuffer only guarded against a null texture, so it attached the invalid handle as a color target. bgfx framebuffer validation then rejected it (Invalid texture attachment) and threw, which aborted the entire headless Playground sweep, not just the offending test.

How

Detect the request (non-null texture whose bgfx handle is invalid) and instead:

  • skip attaching the invalid color handle → build a depth-only framebuffer;
  • allocate the depth attachment as a readable BGFX_TEXTURE_RT (instead of BGFX_TEXTURE_RT_WRITE_ONLY) so it can be sampled;
  • alias the framebuffer's depth attachment back into the caller-supplied texture (ownsHandle = false, since the framebuffer owns/destroys the handle) so Babylon can sample it (e.g. fluid rendering's depth copy).

Verification

  • Built Playground (Win32 D3D11, RelWithDebInfo) off master + this change.
  • The fluid-rendering test that previously threw Invalid texture attachment and aborted the sweep now runs to completion with a clean exit (no abort, no stderr). RenderDoc confirms the aliased depth/stencil texture holds correct scene depth.

Does this re-enable any excluded tests?

Honest answer: not on its own. I A/B-tested every crash-reason–excluded test (built with and without this change, run in isolation):

  • No currently-excluded test except the fluid one exercises this code path, so none are unblocked by this fix.
  • The fluid test itself still cannot be re-enabled: with the crash gone it renders black because of a separate, pre-existing NativeEngine GPU-particle limitation — the fluid particle vertex shader never receives per-instance particle positions (they're uploaded to bgfx instance slot TEXCOORD7/i_data0, but the compiled shader reads POSITION0 from the per-vertex buffer), so all particles render zero-area quads and produce no fragments. That's tracked separately and needs a fix in NativeEngine's instanced-attribute routing.

This PR is therefore a robustness/correctness fix: it removes a whole-sweep abort and makes standalone depth/stencil textures behave per Babylon's contract (sampleable), independent of the unrelated particle issue.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

Babylon's NativeEngine._createDepthStencilTexture requests a standalone,
sampleable depth/stencil texture by calling createFrameBuffer with a
freshly-created (and therefore uninitialized) color texture whose bgfx
handle is still kInvalidHandle. CreateFrameBuffer only guarded against a
null texture, so it attached the invalid handle as a color target. bgfx
framebuffer validation then rejected the attachment ("Invalid texture
attachment") and threw, aborting the entire headless Playground sweep.

Detect this request (non-null texture with an invalid bgfx handle) and:
  - skip attaching the invalid color handle (depth-only framebuffer),
  - allocate the depth attachment as a readable BGFX_TEXTURE_RT so it can
    be sampled, and
  - alias the framebuffer's depth attachment back into the caller-supplied
    texture (ownsHandle=false; the framebuffer owns the handle) so Babylon
    can sample it (e.g. fluid rendering's depth copy).

This removes the whole-sweep abort and makes the depth/stencil texture
sampleable per Babylon's contract.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings June 5, 2026 18:29
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the NativeEngine framebuffer creation path to correctly handle Babylon’s request for a standalone, sampleable depth/stencil texture by interpreting an incoming non-null Graphics::Texture* with an invalid bgfx handle as a “depth-only framebuffer” request and then aliasing the created depth attachment back onto the provided texture for sampling.

Changes:

  • Detect “standalone depth/stencil texture” requests (non-null texture with invalid bgfx handle) and avoid attaching the invalid color target.
  • Create the depth attachment as readable (BGFX_TEXTURE_RT) for the standalone depth/stencil texture case, while keeping existing write-only behavior for normal render-target depth attachments.
  • Alias the framebuffer’s depth attachment back into the caller-provided Graphics::Texture with ownsHandle = false so Babylon can sample it.

Comment on lines +1881 to +1885
Graphics::FrameBuffer* frameBuffer = new Graphics::FrameBuffer(m_deviceContext, frameBufferHandle, width, height, false, generateDepth, generateStencilBuffer, depthStencilAttachmentIndex);

// For a standalone depth/stencil texture request, alias the framebuffer's readable depth attachment back
// into the caller-supplied texture so Babylon can sample it (e.g. fluid rendering's depth copy). The
// framebuffer owns the handle (its destructor destroys it), so the texture must not own it.
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch - fixed in e661432. CreateFrameBuffer now reports hasDepth = (generateDepth || generateStencilBuffer) so the stencil-without-depth path (which allocates a combined D24S8 attachment) no longer reports HasDepth()==false and won't skip depth clear / Z-writes.

…es depth/stencil

When generateStencilBuffer is requested without generateDepth, CreateFrameBuffer still allocates a combined D24S8 depth/stencil attachment, but the FrameBuffer was constructed with hasDepth=generateDepth (false). That made HasDepth() return false, so Clear/DrawInternal skipped depth clear and Z-writes against a depth buffer that actually exists. Report hasDepth as (generateDepth || generateStencilBuffer).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@bkaradzic-microsoft bkaradzic-microsoft marked this pull request as draft June 5, 2026 19:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants