fix: detect and route around corrupted CLIP-L final layer on SDXL

[Rojikku]

TL;DR no cat (white image) if SDXL && clip_skip = 1
Fix: clip_skip min 2 when NaF (Not A Feline) || Made cat not decat

I will not pretend to understand your project, I came from like 2 projects downstream trying to figure out a config issue, turned out to be a bug here. I am confident I understand this fix, but can't be certain of edge cases.
Edit: Fixed for edge case 1 (Which was a lie, but there might be others where it's true)

_{That project defaulted to clip_skip = 1 which is a "harmless default"}

Figured it was easier to make the AI fix it than make a bug report TBH.

• I reproduce the bug on your latest release
• I have tested this fix
• I am reasonably confident the code is not slop
• I have read and confirmed this PR follows the contribution guidelines.

Claude's explanation

Problem

Passing an explicit clip_skip of 1 (or any value below 2) on an SDXL model produces a blank white image — generation completes normally (all steps, VAE decode, HTTP 200), no errors, just solid white output.

Root cause

This is not a precision, overflow, or backend issue — it's corrupted data in the checkpoint file itself.

Verified by reading the raw safetensors bytes directly, bypassing this codebase entirely: in two independent, unrelated SDXL checkpoints (waiIllustriousSDXL_v170 and cyberrealisticXL_v100), every weight matrix in CLIP-L's final transformer layer (q_proj, k_proj, v_proj, out_proj, mlp.fc1, mlp.fc2) is 100% NaN, on disk, in both files. The layer immediately before it (a known-good control) is completely clean. Biases and layer-norm parameters in that same "bad" layer are untouched — only the big weight matrices are NaN.

This tracks with how SDXL was trained: its UNet was only ever conditioned on CLIP-L's penultimate layer (which is why the existing clip_skip<=0 default already resolves to 2 for SDXL). That final layer is architecturally unused, and the common conversion/merge pipeline nearly every distributed SDXL checkpoint shares appears to have never populated it with valid data. CLIP-G's own final layer was checked in both files and is completely valid — this defect is specific to CLIP-L.

Requesting clip_skip=1 reaches that never-populated layer, producing NaN conditioning that propagates through the UNet and VAE decode; the final pixel clamp turns the resulting garbage into flat white.

Why not just clamp clip_skip unconditionally

An earlier version of this PR did exactly that, and was correctly flagged in review: some SDXL derivatives (e.g. cyberrealistic-xl) are documented to want clip_skip=1, and a blanket clamp would silently override that for anyone using a checkpoint whose CLIP-L isn't corrupted (e.g. via a valid standalone --clip-l override).

Fix

Detect the corruption from the actual computed output instead of assuming every checkpoint is affected: if clip_skip<2 is requested on SDXL and the resulting CLIP-L encoder output contains NaN, warn and recompute with clip_skip=2. This costs one extra, CLIP-L-only compute — only in the narrow case of explicitly requesting the affected range on an affected checkpoint. A checkpoint with valid CLIP-L weights (however it's supplied) is never touched by this path.

Verification

Built locally with -DSD_HIPBLAS=ON -DAMDGPU_TARGETS=gfx1100. Tested against both real checkpoints:

• waiIllustriousSDXL_v170 at clip_skip=1: warns, clamps, produces a normal image (was solid white before the fix)
• cyberrealisticXL_v100 at clip_skip=1: same
• Both checkpoints at default (unset) clip_skip: unchanged, no warning, no extra compute (guard is skipped once clip_skip>=2)
• dreamshaper_8 (SD1.5) at clip_skip=1: completely untouched (guard is SDXL-only), unchanged output

Verification

Hardware:

CPU: AMD Ryzen 9 5950X (32) @ 5.09 GHz
GPU: AMD Radeon RX 7900 XTX [Discrete]
Memory: 15.99 GiB / 62.70 GiB (25%)

Downloaded sd-master-97d2990-bin-Linux-Ubuntu-24.04-x86_64-rocm-7.14.0.zip, verified problem reproduces.
Compiled fix, verified fix doesn't reproduce.

Method:

# baseline — no clip-skip
./sd-cli -m /home/rojikku/Cache/KoboldCPP/waiIllustriousSDXL_v170.safetensors \
  -p "cat" --sampling-method euler_a --steps 20 --cfg-scale 5 -W 1024 -H 1024 \
  -o out_cli_baseline.png -v

# with clip-skip 1
./sd-cli -m /home/rojikku/Cache/KoboldCPP/waiIllustriousSDXL_v170.safetensors \
  -p "cat" --sampling-method euler_a --steps 20 --cfg-scale 5 -W 1024 -H 1024 \
  --clip-skip 1 -o out_cli_clipskip1.png -v

I believe this fits all standards, please let me know if I need to make corrections or add context. Edit as you please.

[Rojikku]

The fix has been adjusted. Instead of all SDXL, it specifically checks if CLIP-L produces bad output, then changes clip_skip=2 with a log if it does.