Kernel native causal_conv1d functions

[Cyrilvallez]

What does this PR do?

As per the title. This makes causal_conv1d_fn and causal_conv1d_update kernels native! Needs to be extended to all models using them, but just for qwen to unblock me for now

cc @vasqu

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[vasqu]

Ok I remember why this is a bit more complicated, see especially my last comment. The first is about functionality

We might need a wrapper around use_kernel_func_from_hub along use_kernel_func_from_hub_with_fallback that allows to use the original package as another fallback. But we would need

1. Mandatory kernels install - which we will do
2. This new wrapper that forces the exchange when the og is installed
3. ^ unsure where to support because the registration path for kernels is a bit weird for functions so it probably needs to invoke a forced kernelize only for these kind of functions

[vasqu]

We need

Suggested change

	@use_kernelized_func([causal_conv1d_update, causal_conv1d_fn])
	class Qwen3NextGatedDeltaNet(nn.Module):

here as it otherwise won't use them / kernelize it properly. It's always a bit weird with functions and kernels

[vasqu]

Ok I think I remember why this is a bit more complicated: With this, we will only support torch only and using kernels but not the original package anymore. So users that installed causal-conv1d would no longer use the fast path - imo it's a mistake from the past that we not directly switched to kernels

For this we have (see mamba2)

transformers/src/transformers/models/mamba2/modeling_mamba2.py

Lines 187 to 202 in 35bb43e

	global causal_conv1d_update, causal_conv1d_fn
	causal_conv1d = lazy_load_kernel("causal-conv1d")
	causal_conv1d_update = getattr(causal_conv1d, "causal_conv1d_update", None)
	causal_conv1d_fn = getattr(causal_conv1d, "causal_conv1d_fn", None)
	global selective_state_update, mamba_chunk_scan_combined, mamba_split_conv1d_scan_combined
	mamba_ssm = lazy_load_kernel("mamba-ssm")
	selective_state_update = resolve_internal_import(
	mamba_ssm, chained_path="ops.triton.selective_state_update.selective_state_update"
	)
	mamba_chunk_scan_combined = resolve_internal_import(
	mamba_ssm, chained_path="ops.triton.ssd_combined.mamba_chunk_scan_combined"
	)
	mamba_split_conv1d_scan_combined = resolve_internal_import(
	mamba_ssm, chained_path="ops.triton.ssd_combined.mamba_split_conv1d_scan_combined"
	)

So lazy-loaded kernels would use fallbacks to their original package (resolve_internal_import gives the past in the original package / full path if not exposed on top init)

[vasqu]

Ah ok I see the approach now, yea definitely will be working regardless of the next decorator then 🤔

My only worry is that some functions are not exposed on top init, e.g. see the mamba resolve internal path. Which means it won't properly work in those cases then. It's super annoying but we have to pass it imo

[vasqu]

Other question: if we exchange here, wouldnt it also change the signature and kernelize wouldn't work anymore. Would be interested to see what happens on the interaction of having both kernels installed and the og package

[vasqu]

It's getting complicated, we may revisit later but the interaction between og, kernels, and torch is akward

[github-actions]

[For maintainers] Suggested jobs to run (before merge)

run-slow: qwen3_5, qwen3_5_moe, qwen3_next

[vasqu]

Would only change this to use more kwargs but otherwise should stay the same

[github-actions]

---

CI recap

Dashboard: View test results in Grafana
Latest run: 29275579323:1
Result: success | Jobs: 2 | Tests: 24 | Failures: 0 | Duration: 2m 23s