bug: gateway service fails to start on Ubuntu 22.04 and TLS dir mismatch

[zongqichen]

Agent Diagnostic

• Loaded debug-openshell-cluster skill
• Read deploy/deb/openshell-gateway.service and crates/openshell-server/src/defaults.rs on main
• Ran systemctl --user show openshell-gateway. ExecStartPre resolved %S/openshell/tls to ~/.config/openshell/tls
• Ran ls ~/.local/state/openshell/tls, which is empty. That's where default_local_tls_dir() looks via openshell_state_dir()
• Checked man 5 systemd.unit on Ubuntu 22.04. %S for user managers is $XDG_CONFIG_HOME on systemd <250, changed to $XDG_STATE_HOME in systemd 250
• Tried a drop-in with explicit --tls-cert, --tls-key, --tls-client-ca pointing at ~/.config/openshell/tls. Gateway started cleanly, so the certs are fine, only the lookup path is wrong

Description

Fresh install on Ubuntu 22.04 fails to start: Error: × --tls-cert is required when TLS is enabled (use --disable-tls to skip)

I expected the gateway to come up cleanly, since ExecStartPre runs generate-certs and default_local_tls_dir() is supposed to auto-detect them.

The function ExecStartPre writes certs to %S/openshell/tls, and the binary looks under $XDG_STATE_HOME/openshell/tls. On systemd 249 (Ubuntu 22.04) %S in a user unit resolves to $XDG_CONFIG_HOME, so the two end up in different places (~/.config/openshell/tls vs ~/.local/state/openshell/tls). systemd 250 changed %S for user managers to $XDG_STATE_HOME, which is probably why this wasn't caught. Anyone on 24.04 or newer Fedora won't see it.

Reproduction Steps

curl -LsSf https://raw.githubusercontent.com/NVIDIA/OpenShell/main/install.sh | sh

Environment

• WSL2 (Ubuntu 22.04.5 LTS)
• systemd: 249.11-0ubuntu3.19
• Docker: 27.3.1
• OpenShell: 0.0.49 (installed via dpkg -i openshell_0.0.49-1_amd64.deb from the GitHub release)

Logs

May 27 16:03:28 systemd[969]: Starting OpenShell Gateway...
May 27 16:03:28 openshell-gateway[570674]: 2026-05-27T14:03:28.627272Z  INFO openshell_server::certgen: PKI files created. dir=***/.config/openshell/tls
May 27 16:03:28 systemd[969]: Started OpenShell Gateway.
May 27 16:03:28 openshell-gateway[570695]: Error:   × --tls-cert is required when TLS is enabled (use --disable-tls to skip)
May 27 16:03:28 systemd[969]: openshell-gateway.service: Main process exited, code=exited, status=1/FAILURE
May 27 16:03:28 systemd[969]: openshell-gateway.service: Failed with result 'exit-code'.
May 27 16:03:33 systemd[969]: openshell-gateway.service: Scheduled restart job, restart counter is at 1.
May 27 16:03:33 systemd[969]: Stopped OpenShell Gateway.

Agent-First Checklist

• I pointed my agent at the repo and had it investigate this issue
• I loaded relevant skills (e.g., debug-openshell-cluster, debug-inference, openshell-cli)
• My agent could not resolve this — the diagnostic above explains why

[TaylorMutch]

📋 triage-agent

Triage Assessment

Classification: bug-confirmed

Summary

The report describes a real package-managed gateway startup bug. The Debian user unit generates local TLS material under %S/openshell/tls, but the gateway auto-detects local TLS under the XDG state directory, so Ubuntu 22.04/systemd 249 can generate certs successfully and still fail before binding with --tls-cert is required when TLS is enabled.

Investigation

The service unit currently runs openshell-gateway generate-certs --output-dir %S/openshell/tls in deploy/deb/openshell-gateway.service. The runtime default resolver in crates/openshell-server/src/defaults.rs uses OPENSHELL_LOCAL_TLS_DIR when set, otherwise openshell_core::paths::openshell_state_dir()?.join("tls"); openshell_state_dir() resolves through XDG_STATE_HOME or ~/.local/state in crates/openshell-core/src/paths.rs. When that state-dir bundle is absent, crates/openshell-server/src/cli.rs leaves tls_cert unset and emits the reported --tls-cert is required when TLS is enabled error.

I also reproduced the failure locally by generating certs into a simulated config dir and starting the gateway with a separate simulated state dir; the gateway failed with the same missing --tls-cert error. The reporter's distro impact is directionally correct: Ubuntu 22.04 ships systemd 249, where the %S user-manager path differs from newer systems such as Ubuntu 24.04/systemd 255.

Recommendation

Route this as a confirmed gateway packaging compatibility bug. The fix should make the Debian user service and gateway runtime defaults use the same explicit local TLS directory without relying on version-dependent %S behavior. This does not need a spike; it is ready for an implementation plan through build-from-issue after human review.

[TaylorMutch]

🏗️ build-plan

Implementation Plan

Issue type: fix
Complexity: Low
Confidence: High — clear path

Summary

Fix the Debian package user service so local certificate generation writes to the same XDG state-backed TLS directory that the gateway auto-detects at runtime. Avoid relying on systemd %S semantics, which vary across systemd versions for user managers.

Scope

• deploy/deb/openshell-gateway.service: change ExecStartPre to generate certs under an explicit home-relative state path instead of %S/openshell/tls.
• deploy/man/openshell-gateway.8.md: confirm or update the documented generated TLS path if needed.
• deploy/rpm/CONFIGURATION.md or gateway docs: update only if the Debian service behavior description needs alignment.

Implementation Steps

1. Update the Debian user service ExecStartPre path to a stable ~/.local/state/openshell/tls equivalent supported by systemd user units.
2. Add a package/service regression check that asserts the service does not use %S/openshell/tls and points cert generation at the runtime default location.
3. Review docs that mention package-generated TLS state and update any stale Debian/user-service wording.
4. Run focused tests, then mise run pre-commit.

Test Plan

• Unit tests: Add or update a lightweight packaging/service test if the repo already has deploy file validation; otherwise add the narrowest Rust or script-level test that verifies the service file uses the expected state path.
• Integration tests: N/A — the bug is a static package unit/runtime default mismatch.
• E2E tests: N/A — no e2e files or sandbox infrastructure behavior should change.

Risks & Open Questions

• The service file should use a systemd specifier that is stable across affected versions. %h/.local/state/openshell/tls is the likely target because %h is stable for user units and matches the gateway default when XDG_STATE_HOME is unset. If users set custom XDG_STATE_HOME, the package unit still cannot safely use shell expansion; supporting custom state homes may require an environment override instead of static unit expansion.
• LSM compatibility: no special SELinux/AppArmor concern. This change does not touch process identity, /proc access, binary execution, or inter-process visibility.

Documentation Impact

• Keep package docs/man page aligned with the generated TLS location. No published docs update expected unless user-facing installation docs explicitly describe the Debian unit path.

---

Revision 1 — initial plan