fix(upstream): apply authentication through shared transport

[andrew]

Summary

• apply configured upstream authentication to metadata and artifact requests through one SSRF-safe transport
• discover OCI Bearer challenges and cache scoped registry tokens until shortly before expiry
• match authentication scopes by parsed URL components and path boundaries, preventing lookalike-host credential leaks
• preserve explicit request credentials and apply configured credentials to token service requests

Why

Several Proxy request paths use the raw HTTP client instead of Fetcher, so upstream.auth was not applied consistently. This centralizes authentication below both paths and provides challenge-based OCI authentication without hard-coded Docker Hub token endpoints.

This is PR 1 of 2. The OCI offline-cache fix is stacked on this branch.

Validation

• go test ./... -count=1
• go test -race ./... -count=1
• go tool golangci-lint run ./...

[Copilot]

Pull request overview

This PR centralizes upstream authentication by introducing an authentication-aware HTTP transport that is shared across both metadata and artifact-fetch paths, and adds OCI Bearer-challenge discovery + scoped token caching to avoid hard-coded token endpoints and prevent credential leakage to lookalike hosts.

Changes:

• Wire a shared SSRF-safe, auth-aware transport into both the proxy metadata HTTP client and the artifact fetcher.
• Add internal/httpclient.Transport to apply configured auth headers and handle OCI WWW-Authenticate: Bearer … challenges with token caching.
• Tighten upstream.auth matching to compare parsed URL components (scheme/host/effective port/path boundary), and update docs/examples accordingly.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
internal/server/server.go	Builds shared SSRF-safe client + auth transport; uses it for metadata and artifacts consistently.
internal/httpclient/transport.go	New auth-aware RoundTripper with OCI Bearer challenge discovery and token caching.
internal/httpclient/transport_test.go	Adds unit tests for challenge following, token caching, and configured/explicit auth behavior.
internal/config/config.go	Changes upstream auth matching from raw prefix to parsed URL component + path-boundary matching.
internal/config/config_test.go	Adds coverage for host/scheme/port/path-boundary matching behavior.
docs/configuration.md	Updates auth documentation to reflect URL-scope matching and OCI challenge-based auth.
docs/architecture.md	Notes shared auth transport + OCI token discovery/caching in system architecture.
config.example.yaml	Updates comments to reflect the new URL-scope auth matching rules.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[mxey]

The PR description says:

This centralizes authentication below both paths and provides challenge-based OCI authentication without hard-coded Docker Hub token endpoints.

but this is still in container.go:

	dockerHubAuth      = "https://auth.docker.io"