VS Code + Copilot capability test (for locked-down environments)

A small, read-only kit you can hand to a user in a secured/locked-down environment. It answers:

1. How locked down is their VS Code / machine? (policies, ConstrainedLanguage, proxy, telemetry, extension governance)
2. What AI models can they actually use in Copilot Chat? (the model picker + a live reply test)
3. Can the models / their machine reach the internet? (endpoint egress, TLS inspection, and inference-time web access)

It makes no changes, installs nothing, and collects no secrets (proxy/auth values are masked).

---

How the user runs it — 3 steps

Step 1 — Run the automated probe

Pick the probe for your OS. Both are read-only, take the same options, and write the same report format into results/.

Windows — open the folder in VS Code, open a PowerShell terminal (Ctrl+`), and paste:

powershell -ExecutionPolicy Bypass -File .\Test-VSCodeCapabilities.ps1

macOS / Linux — open a terminal and run the native shell probe (no PowerShell install required):

chmod +x ./test-vscode-capabilities.sh   # first time only
./test-vscode-capabilities.sh

The PowerShell script also runs on macOS/Linux if PowerShell 7 (pwsh) is installed (pwsh -File ./Test-VSCodeCapabilities.ps1), but the .sh needs nothing beyond the built-in bash, curl, and openssl, so it's the better fit for locked-down Macs.

It prints a color summary and writes two files into results/:

• capability-report_<machine>_<time>.md ← human-readable
• capability-report_<machine>_<time>.json ← machine-readable

If script execution is blocked (e.g. running scripts is disabled on this system), the environment is enforcing an execution policy — that itself is a useful finding. Try this instead:

Get-Content .\Test-VSCodeCapabilities.ps1 -Raw | Invoke-Expression

If both are blocked, just do Step 2 by hand and note "scripts blocked by policy."

Step 2 — Do the manual UI checks

Open MANUAL-CHECKS.md and fill in the blanks. These are the things only visible in the UI: Copilot sign-in, the list of models in the picker, a per-model reply test, and whether the model has inference-time internet (use the paste-in prompt in copilot-chat-selftest.md).

Step 3 — Send results back

Send back both:

• the filled-in MANUAL-CHECKS.md, and
• the .json (or .md) file from the results/ folder.

---

What's in this folder

File	Purpose
README.md	This guide.
Test-VSCodeCapabilities.ps1	Automated probe for Windows (lockdown + extensions + network egress).
test-vscode-capabilities.sh	Automated probe for macOS / Linux (same checks, native shell — no PowerShell needed).
endpoints.json	The list of endpoints to test. Edit to add the customer's own Azure OpenAI / Foundry resource. Read by both probes.
MANUAL-CHECKS.md	Fill-in checklist for the UI-only items (models, sign-in, web access).
copilot-chat-selftest.md	A prompt to paste into Copilot Chat to test inference-time web access.
results/	Created on first run; holds the generated reports.

---

Options

# Skip all network tests (lockdown + extensions only)
powershell -ExecutionPolicy Bypass -File .\Test-VSCodeCapabilities.ps1 -SkipNetwork

# Write reports somewhere else / change per-endpoint timeout
.\Test-VSCodeCapabilities.ps1 -OutputPath C:\Temp\caps -TimeoutSeconds 5

# macOS / Linux equivalents
./test-vscode-capabilities.sh --skip-network
./test-vscode-capabilities.sh --output-path ~/caps --timeout 5

Mixed model sources (GitHub-hosted + bring-your-own)

These environments commonly use both GitHub-hosted Copilot models and bring-your-own models (Azure OpenAI / Azure AI Foundry, sometimes OpenAI/Anthropic/Gemini). The probe covers both:

• GitHub-hosted path — the copilot endpoints (required) are tested automatically.
• Bring-your-own path — Azure-native (ai.azure.com, models.inference.ai.azure.com, ...) and common third-party providers (api.openai.com, api.anthropic.com, ...) are tested as optional.

Edit endpoints.json to point at the customer's own resource(s): replace the YOUR-RESOURCE.openai.azure.com / YOUR-PROJECT.services.ai.azure.com rows (or add more). The script skips any URL still containing YOUR-RESOURCE or EDIT-ME. In Copilot Chat, BYOM models are added via "Manage Models" — record them in MANUAL-CHECKS.md section B.

---

How to read the results (guidance you then give the user)

Finding in the report	What it means / what to change
LanguageMode=ConstrainedLanguage	WDAC/AppLocker lockdown. Lean on the UI/manual flow; scripted automation may be limited.
VS Code GPO policies present	Org overrides user settings — confirm what's enforced before advising config changes.
Marketplace endpoint FAIL	Extensions must be side-loaded as approved VSIX or mirrored internally.
Required Copilot endpoint FAIL	Copilot Chat won't work until api.githubcopilot.com etc. are allow-listed.
TLS inspection = possible	Corporate root CA in the path — trust it for VS Code/Node (NODE_EXTRA_CA_CERTS) and set http.proxyStrictSSL.
Model listed but reply test errors	Access blocked at policy/network layer, not "missing model."
WEB_BLOCKED in the self-test	No inference-time web — design for offline/grounded context (RAG, provided files), or add a brokered egress/grounding path.
All Foundry/model endpoints FAIL	Bring-your-own-model needs egress or a gateway/APIM broker.

---

Safety / privacy

• Read-only. No settings, files, or extensions are changed; nothing is installed.
• No secrets. Only an allowlist of setting keys is read; proxy/authorization values are masked (****).
• No data leaves the machine except ordinary HTTPS HEAD/GET requests to the endpoints listed in endpoints.json (to test whether they're reachable). You can read and edit that list first.

Author: UngerAI