docs(testing): align validation guide with current frameworks by omribz156 · Pull Request #520 · microsoft/edge-ai

omribz156 · 2026-05-15T15:55:11Z

Pull Request

IMPORTANT: Before submitting, please remove all sensitive data, secrets, tokens, or confidential information. Ensure you've redacted any NDA-covered information, IP addresses, resource names, or security-related details that shouldn't be publicly disclosed.

Description

Updates docs/contributing/testing-validation.md so the testing guide reflects the current repository layout and tooling. The previous guide still referenced Terratest / go test examples, while the repo now uses native Terraform .tftest.hcl tests, Rust crate tests, and Docusaurus docs checks.

Related Issue

Fixes #144

Type of Change

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
Blueprint modification or addition
Component modification or addition
Documentation update
CI/CD pipeline change
Other (please describe):

Implementation Details

Replaced stale Terratest / go test guidance with native terraform test examples.
Pointed Terraform examples at the existing .tftest.hcl layout under src/000-cloud/000-resource-group/terraform/tests/.
Added current Rust crate-level cargo test guidance.
Added Docusaurus Jest / TypeScript docs test commands.

Testing Performed

Validation Steps

Reviewers can validate this documentation-only change with:

npx markdownlint --config .markdownlint.json docs/contributing/testing-validation.md
npx cspell docs/contributing/testing-validation.md --config .cspell.json
git diff --check

Checklist

I have updated the documentation accordingly
I have added tests to cover my changes
All new and existing tests passed
I have run terraform fmt on all Terraform code
I have run terraform validate on all Terraform code
I have run az bicep format on all Bicep code
I have run az bicep build to validate all Bicep code
I have checked for any sensitive data/tokens that should not be committed
Lint checks pass (run applicable linters for changed file types)

Security Review

No credentials, secrets, or tokens are hardcoded or logged
RBAC and identity changes follow least-privilege principles
No new network exposure or public endpoints introduced without justification
Dependency additions or updates have been reviewed for known vulnerabilities
Container image changes use pinned digests or SHA references

Additional Notes

This was prepared with Codex assistance, with the final diff reviewed before opening.

Screenshots (if applicable)

N/A; documentation-only change.

omribz156 · 2026-05-15T17:26:21Z

@microsoft-github-policy-service agree

rezatnoMsirhC

Thank you for cleaning up this guide and replacing the stale Terratest examples with native terraform test guidance.

The PR description does not follow .github/PULL_REQUEST_TEMPLATE.md. Please update it to use the required sections: Description, Related Issue (with Fixes #144), Type of Change, Implementation Details, Testing Performed, Validation Steps, Checklist, and Security Review.

rezatnoMsirhC · 2026-05-15T21:04:10Z

 |:---------------|:------------------------|:------------------------------------------------------|
 | **Terraform**  | native `terraform test` | One `.tftest.hcl` per component with `command = plan` |
-| **Rust**       | `cargo test`            | `#[cfg(test)]` module covering core logic             |
+| **Rust**       | `cargo test`            | Run from each service crate with a local `Cargo.toml` |


The replacement text Run from each service crate with a local Cargo.toml describes how to invoke tests rather than what to write. The "Minimum Requirement" column specifies authoring requirements. The original #[cfg(test)] module covering core logic was clearer in this respect. The invocation guidance belongs in the Rust Testing section added later in this PR.

Fixed in b2b687d: restored the Rust minimum requirement to the authoring requirement (#[cfg(test)] module covering core logic) and left invocation guidance in the Rust Testing section.

rezatnoMsirhC · 2026-05-15T21:04:19Z

-    keyVaultName := terraform.Output(t, terraformOptions, "key_vault_name")
-    assert.NotEmpty(t, keyVaultName)
+  variables {
+    resource_prefix = "edge-ai"


The example hardcodes resource_prefix = "edge-ai" and asserts azurerm_resource_group.new[0].name == "rg-edge-ai-dev-001", but the Test Data Management section directly below this example says to use setup modules for generated prefixes. The actual naming-convention.tftest.hcl uses resource_prefix = run.setup_tests.resource_prefix from a setup module run, not a hardcoded value.

The example should either use the setup module pattern to stay consistent with the actual file and the surrounding guidance, or include a note that it is a simplified illustration that intentionally deviates from the recommended pattern.

Fixed in b2b687d: updated the example to use the setup_tests module and run.setup_tests.resource_prefix, matching the surrounding Test Data Management guidance.

rezatnoMsirhC · 2026-05-15T21:04:27Z

-location = "East US"
-enable_monitoring = true
+# terraform/tests/setup/main.tf
+resource "random_string" "resource_prefix" {


The setup module code example does not match the actual file at src/000-cloud/000-resource-group/terraform/tests/setup/main.tf. Differences:

Field This example Actual file

Resource name random_string.resource_prefix random_string.prefix

Length 8 4

Output value random_string.resource_prefix.result "a${random_string.prefix.id}"

terraform {} block absent present with required_providers

Since the PR points readers directly to this component as the reference implementation, the example should reflect the actual file contents.

Fixed in b2b687d: replaced the setup module snippet with the actual tests/setup/main.tf structure, including the terraform block, random_string.prefix, length 4, and a${random_string.prefix.id} output value.

rezatnoMsirhC · 2026-05-15T21:04:34Z

 - Output normalization across frameworks

-**Reference Implementation:** [blueprints/full-single-node-cluster/tests/](https://github.com/microsoft/edge-ai/tree/main/blueprints/full-single-node-cluster/tests/)
+**Reference Implementation:** [src/000-cloud/000-resource-group/terraform/tests/](https://github.com/microsoft/edge-ai/tree/main/src/000-cloud/000-resource-group/terraform/tests/)


This reference implementation link now points to a component (src/000-cloud/000-resource-group/), but the surrounding section is titled "Blueprint Test Architecture" and describes blueprint test patterns. Pointing readers from a blueprint architecture section to a component test directory is likely to cause confusion. Consider retaining the original blueprint reference, or updating the section heading and introductory text to reflect the component-level scope.

Fixed in b2b687d: retitled this section to Terraform Test Reference and updated the intro/reference label so the component-level test directory is presented as a Terraform reference, not a blueprint reference implementation.

rezatnoMsirhC · 2026-05-15T21:04:42Z

+# Terraform deployment check
+terraform init
+terraform plan -var-file="test.tfvars"
+terraform apply -var-file="test.tfvars" -auto-approve


-auto-approve bypasses the plan review step and immediately provisions real Azure resources. A brief inline comment noting the cost and credential requirements before running this command would help, for example: # requires ARM_* credentials; provisions billable Azure resources.

Fixed in b2b687d: added an inline warning before terraform apply -auto-approve that ARM_* credentials are required and the command provisions billable Azure resources.

omribz156 · 2026-05-15T21:10:46Z

Thanks, that makes sense. I updated the PR description to follow .github/PULL_REQUEST_TEMPLATE.md, including the requested related issue, change type, implementation details, testing, validation, checklist, and security review sections.

No code/docs diff changed in this follow-up; only the PR body was updated.

This follow-up was Codex-assisted, with the final text reviewed before posting.

docs(testing): align validation guide with current frameworks

349362e

omribz156 marked this pull request as ready for review May 15, 2026 15:55

omribz156 requested a review from a team as a code owner May 15, 2026 15:55

rezatnoMsirhC requested changes May 15, 2026

View reviewed changes

docs: address testing guide review feedback

b2b687d

Field	This example	Actual file
Resource name	`random_string.resource_prefix`	`random_string.prefix`
Length	`8`	`4`
Output value	`random_string.resource_prefix.result`	`"a${random_string.prefix.id}"`
`terraform {}` block	absent	present with `required_providers`

Conversation

omribz156 commented May 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request

Description

Related Issue

Type of Change

Implementation Details

Testing Performed

Validation Steps

Checklist

Security Review

Additional Notes

Screenshots (if applicable)

Uh oh!

omribz156 commented May 15, 2026

Uh oh!

rezatnoMsirhC left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

omribz156 commented May 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

omribz156 commented May 15, 2026 •

edited

Loading