Import 507-ai-inference folder from leak-detection-e2e branch on Azure DevOps.#558
Open
TechPreacher wants to merge 6 commits into
Open
Conversation
This commit introduces several improvements across the AI inference service and development experience: * **Refactor(ai-edge-inference-crate):** Implements thread-safe metric and backend statistics updates using `Mutex` and `RwLock` with graceful error handling for poisoned locks, significantly improving service robustness and reliability of reported metrics. * **Feat(deployment):** Updates the `busybox` init container image to include a SHA256 digest, enhancing image immutability and security. Changes the default `RUST_LOG` level to `info` for reduced verbosity in deployed environments. * **Chore(vscode):** Adds a comprehensive set of VS Code tasks for Terraform (validation, linting, docs, testing, planning), Markdown linting, and "Adhoc" tasks for Cargo, Kustomize, and YAML linting, streamlining developer workflows. * **Docs(readme):** Updates the README to include `protoc` as a manual setup prerequisite. * **Fix(scripts):** Quotes variables in `gen-patch.sh` for improved shell script robustness.
📚 Documentation Health ReportGenerated on: 2026-05-22 06:02:21 UTC 📈 Documentation Statistics
🏗️ Three-Tree Architecture Status
🔍 Quality Metrics
This report is automatically generated by the Documentation Automation workflow. |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #558 +/- ##
==========================================
- Coverage 32.41% 32.09% -0.32%
==========================================
Files 40 40
Lines 5902 5960 +58
==========================================
Hits 1913 1913
- Misses 3989 4047 +58
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
📚 Documentation Health ReportGenerated on: 2026-05-22 07:06:04 UTC 📈 Documentation Statistics
🏗️ Three-Tree Architecture Status
🔍 Quality Metrics
This report is automatically generated by the Documentation Automation workflow. |
📚 Documentation Health ReportGenerated on: 2026-05-22 07:09:35 UTC 📈 Documentation Statistics
🏗️ Three-Tree Architecture Status
🔍 Quality Metrics
This report is automatically generated by the Documentation Automation workflow. |
Contributor
|
Thanks for this addition to the repo! I will add a couple small comments but everything mostly looks great |
kgmwang1
reviewed
May 22, 2026
|
|
||
|
|
||
| let models_dir = PathBuf::from( | ||
| std::env::var("MODELS_DIRECTORY").unwrap_or_else(|_| "/models".to_string()), |
Contributor
There was a problem hiding this comment.
in the Dockerfile, the env variable is MODEL_DIRECTORY and in /src/500-application/507-ai-inference/services/ai-edge-inference/src/config.rs, it is MODELS_DIRECTORY
kgmwang1
reviewed
May 22, 2026
| } | ||
|
|
||
| /// Determine output shape from the flat output vector and model metadata | ||
| fn infer_output_shape(&self, output_len: usize, model: &OnnxModel) -> Vec<usize> { |
Contributor
There was a problem hiding this comment.
the shape value returned on line 240 should be the source of truth for the tensor shape. The value derived in this function may be incorrect if labels are missing. Suggesting the value on 240 be propagated and used as the value of shape.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request
Description
This PR selectively merges the validated 507 AI inference imported changes into the current 507 implementation while preserving the hardened runtime and deployment posture already in the branch.
The merge restores the expected service bootstrap and inference path, aligns the chart/runtime model directory contract, updates the ONNX backend implementation and supporting example/docs, and hardens the ACR container build by fixing ONNX Runtime linking and removing dead scaffold from the image.
Related Issue
Relates to the 507 AI inference imported merge work tracked in
.copilot-tracking/changes/2026-05-21/507-ai-inference-imported-merge-changes.md.Type of Change
Implementation Details
charts/gen-patch.shso image and namespace values can be supplied explicitly without changing default behavior.services/ai-edge-inference/src/main.rswhile keeping the current MQTT session orchestration path.services/ai-edge-inference-crate/src/backends/onnx.rsand updatedengine.rsto preserve the required output mapping and safer metadata handling.Dockerfile.acrto use ONNX Runtime system linking correctly and removed the unused/installrootscaffold from the final image path.protoc/protobuf prerequisite needed for manual builds.Testing Performed
Validation performed for this branch:
cargo check --manifest-path src/500-application/507-ai-inference/services/ai-edge-inference-crate/Cargo.tomlcargo test --manifest-path src/500-application/507-ai-inference/services/ai-edge-inference-crate/Cargo.tomlcargo check --manifest-path src/500-application/507-ai-inference/services/ai-edge-inference/Cargo.tomlbash -n src/500-application/507-ai-inference/charts/gen-patch.shkustomize build src/500-application/507-ai-inference/chartsdocker build -f src/500-application/507-ai-inference/services/ai-edge-inference/Dockerfile src/500-application/507-ai-inference/servicesdocker build -f src/500-application/507-ai-inference/services/ai-edge-inference/Dockerfile.acr src/500-application/507-ai-inference/servicesValidation Steps
src/500-application/507-ai-inference/services/ai-edge-inference-crate/.kustomize build src/500-application/507-ai-inference/charts.src/500-application/507-ai-inference/services.Checklist
terraform fmton all Terraform codeterraform validateon all Terraform codeaz bicep formaton all Bicep codeaz bicep buildto validate all Bicep codeSecurity Review
Additional Notes
The branch contains an imported merge of validated 507 AI inference changes rather than a single isolated fix, so the PR spans runtime, chart, crate, and container build updates.
Screenshots (if applicable)
None.