Skip to content

Add MicrobotsLogAnalyzer custom ADO task support#141

Closed
MadhurAggarwal wants to merge 10 commits into
microsoft:mainfrom
MadhurAggarwal:madagg/log-analyzer-ado-task
Closed

Add MicrobotsLogAnalyzer custom ADO task support#141
MadhurAggarwal wants to merge 10 commits into
microsoft:mainfrom
MadhurAggarwal:madagg/log-analyzer-ado-task

Conversation

@MadhurAggarwal
Copy link
Copy Markdown
Member

@MadhurAggarwal MadhurAggarwal commented May 7, 2026

Adds a custom Azure DevOps task, MicrobotsLogAnalyzer@0, for running Microbots LogAnalysisBot against log files using Azure OpenAI models, with authentication through an Azure Resource Manager Service Connection.

Overview

  • Added the MicrobotsLogAnalyzerTask Azure Pipelines task.
  • Added Azure DevOps extension manifest and npm dependencies required by the task for connecting to Service Connection.
  • Added documentation for Packaging, Publishing, and Using the task in a pipeline.
  • Added a sample Azure Pipelines YAML file.

Change Details

  • .gitignore: Ignores generated node_modules folders used while packaging the ADO task.
  • README.md: Added a brief guide for the new MicrobotsLogAnalyzerTask.
  • azure-pipelines/vss-extension.json: Defines the ADO extension that contributes the custom pipeline task. This file is needed for publishing the task.
  • azure-pipelines/MicrobotsLogAnalyzerTask/task.json: Defines the MicrobotsLogAnalyzer@0 task metadata, inputs, Node 20 handler, and task restrictions (restrictions ensure that LLM Outputs don't setup unwanted ENVs in ADO Pipeline, for example printing ##vso[task.setvariable as part of Log RCA)
  • azure-pipelines/MicrobotsLogAnalyzerTask/index.js: Implements Main Workflow: validates inputs, logs into Azure Service Connection, prepares Python venv, runs LogAnalysisBot, and prints the analysis.
  • azure-pipelines/MicrobotsLogAnalyzerTask/package.json: Declares the Node dependencies required by the task.
  • azure-pipelines/MicrobotsLogAnalyzerTask/package-lock.json: Locks task Node dependency versions for reproducible packaging.
  • docs/azure-pipelines-log-analyzer.md: Documents prerequisites, packaging, publishing, usage, inputs, and task behavior.
  • docs/examples/azure-pipelines/microbots-log-analyzer.yml: Provides a sample ADO YAML for using MicrobotsLogAnalyzer@0.
  • mkdocs.yml: Adds the new Azure Pipelines Log Analyzer guide to the docs navigation.

Added 12 Unit tests:
image

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented May 7, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 64.99%. Comparing base (d41a0a9) to head (8f3babb).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #141      +/-   ##
==========================================
+ Coverage   64.95%   64.99%   +0.04%     
==========================================
  Files          34       34              
  Lines        2374     2374              
==========================================
+ Hits         1542     1543       +1     
+ Misses        832      831       -1     
Flag Coverage Δ
integration 34.33% <ø> (-0.17%) ⬇️
ollama_local 32.05% <ø> (-0.17%) ⬇️
slow-browser 26.83% <ø> (ø)
slow-other 38.54% <ø> (ø)
unit 59.18% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.
see 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread azure-pipelines/MicrobotsLogAnalyzerTask/index.js Outdated
Comment thread azure-pipelines/MicrobotsLogAnalyzerTask/index.js
Comment thread azure-pipelines/MicrobotsLogAnalyzerTask/index.js Outdated
Comment thread azure-pipelines/MicrobotsLogAnalyzerTask/index.js Outdated
@KavyaSree2610
Copy link
Copy Markdown
Collaborator

KavyaSree2610 commented May 11, 2026

You shouldn't be running from a fork for the tests to pass

Comment thread azure-pipelines/MicrobotsLogAnalyzerTask/index.js Outdated
Comment thread azure-pipelines/MicrobotsLogAnalyzerTask/index.js
codebasePath: tl.getPathInput("codebasePath", true, true),
logFilePath: input("logFilePath", true),
timeoutSeconds: input("timeoutSeconds", false) || DEFAULT_TIMEOUT_SECONDS,
};
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As we publish to open-source, we should support key-based authentication too. Update the PR if it is trivial otherwise, we can take it in the next iteration.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, this would be a useful addition.
However, key-based auth would require handling multiple Pipeline input-groupings and some changes + testing required for the new auth flow (currently the task uses AzureRM for login). It might be better to first publish & test the current custom task and take up additional LLM auth methods in subsequent iteration.

Comment thread azure-pipelines/MicrobotsLogAnalyzerTask/package-lock.json
Comment thread azure-pipelines/MicrobotsLogAnalyzerTask/index.js Outdated
Copy link
Copy Markdown
Collaborator

@shivashanmugam shivashanmugam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the PR

Comment thread azure-pipelines/MicrobotsLogAnalyzerTask/package-lock.json
Comment thread azure-pipelines/MicrobotsLogAnalyzerTask/index.js Outdated
Comment thread docs/azure-pipelines-log-analyzer.md

## Use It In A Pipeline

See the complete sample pipeline at [docs/examples/azure-pipelines/microbots-log-analyzer.yml](examples/azure-pipelines/microbots-log-analyzer.yml).
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just want to confirm will the reference work correct in blog post

Comment thread docs/examples/azure-pipelines/microbots-log-analyzer.yml
@MadhurAggarwal MadhurAggarwal force-pushed the madagg/log-analyzer-ado-task branch from cce98a7 to b12284e Compare May 18, 2026 06:27
@MadhurAggarwal MadhurAggarwal marked this pull request as draft May 18, 2026 06:28
@MadhurAggarwal MadhurAggarwal force-pushed the madagg/log-analyzer-ado-task branch from b12284e to dc7143a Compare May 18, 2026 06:36
@MadhurAggarwal MadhurAggarwal marked this pull request as ready for review May 18, 2026 11:35
@MadhurAggarwal
Copy link
Copy Markdown
Member Author

Hi, I've resolved the comments

  • Added Python Script as a separate .py file
  • Added max_iterations Pipeline Parameter
  • Added serviceConnection as an alias to azureSubscription (used by other ADO tasks)
  • Updated ENVs to AZURE_OPENAI instead of previous OPEN_AI ENVs as per the latest merged changes.
  • Added Docker as a Prerequisite in Documentation and added explicit Error Logging for missing docker
  • Added 12 unit tests using Mock Service Connections and Mock LLM Outputs

Kindly re-review the PR.
We can proceed for final publishing and ADO testing.

@MadhurAggarwal
Copy link
Copy Markdown
Member Author

Moved branch from Forked repo to Main Repo for running Github Tests.
New PR: #145

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants