Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -94,9 +94,9 @@ The “Research reference” column records where the catalog field was derived

### D5 — Schema + processor in this OpenSpec; agent-only processor rules

**Choice**: Deliver types, examples, schema tests, **and** catalog processor validation for agent-specific fields. Processor does **not** re-validate core entity fields (`owner`, `lifecycle`, etc.) beyond existing catalog behavior.
**Choice**: Deliver types, examples, schema tests, **and** catalog processor validation for agent-specific fields. The agent processor (`AiResourceAgentProcessor`) lives in `catalog-backend-module-ai-resource-agent`, not in `AIResourceExtensionsProcessor`. The extensions processor remains scope/OCI only. Processor does **not** re-validate core entity fields (`owner`, `lifecycle`, etc.) beyond existing catalog behavior.

**Rationale**: Epic cohesion (15867 + 15868 share one field set). Keep processor focused on agent fields.
**Rationale**: Epic cohesion (15867 + 15868 share one field set). Keep processor focused on agent fields. Agent validation belongs with the agent packages, matching the packaging approach from #4128.

### D6 — Dual-track documentation (rhdh-plugins + upstream)

Expand Down Expand Up @@ -132,14 +132,14 @@ The “Research reference” column records where the catalog field was derived
- Per-type validators / guards: `skillAiResourceEntityV1alpha1Validator`, `ruleAiResourceEntityV1alpha1Validator`, `isSkillAiResourceEntity`, `isRuleAiResourceEntity`
- Kind registration via `aiResourceEntityModel` / `catalogModuleAiResourceEntityModel`

Agent should follow that pattern: add an `AgentAiResourceEntity…` (name TBD) member of the AiResource union (or an RHDH-local extension layer that mirrors it until upstream accepts agent), with a `KindValidator` + type guard keyed on `spec.type: 'agent'`.
Agent should follow that pattern: add an `AgentAiResourceEntity…` (name TBD) member of the AiResource union (or an RHDH-local extension layer that mirrors it until upstream accepts agent), with a `KindValidator` + type guard keyed on `spec.type: agent`.

**Secondary reference — MCP server API discriminated extension** (same catalog-model alpha surface):

- `McpServerApiEntity` with `spec.type: 'mcp-server'`, `mcpServerApiEntityValidator`, `isMcpServerApiEntity`, `mcpServerApiEntityModel`
- `McpServerApiEntity` with `spec.type: mcp-server`, `mcpServerApiEntityValidator`, `isMcpServerApiEntity`, `mcpServerApiEntityModel`
- Shows how Backstage extends an existing kind with a typed `spec.type` branch (useful precedent for dual-track / upstream PR work)

**Local RHDH extension precedent**: this workspace’s `AIResourceExtensionsProcessor` for `spec.scope` / OCI checks—agent **field** validation (RHIDP-15868) should extend that processor path for agent-specific rules, while the **typed schema** itself follows the catalog-model validator pattern above.
**Agent packaging**: Agent schema, `KindValidator`, type guard, and `CatalogModelLayer` live in `catalog-model-ai-resource-agent`. Agent-specific field validation (RHIDP-15868) lives in `catalog-backend-module-ai-resource-agent` as `AiResourceAgentProcessor`, registered alongside the model source in the same backend module. The `AIResourceExtensionsProcessor` in `catalog-backend-module-ai-resource-extensions` remains scope/OCI only and does not contain agent-specific rules. Kind spelling is `AiResource` (matching #4128 / the agent model packages).

**Rationale**: Without these pointers, implementers (human or coding agent) will invent ad-hoc types that diverge from skill/rule and force manual rework. Gabe’s review feedback on this PR.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Platform engineers need a first-class way to register AI agents in the Software
- Add a typed agent schema for `kind: AiResource` with `spec.type: agent` (TypeScript types / schema-style validation; no OpenAI Agents SDK package dependency).
- Encode the decided field mapping (RHIDP-15866) in OpenSpec design; only agent-specific required field is non-empty `spec.instructions`.
- Add example `catalog-info.yaml` and/or test fixtures covering a representative agent (required + optional fields).
- Extend catalog processor validation for agent entities (RHIDP-15868): reject missing/invalid agent-specific fields with actionable errors.
- Add agent-specific catalog processor validation in the agent backend module (`catalog-backend-module-ai-resource-agent`) via `AiResourceAgentProcessor` (RHIDP-15868): reject missing/invalid agent-specific fields with actionable errors. `AIResourceExtensionsProcessor` remains scope/OCI only.
- Update in-repo OpenSpec/design docs for `AiResource` + agent ownership under RHDHPLAN-1507; remove “pending 1113” language for the agent type where it appears.
- Add unit/schema and processor tests for accept/reject paths.
- Align naming with upstream skill/rule style: singular `spec.type: agent` (not `agents`); correct sibling discovery examples accordingly.
Expand Down Expand Up @@ -49,7 +49,7 @@ _(none promoted under `openspec/specs/` yet. Sibling change-local discovery exam
## Impact

- **Schema / types**: New or extended TypeScript for agent-shaped `AiResource` in this workspace.
- **Catalog processor**: Agent-specific validation in the AiResource extensions processor path.
- **Catalog processor**: Agent-specific validation in the agent backend module (`AiResourceAgentProcessor`), not the extensions processor.
- **Examples / fixtures**: New catalog YAML under `examples/` (and/or test fixtures).
- **Tests**: Schema/unit and processor accept/reject coverage.
- **Docs / OpenSpec**: Dual-track (rhdh-plugins + upstream) and singular type discriminator docs.
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,11 @@

## 5. Catalog processor validation (RHIDP-15868)

- [ ] 5.1 Extend the AiResource extensions processor to validate agent-specific fields when `spec.type: agent`
- [ ] 5.1 Add `AiResourceAgentProcessor` in `catalog-backend-module-ai-resource-agent` to validate agent-specific fields when `spec.type: agent`; do **not** extend `AIResourceExtensionsProcessor` (which remains scope/OCI only)
- [ ] 5.2 Reject missing/empty `spec.instructions` and wrong optional agent field shapes with actionable errors
- [ ] 5.3 Do not enforce entity-ref format on `handoffs` / `tools`; do not add new owner/lifecycle processor rules
- [ ] 5.4 Add processor tests for accept and reject paths; ensure non-agent AiResources are unaffected
- [ ] 5.5 Register `AiResourceAgentProcessor` in the agent backend module alongside the model source

## 6. Docs and OpenSpec DoD

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,10 +32,12 @@
"dependencies": {
"@backstage/backend-plugin-api": "^1.9.2",
"@backstage/catalog-model": "^1.9.0",
"@backstage/plugin-catalog-common": "^1.1.10",
"@backstage/plugin-catalog-node": "^2.2.2",
"@red-hat-developer-hub/backstage-plugin-catalog-model-ai-resource-agent": "workspace:^"
},
"devDependencies": {
"@backstage/backend-test-utils": "^1.11.4",
"@backstage/cli": "^0.36.3"
},
"files": [
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,22 @@

```ts
import { BackendFeature } from '@backstage/backend-plugin-api';
import { CatalogProcessor } from '@backstage/plugin-catalog-node';
import { CatalogProcessorEmit } from '@backstage/plugin-catalog-node';
import { Entity } from '@backstage/catalog-model';
import { LocationSpec } from '@backstage/plugin-catalog-common';

// @public
export class AiResourceAgentProcessor implements CatalogProcessor {
// (undocumented)
getProcessorName(): string;
// (undocumented)
preProcessEntity(
entity: Entity,
_location: LocationSpec,
_emit: CatalogProcessorEmit,
): Promise<Entity>;
}

// @public
const catalogModuleAiResourceAgent: BackendFeature;
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,236 @@
/*
* Copyright Red Hat, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

import { Entity } from '@backstage/catalog-model';
import { AiResourceAgentProcessor } from './AiResourceAgentProcessor';

function makeAiResource(
spec: Entity['spec'] = {},
annotations?: Record<string, string>,
): Entity {
return {
apiVersion: 'backstage.io/v1alpha1',
kind: 'AiResource',
metadata: {
name: 'test-resource',
...(annotations ? { annotations } : {}),
},
spec,
};
}

describe('AiResourceAgentProcessor', () => {
let processor: AiResourceAgentProcessor;
const location = { type: 'url', target: 'https://example.com' };
const emit = jest.fn();

beforeEach(() => {
processor = new AiResourceAgentProcessor();
emit.mockClear();
});

it('should return processor name', () => {
expect(processor.getProcessorName()).toBe('AiResourceAgentProcessor');
});

describe('agent validation (spec.type: agent)', () => {
it('should accept a valid agent entity with required fields', async () => {
const entity = makeAiResource({
type: 'agent',
lifecycle: 'production',
owner: 'ai-platform-team',
instructions: 'You are a test agent.',
});

const result = await processor.preProcessEntity(entity, location, emit);

expect(result).toEqual(entity);
});

it('should accept a valid agent with all optional fields', async () => {
const entity = makeAiResource({
type: 'agent',
lifecycle: 'production',
owner: 'ai-platform-team',
instructions: 'You are a fully configured agent.',
handoffDescription: 'Handles everything.',
model: 'gpt-4o',
handoffs: ['agent-a', 'agent-b'],
tools: ['tool-x', 'tool-y'],
toolUseBehavior: 'run_llm_again',
resetToolChoice: true,
modelSettings: { temperature: 0.5 },
outputSchema: { type: 'object' },
});

const result = await processor.preProcessEntity(entity, location, emit);

expect(result).toEqual(entity);
});

it('should accept opaque handoffs and tools strings', async () => {
const entity = makeAiResource({
type: 'agent',
instructions: 'Agent with opaque refs.',
handoffs: ['some-arbitrary-string', 'another-ref'],
tools: ['my-custom-tool'],
});

const result = await processor.preProcessEntity(entity, location, emit);

expect(result).toEqual(entity);
});

it('should reject agent with missing instructions', async () => {
const entity = makeAiResource({
type: 'agent',
lifecycle: 'production',
owner: 'team',
});

await expect(
processor.preProcessEntity(entity, location, emit),
).rejects.toThrow('spec.instructions');
});

it('should reject agent with empty instructions', async () => {
const entity = makeAiResource({
type: 'agent',
instructions: '',
});

await expect(
processor.preProcessEntity(entity, location, emit),
).rejects.toThrow('spec.instructions');
});

it('should reject agent with wrong-type instructions', async () => {
const entity = makeAiResource({
type: 'agent',
instructions: 42,
});

await expect(
processor.preProcessEntity(entity, location, emit),
).rejects.toThrow('spec.instructions');
});

it('should reject agent with handoffs as non-array', async () => {
const entity = makeAiResource({
type: 'agent',
instructions: 'Valid instructions.',
handoffs: 'not-an-array',
});

await expect(
processor.preProcessEntity(entity, location, emit),
).rejects.toThrow('spec.handoffs');
});

it('should reject agent with resetToolChoice as non-boolean', async () => {
const entity = makeAiResource({
type: 'agent',
instructions: 'Valid instructions.',
resetToolChoice: 'yes',
});

await expect(
processor.preProcessEntity(entity, location, emit),
).rejects.toThrow('spec.resetToolChoice');
});

it('should not apply agent instructions rule to skill entities', async () => {
const entity = makeAiResource({
type: 'skill',
lifecycle: 'production',
owner: 'team',
});

const result = await processor.preProcessEntity(entity, location, emit);

expect(result).toEqual(entity);
});

it('should not apply agent instructions rule to entities without spec.type', async () => {
const entity = makeAiResource({
lifecycle: 'production',
owner: 'team',
});

const result = await processor.preProcessEntity(entity, location, emit);

expect(result).toEqual(entity);
});

it('should report multiple agent errors together', async () => {
const entity = makeAiResource({
type: 'agent',
handoffs: 'not-an-array',
});

const error = await processor
.preProcessEntity(entity, location, emit)
.catch((e: Error) => e);

expect(error).toBeInstanceOf(Error);
const message = (error as Error).message;
expect(message).toContain('spec.instructions');
expect(message).toContain('spec.handoffs');
});

it('should not expose internal class names in agent errors', async () => {
const entity = makeAiResource({
type: 'agent',
});

const error = await processor
.preProcessEntity(entity, location, emit)
.catch((e: Error) => e);

expect(error).toBeInstanceOf(Error);
expect((error as Error).message).not.toMatch(/AiResourceAgentProcessor/);
expect((error as Error).message).not.toMatch(/at\s+\w+\.\w+\s+\(/);
});
});

describe('non-AiResource entities', () => {
it('should pass through Component entities unchanged', async () => {
const entity: Entity = {
apiVersion: 'backstage.io/v1alpha1',
kind: 'Component',
metadata: { name: 'my-component' },
spec: { type: 'service', lifecycle: 'production', owner: 'team-a' },
};

const result = await processor.preProcessEntity(entity, location, emit);

expect(result).toEqual(entity);
});

it('should not validate agent fields on non-AiResource kinds', async () => {
const entity: Entity = {
apiVersion: 'backstage.io/v1alpha1',
kind: 'Resource',
metadata: { name: 'my-resource' },
spec: { type: 'agent', owner: 'team-a' },
};

const result = await processor.preProcessEntity(entity, location, emit);

expect(result).toEqual(entity);
});
});
});
Loading
Loading