Skip to content

Port/volume status not refreshed after server attachment due to SSA field owner race #842

Description

@mandre

Problem description

After a port is attached to a server via Nova's interface-attach API, the port controller may never observe the Neutron status transition from DOWN to ACTIVE. The same race affects volumes (availablein-use). This causes test assertions on status.resource.status to time out.

Observed

CI flake in server-update test step 1: assertion port2.status.resource.status == 'ACTIVE' timed out after 4 minutes.

Neutron API logs confirm the port went ACTIVE at 14:08:56, but ORC's last fetch was at 14:08:54 (saw DOWN). No subsequent fetch occurred until namespace deletion at 14:12:45.

Root cause

serverToPortMapFunc detects that a port attached to a server still has status DOWN and attempts to re-trigger reconciliation by:

  1. Patching the port's Progressing condition to True via SSA
  2. Returning a reconcile request

However, the SSA patch uses field owner GetSSAFieldOwner(controllerName) (openstack.k-orc.cloud/portcontroller), while the generic reconciler's UpdateStatus uses GetSSAFieldOwnerWithTxn(controllerName, SSATransactionStatus) (openstack.k-orc.cloud/portcontroller/status). These are different SSA field owners, both with ForceOwnership.

This creates a race:

  1. Port controller reconciles port — fetches from Neutron (DOWN), starts writing status
  2. Server controller writes Server status with port in interfaces → serverToPortMapFunc fires
  3. MapFunc patches Progressing=True (field owner: portcontroller)
  4. Port controller's UpdateStatus writes Progressing=False (field owner: portcontroller/status), overwriting step 3
  5. MapFunc's enqueued reconcile request is processed — shouldReconcile() sees Progressing=False → skips

The port is never re-fetched from Neutron. serverToVolumeMapFunc has the identical pattern.

Affected code

  • internal/controllers/port/controller.goserverToPortMapFunc SSA patch
  • internal/controllers/volume/controller.goserverToVolumeMapFunc SSA patch

ORC version

main

Additional information

No response

Relevant log output

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Fields

    No fields configured for Bug.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions