Skip to content

Port nni removed#514

Draft
YufengXin wants to merge 13 commits intomainfrom
478-link-status-not-updating-to-error-after-nni-attribute-removal-use-case-9
Draft

Port nni removed#514
YufengXin wants to merge 13 commits intomainfrom
478-link-status-not-updating-to-error-after-nni-attribute-removal-use-case-9

Conversation

@YufengXin
Copy link
Copy Markdown
Collaborator

No description provided.

@YufengXin YufengXin self-assigned this Feb 3, 2026
@YufengXin YufengXin marked this pull request as draft February 3, 2026 22:26
@coveralls
Copy link
Copy Markdown

coveralls commented Feb 3, 2026

Coverage Report for CI Build 24199243205

Coverage decreased (-0.09%) to 53.83%

Details

  • Coverage decreased (-0.09%) from the base build.
  • Patch coverage: 8 uncovered changes across 2 files (0 of 8 lines covered, 0.0%).
  • 3 coverage regressions across 2 files.

Uncovered Changes

File Changed Covered %
sdx_controller/handlers/connection_handler.py 5 0 0.0%
sdx_controller/handlers/lc_message_handler.py 3 0 0.0%

Coverage Regressions

3 previously-covered lines in 2 files lost coverage.

File Lines Losing Coverage Coverage
sdx_controller/handlers/connection_handler.py 2 66.5%
sdx_controller/handlers/lc_message_handler.py 1 13.46%

Coverage Stats

Coverage Status
Relevant Lines: 2389
Covered Lines: 1286
Line Coverage: 53.83%
Coverage Strength: 1.08 hits per line

💛 - Coveralls

Yufeng Xin added 3 commits February 11, 2026 14:04
…pdating-to-error-after-nni-attribute-removal-use-case-9
…pdating-to-error-after-nni-attribute-removal-use-case-9
@YufengXin
Copy link
Copy Markdown
Collaborator Author

YufengXin commented Feb 25, 2026

sdx-controller-1 | 2026-02-25T19:51:39.821652706Z DEBUG:sdx_controller.handlers.connection_handler:Found the down port record for port urn:sdx:port:ampath.net:Ampath1:50! sdx-controller-1 | 2026-02-25T19:51:39.822305251Z INFO:sdx_controller.handlers.connection_handler:Updating connection 6b6ba034-975b-45f1-9c5d-b4acf96bc3d5 status to 'down'. sdx-controller-1 | 2026-02-25T19:51:39.823253842Z DEBUG:sdx_controller.handlers.connection_handler:Connection status updated for 6b6ba034-975b-45f1-9c5d-b4acf96bc3d5 sdx-controller-1 | 2026-02-25T19:51:39.823260204Z INFO:sdx_controller.handlers.lc_message_handler:Adding topology ampath.net to db. sdx-controller-1 | 2026-02-25T19:51:39.825682001Z INFO:sdx_controller.handlers.lc_message_handler:Save to database complete. sdx-controller-1 | 2026-02-25T19:51:39.825762522Z DEBUG:sdx_controller.messaging.rpc_queue_consumer:MQ received message:b'{"lc_domain": "ampath.net", "msg_type": "oxp_conn_response", "service_id": "6b6ba034-975b-45f1-9c5d-b4acf96bc3d5", "operation": "delete", "oxp_response_code": 201, "oxp_response": "L2VPN Deleted"}' sdx-controller-1 | 2026-02-25T19:51:39.825769756Z INFO:sdx_controller.handlers.lc_message_handler:MQ received message:b'{"lc_domain": "ampath.net", "msg_type": "oxp_conn_response", "service_id": "6b6ba034-975b-45f1-9c5d-b4acf96bc3d5", "operation": "delete", "oxp_response_code": 201, "oxp_response": "L2VPN Deleted"}' sdx-controller-1 | 2026-02-25T19:51:39.825863632Z INFO:sdx_controller.handlers.lc_message_handler:Received OXP connection response. sdx-controller-1 | 2026-02-25T19:51:39.827290161Z INFO:sdx_controller.handlers.lc_message_handler:Connection updated: 6b6ba034-975b-45f1-9c5d-b4acf96bc3d5 sdx-controller-1 | 2026-02-25T19:51:39.827298106Z DEBUG:sdx_controller.messaging.rpc_queue_consumer:MQ received message:b'{"lc_domain": "sax.net", "msg_type": "oxp_conn_response", "service_id": "6b6ba034-975b-45f1-9c5d-b4acf96bc3d5", "operation": "delete", "oxp_response_code": 201, "oxp_response": "L2VPN Deleted"}' sdx-controller-1 | 2026-02-25T19:51:39.827369750Z INFO:sdx_controller.handlers.lc_message_handler:MQ received message:b'{"lc_domain": "sax.net", "msg_type": "oxp_conn_response", "service_id": "6b6ba034-975b-45f1-9c5d-b4acf96bc3d5", "operation": "delete", "oxp_response_code": 201, "oxp_response": "L2VPN Deleted"}' sdx-controller-1 | 2026-02-25T19:51:39.827375120Z INFO:sdx_controller.handlers.lc_message_handler:Received OXP connection response. sdx-controller-1 | 2026-02-25T19:51:39.828775099Z INFO:sdx_controller.handlers.lc_message_handler:Connection updated: 6b6ba034-975b-45f1-9c5d-b4acf96bc3d5 sdx-controller-1 | 2026-02-25T19:51:55.056439769Z DEBUG:sdx_controller.messaging.rpc_queue_consumer:MQ received message:b'{"lc_domain": "sax.net", "msg_type": "oxp_conn_response", "service_id": "6b6ba034-975b-45f1-9c5d-b4acf96bc3d5", "operation": "post", "oxp_response_code": 201, "oxp_response": {"service_id": "4b334e727e5649"}}' sdx-controller-1 | 2026-02-25T19:51:55.056459196Z INFO:sdx_controller.handlers.lc_message_handler:MQ received message:b'{"lc_domain": "sax.net", "msg_type": "oxp_conn_response", "service_id": "6b6ba034-975b-45f1-9c5d-b4acf96bc3d5", "operation": "post", "oxp_response_code": 201, "oxp_response": {"service_id": "4b334e727e5649"}}' sdx-controller-1 | 2026-02-25T19:51:55.056490735Z INFO:sdx_controller.handlers.lc_message_handler:Received OXP connection response. sdx-controller-1 | 2026-02-25T19:51:55.058690395Z INFO:sdx_controller.handlers.lc_message_handler:Connection updated: 6b6ba034-975b-45f1-9c5d-b4acf96bc3d5 sdx-controller-1 | 2026-02-25T19:51:57.275081963Z DEBUG:sdx_controller.messaging.rpc_queue_consumer:MQ received message:b'{"lc_domain": "ampath.net", "msg_type": "oxp_conn_response", "service_id": "6b6ba034-975b-45f1-9c5d-b4acf96bc3d5", "operation": "post", "oxp_response_code": 201, "oxp_response": {"service_id": "421aebf496444f"}}' sdx-controller-1 | 2026-02-25T19:51:57.275126095Z INFO:sdx_controller.handlers.lc_message_handler:MQ received message:b'{"lc_domain": "ampath.net", "msg_type": "oxp_conn_response", "service_id": "6b6ba034-975b-45f1-9c5d-b4acf96bc3d5", "operation": "post", "oxp_response_code": 201, "oxp_response": {"service_id": "421aebf496444f"}}' sdx-controller-1 | 2026-02-25T19:51:57.275140673Z INFO:sdx_controller.handlers.lc_message_handler:Received OXP connection response. sdx-controller-1 | 2026-02-25T19:51:57.278322828Z ERROR:sdx_controller.messaging.rpc_queue_consumer:Failed to process LC message: Invalid transition from DOWN to UP -- Traceback (most recent call last):, File "/opt/venv/lib/python3.11/site-packages/sdx_controller/messaging/rpc_queue_consumer.py", line 371, in start_sdx_consumer, lc_message_handler.process_lc_json_msg(, File "/opt/venv/lib/python3.11/site-packages/sdx_controller/handlers/lc_message_handler.py", line 77, in process_lc_json_msg, connection, _ = connection_state_machine(, ^^^^^^^^^^^^^^^^^^^^^^^^^, File "/opt/venv/lib/python3.11/site-packages/sdx_controller/handlers/connection_handler.py", line 854, in connection_state_machine, conn_sm.transition(new_state), File "/opt/venv/lib/python3.11/site-packages/sdx_datamodel/connection_sm.py", line 249, in transition, raise ConnectionSMException(, sdx_datamodel.connection_sm.ConnectionSMException: Invalid transition from DOWN to UP,

------------Observed in test_21_use_case_topology.py::TestE2ETopologyUseCases::test_091_port_missing_nni------

  1. all ports "down" from OXPs tenet and sax updates
  2. above loved msg: OXP breakdown response: first "delete", then "operation"....

@YufengXin
Copy link
Copy Markdown
Collaborator Author

@gretelliz @italovalcy see above logs. Were there some recent updates in Kytos?

  1. All the ports from tenet and sax update were "down"
  2. After the controller told the oxps to delete, the OXPs reported breakdowns "delete", then "operation".

@YufengXin
Copy link
Copy Markdown
Collaborator Author

First clean run:
` ampath_api = KYTOS_API % 'ampath'
api_url_ampath = f'{ampath_api}/topology/v3'
response = requests.delete(f"{api_url_ampath}/interfaces/{interfaces_id}/metadata/sdx_nni")
assert response.status_code == 200, response.text

    # Force to send the topology to the SDX-LC
    sdx_api = KYTOS_SDX_API % 'ampath'
    response = requests.post(f"{sdx_api}/topology/2.0.0")
    assert response.status_code == 200, response.text

    time.sleep(5)

    response = requests.get(API_URL_TOPO)
    assert response.status_code == 200, response.text
    updated_topology = response.json()
    for node in updated_topology['nodes']:
        if node['name'] == port_name.split('-')[0]:
            for port in node['ports']:
                if port['name'] == port_name:
                  assert port['nni'] == ''

E AssertionError: assert 'urn:sdx:port....net:Sax01:40' == ''
E + urn:sdx:port:sax.net:Sax01:40

tests/test_21_use_case_topology.py:348: AssertionError
---------------------------- Captured stderr setup -----------------------------
*** Error setting resource limits. Mininet's performance may be affected.
------------------------------ Captured log setup ------------------------------
WARNING mininet:log.py:153 *** Error setting resource limits. Mininet's performance may be affected.
------------------------------- start/stop times -------------------------------
tests/test_21_use_case_topology.py::TestE2ETopologyUseCases::test_091_port_missing_nni: 2026-03-18,14:44:20.149589 - 2026-03-18,14:44:34.373896
=========================== short test summary info ============================
FAILED tests/test_21_use_case_topology.py::TestE2ETopologyUseCases::test_091_port_missing_nni
========================= 1 failed in 62.97s (0:01:02) =========================`

Second clean run:
The same

Second clean run:
The same

@lmarinve lmarinve requested a review from italovalcy March 25, 2026 17:06
@YufengXin YufengXin requested a review from gretelliz March 25, 2026 17:06
@lmarinve lmarinve removed the request for review from italovalcy March 25, 2026 17:07
@gretelliz
Copy link
Copy Markdown
Contributor

gretelliz commented Mar 25, 2026

Hi @YufengXin, I ran the end-to-end test test_091_port_missing_nni on main.
The NNI attribute removal is working correctly. After doing:

DELETE /ampath/topology/v3/interfaces/{interfaces_id}/metadata/sdx_nni

and forcing the topology update with POST /ampath/topology/2.0.0, the port correctly shows:
”nni": “”
The assertion assert port['nni'] == '' passes.
However, the test still fails here:
assert link['status'] == 'error’
The link Ampath1-eth40--Sax01-eth40 remains in status 'up' instead of changing to 'error'.

@YufengXin
Copy link
Copy Markdown
Collaborator Author

YufengXin commented Mar 26, 2026

@gretelliz Do I have to do these two steps (DELETE and POST) manually? Or the two steps in your test_091_port_missing_nni worked now?

And is there a new Kytos image to use?

Thx

@gretelliz
Copy link
Copy Markdown
Contributor

Hello @YufengXin, the DELETE /ampath/topology/v3/interfaces/{id}/metadata/sdx_nni and POST /ampath/topology/2.0.0 steps are executed in the test_091_port_missing_nni and function as expected.

End-to-end tests are always run with the latest available image.

Yufeng Xin added 4 commits March 30, 2026 13:37
Merge remote-tracking branch 'origin/main' into 478-link-status-not-updating-to-error-after-nni-attribute-removal-use-case-9
@YufengXin
Copy link
Copy Markdown
Collaborator Author

YufengXin commented Mar 30, 2026

Initially-------Tenet topology, everything is up
{"name": "Tenet-OXP", "id": "urn:sdx:topology:tenet.ac.za", "version": 9, "timestamp": "2026-03-30T21:05:35Z", "model_version": "2.0.0", "nodes": [{"name": "Tenet01", "id": "urn:sdx:node:tenet.ac.za:Tenet01", "location": {"address": "CapeTown", "latitude": -33.0, "longitude": 18.0, "iso3166_2_lvl4": "ZA-WC", "private": []}, "ports": [{"id": "urn:sdx:port:tenet.ac.za:Tenet01:41", "name": "Tenet01-eth41", "node": "urn:sdx:node:tenet.ac.za:Tenet01", "type": "10GE", "status": "up", "state": "enabled", "mtu": 1500, "nni": "urn:sdx:port:sax.net:Sax01:41", "services": {"l2vpn-ptp": {"vlan_range": [[1, 4094]]}}, "private": ["status"], "entities": []}, {"id": "urn:sdx:port:tenet.ac.za:Tenet01:1", "name": "Tenet01-eth1", "node": "urn:sdx:node:tenet.ac.za:Tenet01", "type": "10GE", "status": "up", "state": "enabled", "mtu": 1500, "nni": "urn:sdx:link:tenet.ac.za:Tenet01/1_Tenet02/1", "services": {"l2vpn-ptp": {"vlan_range": [[1, 4094]]}}, "private": ["status"], "entities": []}, {"id": "urn:sdx:port:tenet.ac.za:Tenet01:2", "name": "Tenet01-eth2", "node": "urn:sdx:node:tenet.ac.za:Tenet01", "type": "10GE", "status": "up", "state": "enabled", "mtu": 1500, "nni": "urn:sdx:link:tenet.ac.za:Tenet01/2_Tenet03/2", "services": {"l2vpn-ptp": {"vlan_range": [[1, 4094]]}}, "private": ["status"], "entities": []}, {"id": "urn:sdx:port:tenet.ac.za:Tenet01:50", "name": "Tenet01-eth50", "node": "urn:sdx:node:tenet.ac.za:Tenet01", "type": "10GE", "status": "up", "state": "enabled", "mtu": 1500, "nni": "", "services": {"l2vpn-ptp": {"vlan_range": [[1, 4094]]}}, "private": ["status"], "entities": []}], "status": "up", "state": "enabled"}, {"name": "Tenet02", "id": "urn:sdx:node:tenet.ac.za:Tenet02", "location": {"address": "Johanesburgo", "latitude": -26.0, "longitude": 28.0, "iso3166_2_lvl4": "ZA-GP", "private": []}, "ports": [{"id": "urn:sdx:port:tenet.ac.za:Tenet02:50", "name": "Tenet02-eth50", "node": "urn:sdx:node:tenet.ac.za:Tenet02", "type": "10GE", "status": "up", "state": "enabled", "mtu": 1500, "nni": "", "services": {"l2vpn-ptp": {"vlan_range": [[1, 4094]]}}, "private": ["status"], "entities": []}, {"id": "urn:sdx:port:tenet.ac.za:Tenet02:41", "name": "Tenet02-eth41", "node": "urn:sdx:node:tenet.ac.za:Tenet02", "type": "10GE", "status": "up", "state": "enabled", "mtu": 1500, "nni": "urn:sdx:port:sax.net:Sax02:41", "services": {"l2vpn-ptp": {"vlan_range": [[1, 4094]]}}, "private": ["status"], "entities": []}, {"id": "urn:sdx:port:tenet.ac.za:Tenet02:1", "name": "Tenet02-eth1", "node": "urn:sdx:node:tenet.ac.za:Tenet02", "type": "10GE", "status": "up", "state": "enabled", "mtu": 1500, "nni": "urn:sdx:link:tenet.ac.za:Tenet01/1_Tenet02/1", "services": {"l2vpn-ptp": {"vlan_range": [[1, 4094]]}}, "private": ["status"], "entities": []}], "status": "up", "state": "enabled"}, {"name": "Tenet03", "id": "urn:sdx:node:tenet.ac.za:Tenet03", "location": {"address": "EastLondon", "latitude": -33.0, "longitude": 27.0, "iso3166_2_lvl4": "ZA-EC", "private": []}, "ports": [{"id": "urn:sdx:port:tenet.ac.za:Tenet03:50", "name": "Tenet03-eth50", "node": "urn:sdx:node:tenet.ac.za:Tenet03", "type": "10GE", "status": "up", "state": "enabled", "mtu": 1500, "nni": "", "services": {"l2vpn-ptp": {"vlan_range": [[1, 4094]]}}, "private": ["status"], "entities": []}, {"id": "urn:sdx:port:tenet.ac.za:Tenet03:2", "name": "Tenet03-eth2", "node": "urn:sdx:node:tenet.ac.za:Tenet03", "type": "10GE", "status": "up", "state": "enabled", "mtu": 1500, "nni": "urn:sdx:link:tenet.ac.za:Tenet01/2_Tenet03/2", "services": {"l2vpn-ptp": {"vlan_range": [[1, 4094]]}}, "private": ["status"], "entities": []}], "status": "up", "state": "enabled"}], "links": [{"name": "Tenet01/1_Tenet02/1", "id": "urn:sdx:link:tenet.ac.za:Tenet01/1_Tenet02/1", "ports": ["urn:sdx:port:tenet.ac.za:Tenet01:1", "urn:sdx:port:tenet.ac.za:Tenet02:1"], "type": "intra", "bandwidth": 10, "residual_bandwidth": 100, "latency": 0, "packet_loss": 0, "availability": 100, "status": "up", "state": "enabled", "private": ["packet_loss"]}, {"name": "Tenet01/2_Tenet03/2", "id": "urn:sdx:link:tenet.ac.za:Tenet01/2_Tenet03/2", "ports": ["urn:sdx:port:tenet.ac.za:Tenet01:2", "urn:sdx:port:tenet.ac.za:Tenet03:2"], "type": "intra", "bandwidth": 10, "residual_bandwidth": 100, "latency": 0, "packet_loss": 0, "availability": 100, "status": "up", "state": "enabled", "private": ["packet_loss"]}], "services": ["l2vpn-ptp"]}

Afterwards---removing one 'nni' caused multiple links down
{'name': 'Tenet-OXP', 'id': 'urn:sdx:topology:tenet.ac.za', 'version': 12, 'timestamp': '2026-03-30T21:07:28Z', 'model_version': '2.0.0', 'nodes': [{'name': 'Tenet01', 'id': 'urn:sdx:node:tenet.ac.za:Tenet01', 'location': {'address': 'CapeTown', 'latitude': -33.0, 'longitude': 18.0, 'iso3166_2_lvl4': 'ZA-WC', 'private': []}, 'ports': [{'id': 'urn:sdx:port:tenet.ac.za:Tenet01:41', 'name': 'Tenet01-eth41', 'node': 'urn:sdx:node:tenet.ac.za:Tenet01', 'type': '10GE', 'status': 'down', 'state': 'enabled', 'mtu': 1500, 'nni': 'urn:sdx:port:sax.net:Sax01:41', 'services': {'l2vpn-ptp': {'vlan_range': [[1, 4094]]}}, 'private': ['status'], 'entities': []}, {'id': 'urn:sdx:port:tenet.ac.za:Tenet01:1', 'name': 'Tenet01-eth1', 'node': 'urn:sdx:node:tenet.ac.za:Tenet01', 'type': '10GE', 'status': 'down', 'state': 'enabled', 'mtu': 1500, 'nni': 'urn:sdx:link:tenet.ac.za:Tenet01/1_Tenet02/1', 'services': {'l2vpn-ptp': {'vlan_range': [[1, 4094]]}}, 'private': ['status'], 'entities': []}, {'id': 'urn:sdx:port:tenet.ac.za:Tenet01:2', 'name': 'Tenet01-eth2', 'node': 'urn:sdx:node:tenet.ac.za:Tenet01', 'type': '10GE', 'status': 'down', 'state': 'enabled', 'mtu': 1500, 'nni': 'urn:sdx:link:tenet.ac.za:Tenet01/2_Tenet03/2', 'services': {'l2vpn-ptp': {'vlan_range': [[1, 4094]]}}, 'private': ['status'], 'entities': []}, {'id': 'urn:sdx:port:tenet.ac.za:Tenet01:50', 'name': 'Tenet01-eth50', 'node': 'urn:sdx:node:tenet.ac.za:Tenet01', 'type': '10GE', 'status': 'down', 'state': 'enabled', 'mtu': 1500, 'nni': '', 'services': {'l2vpn-ptp': {'vlan_range': [[1, 4094]]}}, 'private': ['status'], 'entities': []}], 'status': 'down', 'state': 'enabled'}, {'name': 'Tenet02', 'id': 'urn:sdx:node:tenet.ac.za:Tenet02', 'location': {'address': 'Johanesburgo', 'latitude': -26.0, 'longitude': 28.0, 'iso3166_2_lvl4': 'ZA-GP', 'private': []}, 'ports': [{'id': 'urn:sdx:port:tenet.ac.za:Tenet02:50', 'name': 'Tenet02-eth50', 'node': 'urn:sdx:node:tenet.ac.za:Tenet02', 'type': '10GE', 'status': 'down', 'state': 'enabled', 'mtu': 1500, 'nni': '', 'services': {'l2vpn-ptp': {'vlan_range': [[1, 4094]]}}, 'private': ['status'], 'entities': []}, {'id': 'urn:sdx:port:tenet.ac.za:Tenet02:41', 'name': 'Tenet02-eth41', 'node': 'urn:sdx:node:tenet.ac.za:Tenet02', 'type': '10GE', 'status': 'down', 'state': 'enabled', 'mtu': 1500, 'nni': 'urn:sdx:port:sax.net:Sax02:41', 'services': {'l2vpn-ptp': {'vlan_range': [[1, 4094]]}}, 'private': ['status'], 'entities': []}, {'id': 'urn:sdx:port:tenet.ac.za:Tenet02:1', 'name': 'Tenet02-eth1', 'node': 'urn:sdx:node:tenet.ac.za:Tenet02', 'type': '10GE', 'status': 'down', 'state': 'enabled', 'mtu': 1500, 'nni': 'urn:sdx:link:tenet.ac.za:Tenet01/1_Tenet02/1', 'services': {'l2vpn-ptp': {'vlan_range': [[1, 4094]]}}, 'private': ['status'], 'entities': []}], 'status': 'down', 'state': 'enabled'}, {'name': 'Tenet03', 'id': 'urn:sdx:node:tenet.ac.za:Tenet03', 'location': {'address': 'EastLondon', 'latitude': -33.0, 'longitude': 27.0, 'iso3166_2_lvl4': 'ZA-EC', 'private': []}, 'ports': [{'id': 'urn:sdx:port:tenet.ac.za:Tenet03:50', 'name': 'Tenet03-eth50', 'node': 'urn:sdx:node:tenet.ac.za:Tenet03', 'type': '10GE', 'status': 'down', 'state': 'enabled', 'mtu': 1500, 'nni': '', 'services': {'l2vpn-ptp': {'vlan_range': [[1, 4094]]}}, 'private': ['status'], 'entities': []}, {'id': 'urn:sdx:port:tenet.ac.za:Tenet03:2', 'name': 'Tenet03-eth2', 'node': 'urn:sdx:node:tenet.ac.za:Tenet03', 'type': '10GE', 'status': 'down', 'state': 'enabled', 'mtu': 1500, 'nni': 'urn:sdx:link:tenet.ac.za:Tenet01/2_Tenet03/2', 'services': {'l2vpn-ptp': {'vlan_range': [[1, 4094]]}}, 'private': ['status'], 'entities': []}], 'status': 'down', 'state': 'enabled'}], 'links': [{'name': 'Tenet01/1_Tenet02/1', 'id': 'urn:sdx:link:tenet.ac.za:Tenet01/1_Tenet02/1', 'ports': ['urn:sdx:port:tenet.ac.za:Tenet01:1', 'urn:sdx:port:tenet.ac.za:Tenet02:1'], 'type': 'intra', 'bandwidth': 10, 'residual_bandwidth': 100, 'latency': 0, 'packet_loss': 0, 'availability': 100, 'status': 'down', 'state': 'enabled', 'private': ['packet_loss']}, {'name': 'Tenet01/2_Tenet03/2', 'id': 'urn:sdx:link:tenet.ac.za:Tenet01/2_Tenet03/2', 'ports': ['urn:sdx:port:tenet.ac.za:Tenet01:2', 'urn:sdx:port:tenet.ac.za:Tenet03:2'], 'type': 'intra', 'bandwidth': 10, 'residual_bandwidth': 100, 'latency': 0, 'packet_loss': 0, 'availability': 100, 'status': 'down', 'state': 'enabled', 'private': ['packet_loss']}], 'services': ['l2vpn-ptp'], 'domain_name': 'tenet.ac.za'}

Comments:
The OXP is not stable: in this test, it's the Tenet.

  1. 'nni' removal works once a while.
  2. When its removal works, multiple links were reported 'down'.

@gretelliz I am not going to continue this PR and will revisit later. Thx.
1.

@italovalcy
Copy link
Copy Markdown
Contributor

update from today's meeting: we had a joint troubleshooting session about this problem and we manage to. find that Kytos is sending the outdated topology to SDX-LC via push method (version 9), while the topology updated with the NNI removed (version 10) was not sent to SDX-LC. Something is happening at Kytos that is sending an outdated topology. I've filed an issue with Kytos to troubleshoot this atlanticwave-sdx/kytos-sdx#98

@italovalcy
Copy link
Copy Markdown
Contributor

I've dig deeper into this issue and submitted two PRs to fix the end-to-end tests (to add proper sleep times after NNI metadata removal) and also implement a few improvements to the Kytos SDX Napp to avoid sending the wrong topology.

After those changes, the tests are still failing with the following error (15 repeats):

========================================================================================================= FAILURES =========================================================================================================
____________________________________________________________________________________ TestE2ETopologyUseCases.test_091_port_missing_nni _____________________________________________________________________________________

self = <test_21_use_case_topology.TestE2ETopologyUseCases object at 0x7fd33d083e50>

    def test_091_port_missing_nni(self):
        """
        Use Case 9: OXPO sends a topology update with a Port missing
        """
        l2vpn_data = self.create_new_l2vpn(vlan='910', node1='Ampath1', node2= 'Sax01')
        l2vpn_id = l2vpn_data['id']
        port_name = 'Ampath1-eth40'
        link_name = 'Ampath1-eth40--Sax01-eth40'
        interfaces_id = 'aa:00:00:00:00:00:00:01:40'

        # Verify the link is up in the topology
        response = requests.get(API_URL_TOPO)
        assert response.status_code == 200, response.text
        topology = response.json()
        for link in topology['links']:
            if link['name'] == link_name:
                assert link['status'] == 'up'
                break

        ampath_api = KYTOS_API % 'ampath'
        api_url_ampath = f'{ampath_api}/topology/v3'
        response = requests.delete(f"{api_url_ampath}/interfaces/{interfaces_id}/metadata/sdx_nni")
        assert response.status_code == 200, response.text

        time.sleep(5)

        # Force to send the topology to the SDX-LC
        sdx_api = KYTOS_SDX_API % 'ampath'
        response = requests.post(f"{sdx_api}/topology/2.0.0")
        assert response.status_code == 200, response.text

        time.sleep(5)

        response = requests.get(f"{sdx_api}/topology/2.0.0")
        kytos_topo = response.json()
        for node in kytos_topo["nodes"]:
            for port in node["ports"]:
                if port["id"] == "urn:sdx:port:ampath.net:Ampath1:40":
                    assert port["nni"] == "", f"kytos_topo={kytos_topo}"
                    break

        response = requests.get(API_URL_TOPO)
        assert response.status_code == 200, response.text
        updated_topology = response.json()
        for node in updated_topology['nodes']:
            if node['name'] == port_name.split('-')[0]:
                for port in node['ports']:
                    if port['name'] == port_name:
                        assert port['nni'] == '', f"sdx_topo={updated_topology} kytos_topo={kytos_topo}"
        for link in updated_topology['links']:
            if link['name'] == link_name:
>               assert link['status'] == 'error', f"link={link}"
E               AssertionError: link={'availability': 100, 'bandwidth': 10, 'id': 'urn:sdx:link:interdomain:ampath.net:Ampath1:40:sax.net:Sax01:40', 'latency': 0, 'measurement_period': None, 'name': 'Ampath1-eth40--Sax01-eth40', 'packet_loss': 0, 'ports': ['urn:sdx:port:ampath.net:Ampath1:40', 'urn:sdx:port:sax.net:Sax01:40'], 'private_attributes': None, 'residual_bandwidth': 100.0, 'short_name': None, 'state': 'enabled', 'status': 'up', 'timestamp': None}
E               assert 'up' == 'error'
E                 - error
E                 + up

tests/test_21_use_case_topology.py:361: AssertionError
-------------------------------------------------------------------------------------------------- Captured stderr setup ---------------------------------------------------------------------------------------------------
*** Error setting resource limits. Mininet's performance may be affected.
---------------------------------------------------------------------------------------------------- Captured log setup ----------------------------------------------------------------------------------------------------
WARNING  mininet:log.py:153 *** Error setting resource limits. Mininet's performance may be affected.
----------------------------------------------------------------------------------------------------- start/stop times -----------------------------------------------------------------------------------------------------
tests/test_21_use_case_topology.py::TestE2ETopologyUseCases::test_091_port_missing_nni: 2026-04-08,20:44:18.358658 - 2026-04-08,20:44:37.683566
================================================================================================= short test summary info ==================================================================================================
FAILED tests/test_21_use_case_topology.py::TestE2ETopologyUseCases::test_091_port_missing_nni - AssertionError: link={'availability': 100, 'bandwidth': 10, 'id': 'urn:sdx:link:interdomain:ampath.net:Ampath1:40:sax.net:Sax01:40', 'latency': 0, 'measurement_period': None, 'name': 'Ampath1-eth40--Sax01-eth40', 'p...
=============================================================================================== 1 failed in 78.54s (0:01:18) ===============================================================================================

@YufengXin
Copy link
Copy Markdown
Collaborator Author

YufengXin commented Apr 9, 2026 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Link status not updating to "error" after nni attribute removal (Use Case 9)

5 participants