2025-09-11 03:05:17,464 - INFO - Domain Default found not creating 2025-09-11 03:05:17,633 - INFO - Project ctest-DisablePolicyEcmpIpv6-65093989 not found, creating it 2025-09-11 03:05:18,182 - INFO - Created Project:ctest-DisablePolicyEcmpIpv6-65093989, ID : 4ca31dd1-2a25-4813-b78c-80c16a93617a 2025-09-11 03:05:19,751 - INFO - ================================================================================ 2025-09-11 03:05:19,751 - INFO - STARTING TEST : test_ecmp_with_static_routes 2025-09-11 03:05:19,751 - INFO - TEST DESCRIPTION : Description: Verify disabling policy for ECMP routes with static routes on VM Steps: 1. launch 1 VN and launch 3 VMs in it. 2. create a static route for a new subnet prefix and add this on 2 VMIs. this will create 2 ECMP routes. 3. Disable the policy on all VMIs. 4. Now from 3rd VM send traffic to an IP from static route prefix 5. add new ECMP destinations and verify load is distributed to new destinations too 6. remove ECMP destinations and verify load is distributed to remaining destinations Pass criteria: 1. traffic should go through fine 2. flows should not be created 3. load should be distributed among ecmp routes. 2025-09-11 03:05:21,029 - DEBUG - Nothing to compare xmpp stats {'10.0.0.254': {'10.20.0.17': '0', '10.20.0.25': '0'}, '10.0.0.19': {'10.20.0.17': '0', '10.20.0.25': '0'}} with 2025-09-11 03:05:21,029 - INFO - Address family configured is dual 2025-09-11 03:05:21,029 - INFO - we are setting address family as v6 2025-09-11 03:05:21,029 - INFO - we are setting address family as v6 2025-09-11 03:05:21,029 - INFO - Initial checks done. Running the testcase now 2025-09-11 03:05:21,029 - INFO - 2025-09-11 03:05:21,054 - DEBUG - Services list from nova: [, , , ] 2025-09-11 03:05:21,054 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-4079-1', 'an-jenkins-deploy-platform-ansible-os-4079-2']} 2025-09-11 03:05:21,720 - DEBUG - Response for create_network : {'network': {'id': '44d5e35f-daab-439f-b861-67bbb3f88350', 'name': 'ctest-vn-68223569', 'tenant_id': '4ca31dd12a254813b78c80c16a93617a', 'project_id': '4ca31dd12a254813b78c80c16a93617a', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-DisablePolicyEcmpIpv6-65093989', 'ctest-vn-68223569'], 'port_security_enabled': True, 'description': ''}} 2025-09-11 03:05:22,015 - DEBUG - Response for create_subnet : {'subnet': {'id': '4d6437ca-7b33-47f8-9014-49fb4ccc246c', 'name': '', 'tenant_id': '4ca31dd12a254813b78c80c16a93617a', 'network_id': '44d5e35f-daab-439f-b861-67bbb3f88350', 'ip_version': 6, 'cidr': '278d:5578:71ee:817e:5040:ce18::/96', 'allocation_pools': [{'start': '278d:5578:71ee:817e:5040:ce18:0:2', 'end': '278d:5578:71ee:817e:5040:ce18:ffff:fffe'}], 'gateway_ip': '278d:5578:71ee:817e:5040:ce18:0:1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '278d:5578:71ee:817e:5040:ce18:0:2', 'tags': [], 'project_id': '4ca31dd12a254813b78c80c16a93617a'}} 2025-09-11 03:05:22,244 - DEBUG - Response for create_subnet : {'subnet': {'id': '182f1659-794c-485e-90f2-0f6b30e1ce1e', 'name': '', 'tenant_id': '4ca31dd12a254813b78c80c16a93617a', 'network_id': '44d5e35f-daab-439f-b861-67bbb3f88350', 'ip_version': 4, 'cidr': '217.186.121.64/26', 'allocation_pools': [{'start': '217.186.121.66', 'end': '217.186.121.126'}], 'gateway_ip': '217.186.121.65', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '217.186.121.66', 'tags': [], 'project_id': '4ca31dd12a254813b78c80c16a93617a'}} 2025-09-11 03:05:22,296 - INFO - Created VN ctest-vn-68223569 2025-09-11 03:05:22,327 - DEBUG - VN ctest-vn-68223569 UUID is 44d5e35f-daab-439f-b861-67bbb3f88350 2025-09-11 03:05:22,444 - DEBUG - Requesting: http://10.0.0.249:8082/domains 2025-09-11 03:05:22,736 - DEBUG - Requesting: http://10.0.0.249:8082/domains 2025-09-11 03:05:22,763 - DEBUG - Requesting: http://10.0.0.249:8082/domain/6cefd47d-baca-4df6-a10c-ec963319da07 2025-09-11 03:05:22,786 - DEBUG - Requesting: http://10.0.0.249:8082/project/4ca31dd1-2a25-4813-b78c-80c16a93617a 2025-09-11 03:05:22,880 - DEBUG - Requesting: http://10.0.0.249:8082/virtual-network/44d5e35f-daab-439f-b861-67bbb3f88350 2025-09-11 03:05:22,898 - DEBUG - Requesting: http://10.0.0.249:8082/virtual-network/44d5e35f-daab-439f-b861-67bbb3f88350 2025-09-11 03:05:22,910 - DEBUG - Requesting: http://10.0.0.249:8082/routing-instance/5ac0826b-59ba-47ac-a2f2-ee2184bd25a4 2025-09-11 03:05:22,925 - DEBUG - Requesting: http://10.0.0.249:8082/routing-instance/5ac0826b-59ba-47ac-a2f2-ee2184bd25a4 2025-09-11 03:05:22,932 - DEBUG - Requesting: http://10.0.0.249:8082/route-target/c334ddc6-3e43-44d5-9d51-9f25bf09fc7c 2025-09-11 03:05:22,952 - DEBUG - Route Targets: ['target:64512:8000004'] 2025-09-11 03:05:22,952 - DEBUG - Requesting: http://10.0.0.249:8082/virtual-network/44d5e35f-daab-439f-b861-67bbb3f88350 2025-09-11 03:05:22,965 - DEBUG - Requesting: http://10.0.0.249:8082/routing-instance/5ac0826b-59ba-47ac-a2f2-ee2184bd25a4 2025-09-11 03:05:22,984 - INFO - Verified VN network id 14 for VN 44d5e35f-daab-439f-b861-67bbb3f88350 2025-09-11 03:05:22,984 - INFO - Verifications in API Server for VN ctest-vn-68223569 passed 2025-09-11 03:05:22,984 - DEBUG - Requesting: http://10.0.0.249:8082/virtual-network/44d5e35f-daab-439f-b861-67bbb3f88350 2025-09-11 03:05:22,997 - DEBUG - Requesting: http://10.0.0.249:8082/routing-instance/5ac0826b-59ba-47ac-a2f2-ee2184bd25a4 2025-09-11 03:05:23,007 - DEBUG - Requesting: http://10.0.0.249:8082/routing-instance/5ac0826b-59ba-47ac-a2f2-ee2184bd25a4 2025-09-11 03:05:23,014 - DEBUG - Requesting: http://10.0.0.249:8082/route-target/c334ddc6-3e43-44d5-9d51-9f25bf09fc7c 2025-09-11 03:05:23,027 - DEBUG - Control-node 10.0.0.249 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-DisablePolicyEcmpIpv6-65093989:ctest-vn-68223569', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-DisablePolicyEcmpIpv6-65093989', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '4960120565448524703', 'uuid-lslong': '13286014431595627344'}, 'enable': 'true', 'created': '2025-09-11T03:05:21', 'last-modified': '2025-09-11T03:05:22', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.835136'} 2025-09-11 03:05:23,033 - DEBUG - Route Targets: ['target:64512:8000004'] 2025-09-11 03:05:23,038 - DEBUG - Control-node 10.0.0.22 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-DisablePolicyEcmpIpv6-65093989:ctest-vn-68223569', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-DisablePolicyEcmpIpv6-65093989', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '4960120565448524703', 'uuid-lslong': '13286014431595627344'}, 'enable': 'true', 'created': '2025-09-11T03:05:21', 'last-modified': '2025-09-11T03:05:22', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.851464'} 2025-09-11 03:05:23,044 - DEBUG - Route Targets: ['target:64512:8000004'] 2025-09-11 03:05:23,050 - DEBUG - Control-node 10.0.0.49 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-DisablePolicyEcmpIpv6-65093989:ctest-vn-68223569', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-DisablePolicyEcmpIpv6-65093989', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '4960120565448524703', 'uuid-lslong': '13286014431595627344'}, 'enable': 'true', 'created': '2025-09-11T03:05:21', 'last-modified': '2025-09-11T03:05:22', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.862127'} 2025-09-11 03:05:23,056 - DEBUG - Route Targets: ['target:64512:8000004'] 2025-09-11 03:05:23,056 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn-68223569 passed 2025-09-11 03:05:23,056 - DEBUG - ====Verifying policy data for ctest-vn-68223569 in API_Server ====== 2025-09-11 03:05:23,056 - DEBUG - Requesting: http://10.0.0.249:8082/domains 2025-09-11 03:05:23,062 - DEBUG - Requesting: http://10.0.0.249:8082/domain/6cefd47d-baca-4df6-a10c-ec963319da07 2025-09-11 03:05:23,074 - DEBUG - Requesting: http://10.0.0.249:8082/project/4ca31dd1-2a25-4813-b78c-80c16a93617a 2025-09-11 03:05:23,168 - DEBUG - Requesting: http://10.0.0.249:8082/virtual-network/44d5e35f-daab-439f-b861-67bbb3f88350 2025-09-11 03:05:23,181 - DEBUG - =>VN ctest-vn-68223569 has no policy to be verified 2025-09-11 03:05:23,181 - DEBUG - Verifying the vn in opserver 2025-09-11 03:05:23,181 - DEBUG - Verifying the default-domain:ctest-DisablePolicyEcmpIpv6-65093989:ctest-vn-68223569 virtual network link through opserver 10.0.0.249 2025-09-11 03:05:23,181 - DEBUG - Requesting: http://10.0.0.249:8081/analytics/uves/virtual-networks 2025-09-11 03:05:23,492 - DEBUG - Requesting: http://10.0.0.249:8081/analytics/uves/virtual-networks 2025-09-11 03:05:23,574 - DEBUG - vn link and name as {'name': 'default-domain:ctest-DisablePolicyEcmpIpv6-65093989:ctest-vn-68223569', 'href': 'http://10.0.0.249:8081/analytics/uves/virtual-network/default-domain:ctest-DisablePolicyEcmpIpv6-65093989:ctest-vn-68223569?flat'} 2025-09-11 03:05:23,574 - DEBUG - Verifying the default-domain:ctest-DisablePolicyEcmpIpv6-65093989:ctest-vn-68223569 virtual network link through opserver 10.0.0.22 2025-09-11 03:05:23,574 - DEBUG - Requesting: http://10.0.0.22:8081/analytics/uves/virtual-networks 2025-09-11 03:05:23,885 - DEBUG - Requesting: http://10.0.0.22:8081/analytics/uves/virtual-networks 2025-09-11 03:05:23,954 - DEBUG - vn link and name as {'name': 'default-domain:ctest-DisablePolicyEcmpIpv6-65093989:ctest-vn-68223569', 'href': 'http://10.0.0.22:8081/analytics/uves/virtual-network/default-domain:ctest-DisablePolicyEcmpIpv6-65093989:ctest-vn-68223569?flat'} 2025-09-11 03:05:23,954 - DEBUG - Verifying the default-domain:ctest-DisablePolicyEcmpIpv6-65093989:ctest-vn-68223569 virtual network link through opserver 10.0.0.49 2025-09-11 03:05:23,955 - DEBUG - Requesting: http://10.0.0.49:8081/analytics/uves/virtual-networks 2025-09-11 03:05:24,331 - DEBUG - Requesting: http://10.0.0.49:8081/analytics/uves/virtual-networks 2025-09-11 03:05:24,395 - DEBUG - vn link and name as {'name': 'default-domain:ctest-DisablePolicyEcmpIpv6-65093989:ctest-vn-68223569', 'href': 'http://10.0.0.49:8081/analytics/uves/virtual-network/default-domain:ctest-DisablePolicyEcmpIpv6-65093989:ctest-vn-68223569?flat'} 2025-09-11 03:05:24,396 - INFO - Validated that VN default-domain:ctest-DisablePolicyEcmpIpv6-65093989:ctest-vn-68223569 is found in opserver 2025-09-11 03:05:24,421 - DEBUG - Do not have enough data to verify VN in agent 2025-09-11 03:05:24,440 - DEBUG - VRF ids for VN ctest-vn-68223569: {} 2025-09-11 03:05:24,441 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-4079-1', 'an-jenkins-deploy-platform-ansible-os-4079-2']} 2025-09-11 03:05:25,839 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4079-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4079-1) 2025-09-11 03:05:25,840 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-4079-1', 'an-jenkins-deploy-platform-ansible-os-4079-2']} 2025-09-11 03:05:27,744 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4079-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4079-2) 2025-09-11 03:05:29,110 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4079-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4079-2) 2025-09-11 03:05:29,111 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-4079-1', 'an-jenkins-deploy-platform-ansible-os-4079-2']} 2025-09-11 03:05:30,522 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4079-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4079-1) 2025-09-11 03:05:30,522 - INFO - Waiting for VM ctest-DisablePolicyEcmpIpv6-65093989-71519865 to be up.. 2025-09-11 03:05:30,593 - DEBUG - VM is in ERROR state now 2025-09-11 03:05:30,593 - ERROR - VM in error state. 2025-09-11 03:05:30,593 - ERROR - VM is not in ACTIVE state 2025-09-11 03:05:30,693 - ERROR - Fatal Nova Exception while getting VM detail 2025-09-11 03:05:30,694 - DEBUG - None 2025-09-11 03:05:30,698 - INFO - Skip interface_detach for VM ctest-DisablePolicyEcmpIpv6-65093989-57559965 in state BUILD 2025-09-11 03:05:30,698 - INFO - Deleting VM ctest-DisablePolicyEcmpIpv6-65093989-57559965 2025-09-11 03:05:30,698 - ERROR - VM ctest-DisablePolicyEcmpIpv6-65093989-57559965 has failed to come up 2025-09-11 03:05:30,699 - ERROR - Nova failed to get host of the VM 2025-09-11 03:05:30,699 - ERROR - VM ctest-DisablePolicyEcmpIpv6-65093989-57559965 has failed to come up 2025-09-11 03:05:30,699 - ERROR - Nova failed to get host of the VM 2025-09-11 03:05:30,794 - INFO - Skip interface_detach for VM ctest-DisablePolicyEcmpIpv6-65093989-06161631 in state ERROR 2025-09-11 03:05:30,795 - INFO - Deleting VM ctest-DisablePolicyEcmpIpv6-65093989-06161631 2025-09-11 03:05:30,795 - ERROR - VM ctest-DisablePolicyEcmpIpv6-65093989-06161631 has failed to come up 2025-09-11 03:05:30,795 - ERROR - Fault seen in nova show is: {'code': 500, 'created': '2025-09-11T03:05:29Z', 'message': 'No valid host was found. ', 'details': 'Traceback (most recent call last):\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 1581, in schedule_and_build_instances\n instance_uuids, return_alternates=True)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 942, in _schedule_instances\n return_alternates=return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations\n instance_uuids, return_objects, return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations\n return cctxt.call(ctxt, \'select_destinations\', **msg_args)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 192, in call\n retry=self.retry, transport_options=self.transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 691, in send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send\n raise result\nnova.exception_Remote.NoValidHost_Remote: No valid host was found. \nTraceback (most recent call last):\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner\n return func(*args, **kwargs)\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/manager.py", line 209, in select_destinations\n raise exception.NoValidHost(reason="")\n\nnova.exception.NoValidHost: No valid host was found. \n\n'} 2025-09-11 03:05:30,795 - INFO - Skip interface_detach for VM ctest-DisablePolicyEcmpIpv6-65093989-71519865 in state ERROR 2025-09-11 03:05:30,795 - INFO - Deleting VM ctest-DisablePolicyEcmpIpv6-65093989-71519865 2025-09-11 03:05:30,795 - ERROR - VM ctest-DisablePolicyEcmpIpv6-65093989-71519865 has failed to come up 2025-09-11 03:05:30,795 - ERROR - Fault seen in nova show is: {'code': 500, 'created': '2025-09-11T03:05:27Z', 'message': 'No valid host was found. ', 'details': 'Traceback (most recent call last):\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 1581, in schedule_and_build_instances\n instance_uuids, return_alternates=True)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 942, in _schedule_instances\n return_alternates=return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations\n instance_uuids, return_objects, return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations\n return cctxt.call(ctxt, \'select_destinations\', **msg_args)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 192, in call\n retry=self.retry, transport_options=self.transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 691, in send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send\n raise result\nnova.exception_Remote.NoValidHost_Remote: No valid host was found. \nTraceback (most recent call last):\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner\n return func(*args, **kwargs)\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/manager.py", line 209, in select_destinations\n raise exception.NoValidHost(reason="")\n\nnova.exception.NoValidHost: No valid host was found. \n\n'} 2025-09-11 03:05:30,795 - INFO - Skip interface_detach for VM ctest-DisablePolicyEcmpIpv6-65093989-17413420 in state BUILD 2025-09-11 03:05:30,795 - INFO - Deleting VM ctest-DisablePolicyEcmpIpv6-65093989-17413420 2025-09-11 03:05:30,795 - ERROR - VM ctest-DisablePolicyEcmpIpv6-65093989-17413420 has failed to come up 2025-09-11 03:05:30,795 - ERROR - Nova failed to get host of the VM 2025-09-11 03:05:30,795 - ERROR - VM ctest-DisablePolicyEcmpIpv6-65093989-17413420 has failed to come up 2025-09-11 03:05:30,795 - ERROR - Nova failed to get host of the VM 2025-09-11 03:05:30,864 - INFO - Deleting VN ctest-vn-68223569 2025-09-11 03:05:30,904 - DEBUG - VN 44d5e35f-daab-439f-b861-67bbb3f88350 still in use: Unable to complete operation on network 44d5e35f-daab-439f-b861-67bbb3f88350. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-33af684e-9fa7-4956-8b90-1e18b7a8a3fd'] 2025-09-11 03:05:30,904 - WARNING - Deleting VN ctest-vn-68223569 failed..Will retry 2025-09-11 03:05:33,104 - DEBUG - Response for deleting network () 2025-09-11 03:05:33,104 - DEBUG - Requesting: http://10.0.0.249:8082/routing-instance/5ac0826b-59ba-47ac-a2f2-ee2184bd25a4 2025-09-11 03:05:33,110 - DEBUG - Response Code: 404 2025-09-11 03:05:33,111 - DEBUG - Requesting: http://10.0.0.249:8082/domains 2025-09-11 03:05:33,117 - DEBUG - Requesting: http://10.0.0.249:8082/domain/6cefd47d-baca-4df6-a10c-ec963319da07 2025-09-11 03:05:33,169 - DEBUG - Requesting: http://10.0.0.249:8082/project/4ca31dd1-2a25-4813-b78c-80c16a93617a 2025-09-11 03:05:33,256 - INFO - Validated that VN ctest-vn-68223569 is not found in API Server 2025-09-11 03:05:33,269 - DEBUG - VN ctest-vn-68223569 is not present in Agent 10.0.0.254 2025-09-11 03:05:33,281 - DEBUG - VN ctest-vn-68223569 is not present in Agent 10.0.0.19 2025-09-11 03:05:33,281 - INFO - Validated that VN ctest-vn-68223569 is not in any agent 2025-09-11 03:05:33,312 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-vn-68223569 info 2025-09-11 03:05:34,559 - ERROR - AssertionError Python 3.9.21: /usr/bin/python3 Thu Sep 11 03:05:30 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={}) 78 log.info('Initial checks done. Running the testcase now') 79 log.info('') 80 result = function(self, *args, **kwargs) 81 if self.inputs.upgrade: 82 pid = os.getpid() result = None function = self = args = () kwargs = {} /contrail-test/scripts/vrouter/test_disable_policy.py in test_ecmp_with_static_routes(self=) 63 node_name=compute_hosts[0], image_name=image)[0] 64 65 self.verify_vms(vm_fixtures) 66 self.verify_vms(vm1_fixture) 67 vm1_fixture = vm1_fixture[0] self = self.verify_vms = > vm_fixtures = [, ] /contrail-test/common/vrouter/base.py in verify_vms(self=, vm_fixtures=[, ]) 198 def verify_vms(self, vm_fixtures): 199 for vm_fixture in vm_fixtures: 200 assert vm_fixture.wait_till_vm_is_up() 201 202 def add_static_routes_on_vms(self,prefix, vm_fixtures, ip=None): vm_fixture = vm_fixture.wait_till_vm_is_up = > AssertionError: __cause__ = None __class__ = __context__ = None __delattr__ = __dict__ = {} __dir__ = __doc__ = 'Assertion failed.' __eq__ = __format__ = __ge__ = __getattribute__ = __gt__ = __hash__ = __init__ = __init_subclass__ = __le__ = __lt__ = __ne__ = __new__ = __reduce__ = __reduce_ex__ = __repr__ = __setattr__ = __setstate__ = __sizeof__ = __str__ = __subclasshook__ = __suppress_context__ = False __traceback__ = args = () with_traceback = The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper result = function(self, *args, **kwargs) File "/contrail-test/scripts/vrouter/test_disable_policy.py", line 65, in test_ecmp_with_static_routes self.verify_vms(vm_fixtures) File "/contrail-test/common/vrouter/base.py", line 200, in verify_vms assert vm_fixture.wait_till_vm_is_up() AssertionError 2025-09-11 03:05:34,572 - DEBUG - No XMPP flaps were noticed during the test 2025-09-11 03:05:34,572 - INFO - 2025-09-11 03:05:34,572 - INFO - END TEST : test_ecmp_with_static_routes : FAILED[0:00:15] 2025-09-11 03:05:34,572 - INFO - -------------------------------------------------------------------------------- 2025-09-11 03:05:35,287 - INFO - Deleted project: ctest-DisablePolicyEcmpIpv6-65093989, ID : 4ca31dd1-2a25-4813-b78c-80c16a93617a