2025-12-22 03:49:22,311 - INFO - Domain Default found not creating
2025-12-22 03:49:22,486 - INFO - Project ctest-TestBGPaasZone-53547593 not found, creating it
2025-12-22 03:49:23,241 - INFO - Created Project:ctest-TestBGPaasZone-53547593, ID : 7b526cb0-ef57-466c-84c2-5bff8d0bce62 
2025-12-22 03:49:24,891 - DEBUG - Services list from nova: [<Service: 1>, <Service: 1>, <Service: 2>, <Service: 3>]
2025-12-22 03:49:24,891 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-4918-1', 'an-jenkins-deploy-platform-ansible-os-4918-2']}
2025-12-22 03:49:24,893 - INFO - ================================================================================
2025-12-22 03:49:24,893 - INFO - STARTING TEST    : test_bgp_control_node_zone
2025-12-22 03:49:26,167 - DEBUG - Skipping xmpp flap check
2025-12-22 03:49:26,167 - INFO - Initial checks done. Running the testcase now
2025-12-22 03:49:26,167 - INFO - 
2025-12-22 03:49:26,167 - INFO - executing bgp_control_zone test
2025-12-22 03:49:26,934 - DEBUG - Response for create_network : {'network': {'id': 'd243e9f9-547b-4e6f-be16-353855ce9677', 'name': 'ctest-cnz_vn-37819795', 'tenant_id': '7b526cb0ef57466c84c25bff8d0bce62', 'project_id': '7b526cb0ef57466c84c25bff8d0bce62', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestBGPaasZone-53547593', 'ctest-cnz_vn-37819795'], 'port_security_enabled': True, 'description': ''}}
2025-12-22 03:49:27,244 - DEBUG - Response for create_subnet : {'subnet': {'id': 'f788cd8f-7db4-4c38-81d3-f7f4aacba5a9', 'name': '', 'tenant_id': '7b526cb0ef57466c84c25bff8d0bce62', 'network_id': 'd243e9f9-547b-4e6f-be16-353855ce9677', 'ip_version': 4, 'cidr': '89.13.141.64/26', 'allocation_pools': [{'start': '89.13.141.66', 'end': '89.13.141.126'}], 'gateway_ip': '89.13.141.65', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '89.13.141.66', 'tags': [], 'project_id': '7b526cb0ef57466c84c25bff8d0bce62'}}
2025-12-22 03:49:27,272 - INFO - Created VN ctest-cnz_vn-37819795
2025-12-22 03:49:27,289 - DEBUG - VN ctest-cnz_vn-37819795 UUID is d243e9f9-547b-4e6f-be16-353855ce9677
2025-12-22 03:49:28,571 - INFO - VM ([<Server: ctest-TestBGPaasZone-53547593-55707067>]) created on node: (an-jenkins-deploy-platform-ansible-os-4918-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4918-1)
2025-12-22 03:49:29,769 - INFO - VM ([<Server: ctest-TestBGPaasZone-53547593-94137805>]) created on node: (an-jenkins-deploy-platform-ansible-os-4918-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4918-2)
2025-12-22 03:49:31,638 - INFO - VM ([<Server: ctest-TestBGPaasZone-53547593-47918528>]) created on node: (an-jenkins-deploy-platform-ansible-os-4918-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4918-1)
2025-12-22 03:49:31,638 - INFO - Waiting for VM ctest-TestBGPaasZone-53547593-55707067 to be up..
2025-12-22 03:49:31,715 - DEBUG - VM <Server: ctest-TestBGPaasZone-53547593-55707067> is in ERROR state now
2025-12-22 03:49:31,715 - ERROR - VM in error state.
2025-12-22 03:49:31,715 - ERROR - VM is not in ACTIVE state
2025-12-22 03:49:31,815 - ERROR - Fatal Nova Exception while getting VM detail
2025-12-22 03:49:31,815 - DEBUG - None
2025-12-22 03:49:31,815 - ERROR - VM ctest-TestBGPaasZone-53547593-55707067 has not booted
2025-12-22 03:49:31,815 - INFO - Skip interface_detach for VM ctest-TestBGPaasZone-53547593-55707067 in state ERROR
2025-12-22 03:49:31,815 - INFO - Deleting VM ctest-TestBGPaasZone-53547593-55707067
2025-12-22 03:49:31,841 - ERROR - VM ctest-TestBGPaasZone-53547593-55707067 has failed to come up
2025-12-22 03:49:31,841 - ERROR - Fault seen in nova show <vm-uuid> is:  {'code': 500, 'created': '2025-12-22T03:49:28Z', 'message': 'No valid host was found. ', 'details': 'Traceback (most recent call last):\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 1581, in schedule_and_build_instances\n    instance_uuids, return_alternates=True)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 942, in _schedule_instances\n    return_alternates=return_alternates)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations\n    instance_uuids, return_objects, return_alternates)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations\n    return cctxt.call(ctxt, \'select_destinations\', **msg_args)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 192, in call\n    retry=self.retry, transport_options=self.transport_options)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send\n    transport_options=transport_options)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 691, in send\n    transport_options=transport_options)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send\n    raise result\nnova.exception_Remote.NoValidHost_Remote: No valid host was found. \nTraceback (most recent call last):\n\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner\n    return func(*args, **kwargs)\n\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/manager.py", line 209, in select_destinations\n    raise exception.NoValidHost(reason="")\n\nnova.exception.NoValidHost: No valid host was found. \n\n'}
2025-12-22 03:49:31,844 - INFO - Skip interface_detach for VM ctest-TestBGPaasZone-53547593-47918528 in state ERROR
2025-12-22 03:49:31,844 - INFO - Deleting VM ctest-TestBGPaasZone-53547593-47918528
2025-12-22 03:49:31,844 - ERROR - VM ctest-TestBGPaasZone-53547593-47918528 has failed to come up
2025-12-22 03:49:31,844 - ERROR - Fault seen in nova show <vm-uuid> is:  {'code': 500, 'created': '2025-12-22T03:49:31Z', 'message': 'No valid host was found. ', 'details': 'Traceback (most recent call last):\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 1581, in schedule_and_build_instances\n    instance_uuids, return_alternates=True)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 942, in _schedule_instances\n    return_alternates=return_alternates)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations\n    instance_uuids, return_objects, return_alternates)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations\n    return cctxt.call(ctxt, \'select_destinations\', **msg_args)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 192, in call\n    retry=self.retry, transport_options=self.transport_options)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send\n    transport_options=transport_options)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 691, in send\n    transport_options=transport_options)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send\n    raise result\nnova.exception_Remote.NoValidHost_Remote: No valid host was found. \nTraceback (most recent call last):\n\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner\n    return func(*args, **kwargs)\n\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/manager.py", line 209, in select_destinations\n    raise exception.NoValidHost(reason="")\n\nnova.exception.NoValidHost: No valid host was found. \n\n'}
2025-12-22 03:49:31,844 - INFO - Skip interface_detach for VM ctest-TestBGPaasZone-53547593-94137805 in state BUILD
2025-12-22 03:49:31,844 - INFO - Deleting VM ctest-TestBGPaasZone-53547593-94137805
2025-12-22 03:49:31,844 - ERROR - VM ctest-TestBGPaasZone-53547593-94137805 has failed to come up
2025-12-22 03:49:31,844 - ERROR - Nova failed to get host of the VM
2025-12-22 03:49:31,845 - ERROR - VM ctest-TestBGPaasZone-53547593-94137805 has failed to come up
2025-12-22 03:49:31,845 - ERROR - Nova failed to get host of the VM
2025-12-22 03:49:31,903 - INFO - Skip interface_detach for VM ctest-TestBGPaasZone-53547593-55707067 in state ERROR
2025-12-22 03:49:31,903 - INFO - Deleting VM ctest-TestBGPaasZone-53547593-55707067
2025-12-22 03:49:31,903 - ERROR - VM ctest-TestBGPaasZone-53547593-55707067 has failed to come up
2025-12-22 03:49:31,903 - ERROR - Fault seen in nova show <vm-uuid> is:  {'code': 500, 'created': '2025-12-22T03:49:28Z', 'message': 'No valid host was found. ', 'details': 'Traceback (most recent call last):\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 1581, in schedule_and_build_instances\n    instance_uuids, return_alternates=True)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 942, in _schedule_instances\n    return_alternates=return_alternates)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations\n    instance_uuids, return_objects, return_alternates)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations\n    return cctxt.call(ctxt, \'select_destinations\', **msg_args)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 192, in call\n    retry=self.retry, transport_options=self.transport_options)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send\n    transport_options=transport_options)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 691, in send\n    transport_options=transport_options)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send\n    raise result\nnova.exception_Remote.NoValidHost_Remote: No valid host was found. \nTraceback (most recent call last):\n\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner\n    return func(*args, **kwargs)\n\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/manager.py", line 209, in select_destinations\n    raise exception.NoValidHost(reason="")\n\nnova.exception.NoValidHost: No valid host was found. \n\n'}
2025-12-22 03:49:31,904 - INFO - Deleting VN ctest-cnz_vn-37819795 
2025-12-22 03:49:31,967 - DEBUG - VN d243e9f9-547b-4e6f-be16-353855ce9677 still in use: Unable to complete operation on network d243e9f9-547b-4e6f-be16-353855ce9677. There are one or more ports still in use on the network.
Neutron server returns request_ids: ['req-8c32645a-b6bb-44e0-a623-ac420b8b8802']
2025-12-22 03:49:31,968 - WARNING - Deleting VN ctest-cnz_vn-37819795 failed..Will retry
2025-12-22 03:49:34,030 - DEBUG - VN d243e9f9-547b-4e6f-be16-353855ce9677 still in use: Unable to complete operation on network d243e9f9-547b-4e6f-be16-353855ce9677. There are one or more ports still in use on the network.
Neutron server returns request_ids: ['req-76ba1461-550a-49b0-901d-b84ef250c228']
2025-12-22 03:49:34,030 - WARNING - Deleting VN ctest-cnz_vn-37819795 failed..Will retry
2025-12-22 03:49:36,105 - DEBUG - VN d243e9f9-547b-4e6f-be16-353855ce9677 still in use: Unable to complete operation on network d243e9f9-547b-4e6f-be16-353855ce9677. There are one or more ports still in use on the network.
Neutron server returns request_ids: ['req-ddc88329-ce9a-40a6-82f7-d566063fd824']
2025-12-22 03:49:36,105 - WARNING - Deleting VN ctest-cnz_vn-37819795 failed..Will retry
2025-12-22 03:49:38,153 - DEBUG - VN d243e9f9-547b-4e6f-be16-353855ce9677 still in use: Unable to complete operation on network d243e9f9-547b-4e6f-be16-353855ce9677. There are one or more ports still in use on the network.
Neutron server returns request_ids: ['req-3cde0993-c750-45bf-bdb9-cf7c8af2aedd']
2025-12-22 03:49:38,153 - WARNING - Deleting VN ctest-cnz_vn-37819795 failed..Will retry
2025-12-22 03:49:40,205 - DEBUG - VN d243e9f9-547b-4e6f-be16-353855ce9677 still in use: Unable to complete operation on network d243e9f9-547b-4e6f-be16-353855ce9677. There are one or more ports still in use on the network.
Neutron server returns request_ids: ['req-0fea1bc1-073c-4fc4-9492-25d886acc7aa']
2025-12-22 03:49:40,206 - WARNING - Deleting VN ctest-cnz_vn-37819795 failed..Will retry
2025-12-22 03:49:42,255 - DEBUG - VN d243e9f9-547b-4e6f-be16-353855ce9677 still in use: Unable to complete operation on network d243e9f9-547b-4e6f-be16-353855ce9677. There are one or more ports still in use on the network.
Neutron server returns request_ids: ['req-854bf48f-85db-4a17-8693-0a88ad6056ca']
2025-12-22 03:49:42,255 - WARNING - Deleting VN ctest-cnz_vn-37819795 failed..Will retry
2025-12-22 03:49:44,300 - DEBUG - VN d243e9f9-547b-4e6f-be16-353855ce9677 still in use: Unable to complete operation on network d243e9f9-547b-4e6f-be16-353855ce9677. There are one or more ports still in use on the network.
Neutron server returns request_ids: ['req-30bae467-c280-4daa-9709-f7682e323f1f']
2025-12-22 03:49:44,300 - WARNING - Deleting VN ctest-cnz_vn-37819795 failed..Will retry
2025-12-22 03:49:46,357 - DEBUG - VN d243e9f9-547b-4e6f-be16-353855ce9677 still in use: Unable to complete operation on network d243e9f9-547b-4e6f-be16-353855ce9677. There are one or more ports still in use on the network.
Neutron server returns request_ids: ['req-98c64c5a-2746-4ca4-868e-114e34e44e72']
2025-12-22 03:49:46,357 - WARNING - Deleting VN ctest-cnz_vn-37819795 failed..Will retry
2025-12-22 03:49:48,409 - DEBUG - VN d243e9f9-547b-4e6f-be16-353855ce9677 still in use: Unable to complete operation on network d243e9f9-547b-4e6f-be16-353855ce9677. There are one or more ports still in use on the network.
Neutron server returns request_ids: ['req-8593f778-058b-492a-ab6e-15cad845ee1b']
2025-12-22 03:49:48,409 - WARNING - Deleting VN ctest-cnz_vn-37819795 failed..Will retry
2025-12-22 03:49:50,449 - DEBUG - VN d243e9f9-547b-4e6f-be16-353855ce9677 still in use: Unable to complete operation on network d243e9f9-547b-4e6f-be16-353855ce9677. There are one or more ports still in use on the network.
Neutron server returns request_ids: ['req-00b3e432-c07e-4d3f-a3a6-298d026a7dae']
2025-12-22 03:49:50,450 - WARNING - Deleting VN ctest-cnz_vn-37819795 failed..Will retry
2025-12-22 03:49:52,489 - DEBUG - VN d243e9f9-547b-4e6f-be16-353855ce9677 still in use: Unable to complete operation on network d243e9f9-547b-4e6f-be16-353855ce9677. There are one or more ports still in use on the network.
Neutron server returns request_ids: ['req-a7eab6ca-968e-4e49-a3d7-96c81717f700']
2025-12-22 03:49:52,490 - WARNING - Deleting VN ctest-cnz_vn-37819795 failed..Will retry
2025-12-22 03:49:54,532 - DEBUG - VN d243e9f9-547b-4e6f-be16-353855ce9677 still in use: Unable to complete operation on network d243e9f9-547b-4e6f-be16-353855ce9677. There are one or more ports still in use on the network.
Neutron server returns request_ids: ['req-18cbc6db-f9be-4d65-ac16-b23e17e55333']
2025-12-22 03:49:54,532 - WARNING - Deleting VN ctest-cnz_vn-37819795 failed..Will retry
2025-12-22 03:49:56,593 - DEBUG - VN d243e9f9-547b-4e6f-be16-353855ce9677 still in use: Unable to complete operation on network d243e9f9-547b-4e6f-be16-353855ce9677. There are one or more ports still in use on the network.
Neutron server returns request_ids: ['req-bb72fcf6-d88a-40ee-8ebc-1a08e99c2755']
2025-12-22 03:49:56,593 - WARNING - Deleting VN ctest-cnz_vn-37819795 failed..Will retry
2025-12-22 03:49:58,636 - DEBUG - VN d243e9f9-547b-4e6f-be16-353855ce9677 still in use: Unable to complete operation on network d243e9f9-547b-4e6f-be16-353855ce9677. There are one or more ports still in use on the network.
Neutron server returns request_ids: ['req-a70505ea-0662-4991-a7a0-530d52b630f7']
2025-12-22 03:49:58,636 - WARNING - Deleting VN ctest-cnz_vn-37819795 failed..Will retry
2025-12-22 03:50:00,680 - DEBUG - VN d243e9f9-547b-4e6f-be16-353855ce9677 still in use: Unable to complete operation on network d243e9f9-547b-4e6f-be16-353855ce9677. There are one or more ports still in use on the network.
Neutron server returns request_ids: ['req-a0e19242-f4e4-454e-8a54-675b5e7610e3']
2025-12-22 03:50:00,681 - WARNING - Deleting VN ctest-cnz_vn-37819795 failed..Will retry
2025-12-22 03:50:02,721 - DEBUG - VN d243e9f9-547b-4e6f-be16-353855ce9677 still in use: Unable to complete operation on network d243e9f9-547b-4e6f-be16-353855ce9677. There are one or more ports still in use on the network.
Neutron server returns request_ids: ['req-6628b657-d039-41c5-b652-9be5b8a24311']
2025-12-22 03:50:02,721 - WARNING - Deleting VN ctest-cnz_vn-37819795 failed..Will retry
2025-12-22 03:50:04,901 - DEBUG - Response for deleting network ()
2025-12-22 03:50:06,191 - ERROR - AssertionError
Python 3.9.21: /usr/bin/python3
Mon Dec 22 03:49:31 2025

A problem occurred in a Python script.  Here is the sequence of
function calls leading up to the error, in the order they occurred.

 /contrail-test/tcutils/wrappers.py in wrapper(self=<serial_scripts.bgpaas.test_bgpaas_zone.TestBGPa..._bgp_control_node_zone[sanity] id=0x7fcc2f15b5e0>, *args=(), **kwargs={})
   78             log.info('Initial checks done. Running the testcase now')
   79             log.info('')
   80             result = function(self, *args, **kwargs)
   81             if self.inputs.upgrade:
   82                 pid = os.getpid()
result = None
function = <function TestBGPaasZone.test_bgp_control_node_zone>
self = <serial_scripts.bgpaas.test_bgpaas_zone.TestBGPa..._bgp_control_node_zone[sanity] id=0x7fcc2f15b5e0>
args = ()
kwargs = {}

 /contrail-test/serial_scripts/bgpaas/test_bgpaas_zone.py in test_bgp_control_node_zone(self=<serial_scripts.bgpaas.test_bgpaas_zone.TestBGPa..._bgp_control_node_zone[sanity] id=0x7fcc2f15b5e0>)
   57         cnt = 0
   58         self.logger.info('executing bgp_control_zone test')
   59         assert self.basic_control_node_zone_setup()
   60         # update control node zone with different bgp routers
   61         self.update_control_node_zones(self.cnzs)
self = <serial_scripts.bgpaas.test_bgpaas_zone.TestBGPa..._bgp_control_node_zone[sanity] id=0x7fcc2f15b5e0>
self.basic_control_node_zone_setup = <bound method TestBGPaasZone.basic_control_node_...bgp_control_node_zone[sanity] id=0x7fcc2f15b5e0>>

 /contrail-test/serial_scripts/bgpaas/test_bgpaas_zone.py in basic_control_node_zone_setup(self=<serial_scripts.bgpaas.test_bgpaas_zone.TestBGPa..._bgp_control_node_zone[sanity] id=0x7fcc2f15b5e0>, ctrl_zone=None)
   31             self.vms.append(self.create_vm(vn_fixture=self.vn, image_name='ubuntu-bird'))
   32         self.client_vm = self.create_vm(vn_fixture=self.vn, image_name='ubuntu-bird')
   33         self.check_vms_booted(self.vms + [self.client_vm])
   34         for bgp_name in self.inputs.bgp_names:
   35             self.cnzs  +=  self.create_control_node_zones("test-zone",[bgp_name])
self = <serial_scripts.bgpaas.test_bgpaas_zone.TestBGPa..._bgp_control_node_zone[sanity] id=0x7fcc2f15b5e0>
self.check_vms_booted = <bound method GenericTestBase.check_vms_booted o...scripts.bgpaas.test_bgpaas_zone.TestBGPaasZone'>>
self.vms = [<VMFixture: ctest-TestBGPaasZone-53547593-55707067>, <VMFixture: ctest-TestBGPaasZone-53547593-94137805>]
self.client_vm = <VMFixture: ctest-TestBGPaasZone-53547593-47918528>

 /contrail-test/common/base.py in check_vms_booted(cls=<class 'serial_scripts.bgpaas.test_bgpaas_zone.TestBGPaasZone'>, vms_list=[<VMFixture: ctest-TestBGPaasZone-53547593-55707067>, <VMFixture: ctest-TestBGPaasZone-53547593-94137805>, <VMFixture: ctest-TestBGPaasZone-53547593-47918528>], do_assert=True)
  719         if failed and do_assert:
  720             for vm_fixture in vms_list:
  721                 vm_fixture.cleanUp()
  722             assert False, 'One or more vm-boots failed. Check logs'
  723         if failed:
vm_fixture = <VMFixture: ctest-TestBGPaasZone-53547593-55707067>
vm_fixture.cleanUp = <bound method VMFixture.cleanUp of <VMFixture: ctest-TestBGPaasZone-53547593-55707067>>

 /contrail-test/fixtures/vm_test.py in cleanUp(self=<VMFixture: ctest-TestBGPaasZone-53547593-55707067>)
 2050 
 2051     def cleanUp(self):
 2052         self.delete()
 2053         super(VMFixture, self).cleanUp()
 2054 
self = <VMFixture: ctest-TestBGPaasZone-53547593-55707067>
self.delete = <bound method VMFixture.delete of <VMFixture: ctest-TestBGPaasZone-53547593-55707067>>

 /contrail-test/fixtures/vm_test.py in delete(self=<VMFixture: ctest-TestBGPaasZone-53547593-55707067>, verify=False, force=False)
 2080                     self.webui.delete_vm(self)
 2081                 else:
 2082                     self.orch.delete_vm(vm_obj, force=force)
 2083                     self.vm_objs.remove(vm_obj)
 2084                 if self.inputs.ns_agilio_vrouter_data:
self = <VMFixture: ctest-TestBGPaasZone-53547593-55707067>
self.orch = <openstack.OpenstackOrchestrator object>
self.orch.delete_vm = <bound method OpenstackOrchestrator.delete_vm of <openstack.OpenstackOrchestrator object>>
vm_obj = <Server: ctest-TestBGPaasZone-53547593-55707067>
force = False

 /contrail-test/fixtures/openstack.py in delete_vm(self=<openstack.OpenstackOrchestrator object>, vm_obj=<Server: ctest-TestBGPaasZone-53547593-55707067>, **kwargs={'force': False})
  119 
  120    def delete_vm(self, vm_obj, **kwargs):
  121        return self.nova_h.delete_vm(vm_obj, **kwargs)
  122 
  123    def is_vm_deleted(self, vm_obj, **kwargs):
self = <openstack.OpenstackOrchestrator object>
self.nova_h = <nova_test.NovaHelper object>
self.nova_h.delete_vm = <bound method NovaHelper.delete_vm of <nova_test.NovaHelper object>>
vm_obj = <Server: ctest-TestBGPaasZone-53547593-55707067>
kwargs = {'force': False}

 /contrail-test/fixtures/nova_test.py in delete_vm(self=<nova_test.NovaHelper object>, vm_obj=<Server: ctest-TestBGPaasZone-53547593-55707067>, force=False)
  763     def delete_vm(self, vm_obj, force=False):
  764         if not force:
  765             self.get_nova_host_of_vm(vm_obj)
  766         vm_obj.delete()
  767     # end _delete_vm
self = <nova_test.NovaHelper object>
self.get_nova_host_of_vm = <bound method NovaHelper.get_nova_host_of_vm of <nova_test.NovaHelper object>>
vm_obj = <Server: ctest-TestBGPaasZone-53547593-55707067>

 /contrail-test/fixtures/nova_test.py in get_nova_host_of_vm(self=<nova_test.NovaHelper object>, vm_obj=<Server: ctest-TestBGPaasZone-53547593-55707067>)
  734                     self.logger.error('VM %s has failed to come up' % vm_obj.name)
  735                     self.logger.error('Fault seen in nova show <vm-uuid> is:  %s' % vm_obj.__dict__['fault'])
  736                     assert False, 'Fault seen in nova show %s is:  %s' % (
  737                         vm_obj.id, vm_obj.__dict__['fault'])
  738                 else:
vm_obj = <Server: ctest-TestBGPaasZone-53547593-55707067>
vm_obj.id = 'a1185cc9-10e1-4898-9232-3b5df1515e05'
vm_obj.__dict__ = {'OS-DCF:diskConfig': 'MANUAL', 'OS-EXT-AZ:availability_zone': 'nova', 'OS-EXT-SRV-ATTR:host': None, 'OS-EXT-SRV-ATTR:hypervisor_hostname': None, 'OS-EXT-SRV-ATTR:instance_name': 'instance-00000004', 'OS-EXT-STS:power_state': 0, 'OS-EXT-STS:task_state': None, 'OS-EXT-STS:vm_state': 'error', 'OS-SRV-USG:launched_at': None, 'OS-SRV-USG:terminated_at': None, ...}
AssertionError: Fault seen in nova show a1185cc9-10e1-4898-9232-3b5df1515e05 is:  {'code': 500, 'created': '2025-12-22T03:49:28Z', 'message': 'No valid host was found. ', 'details': 'Traceback (most recent call last):\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 1581, in schedule_and_build_instances\n    instance_uuids, return_alternates=True)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 942, in _schedule_instances\n    return_alternates=return_alternates)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations\n    instance_uuids, return_objects, return_alternates)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations\n    return cctxt.call(ctxt, \'select_destinations\', **msg_args)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 192, in call\n    retry=self.retry, transport_options=self.transport_options)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send\n    transport_options=transport_options)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 691, in send\n    transport_options=transport_options)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send\n    raise result\nnova.exception_Remote.NoValidHost_Remote: No valid host was found. \nTraceback (most recent call last):\n\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner\n    return func(*args, **kwargs)\n\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/manager.py", line 209, in select_destinations\n    raise exception.NoValidHost(reason="")\n\nnova.exception.NoValidHost: No valid host was found. \n\n'}
    __cause__ = None
    __class__ = <class 'AssertionError'>
    __context__ = None
    __delattr__ = <method-wrapper '__delattr__' of AssertionError object>
    __dict__ = {}
    __dir__ = <built-in method __dir__ of AssertionError object>
    __doc__ = 'Assertion failed.'
    __eq__ = <method-wrapper '__eq__' of AssertionError object>
    __format__ = <built-in method __format__ of AssertionError object>
    __ge__ = <method-wrapper '__ge__' of AssertionError object>
    __getattribute__ = <method-wrapper '__getattribute__' of AssertionError object>
    __gt__ = <method-wrapper '__gt__' of AssertionError object>
    __hash__ = <method-wrapper '__hash__' of AssertionError object>
    __init__ = <method-wrapper '__init__' of AssertionError object>
    __init_subclass__ = <built-in method __init_subclass__ of type object>
    __le__ = <method-wrapper '__le__' of AssertionError object>
    __lt__ = <method-wrapper '__lt__' of AssertionError object>
    __ne__ = <method-wrapper '__ne__' of AssertionError object>
    __new__ = <built-in method __new__ of type object>
    __reduce__ = <built-in method __reduce__ of AssertionError object>
    __reduce_ex__ = <built-in method __reduce_ex__ of AssertionError object>
    __repr__ = <method-wrapper '__repr__' of AssertionError object>
    __setattr__ = <method-wrapper '__setattr__' of AssertionError object>
    __setstate__ = <built-in method __setstate__ of AssertionError object>
    __sizeof__ = <built-in method __sizeof__ of AssertionError object>
    __str__ = <method-wrapper '__str__' of AssertionError object>
    __subclasshook__ = <built-in method __subclasshook__ of type object>
    __suppress_context__ = False
    __traceback__ = <traceback object>
    args = (r"Fault seen in nova show a1185cc9-10e1-4898-9232-...tion.NoValidHost: No valid host was found. \n\n'}",)
    with_traceback = <built-in method with_traceback of AssertionError object>

The above is a description of an error in a Python program.  Here is
the original traceback:

Traceback (most recent call last):
  File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper
    result = function(self, *args, **kwargs)
  File "/contrail-test/serial_scripts/bgpaas/test_bgpaas_zone.py", line 59, in test_bgp_control_node_zone
    assert self.basic_control_node_zone_setup()
  File "/contrail-test/serial_scripts/bgpaas/test_bgpaas_zone.py", line 33, in basic_control_node_zone_setup
    self.check_vms_booted(self.vms + [self.client_vm])
  File "/contrail-test/common/base.py", line 721, in check_vms_booted
    vm_fixture.cleanUp()
  File "/contrail-test/fixtures/vm_test.py", line 2052, in cleanUp
    self.delete()
  File "/contrail-test/fixtures/vm_test.py", line 2082, in delete
    self.orch.delete_vm(vm_obj, force=force)
  File "/contrail-test/fixtures/openstack.py", line 121, in delete_vm
    return self.nova_h.delete_vm(vm_obj, **kwargs)
  File "/contrail-test/fixtures/nova_test.py", line 765, in delete_vm
    self.get_nova_host_of_vm(vm_obj)
  File "/contrail-test/fixtures/nova_test.py", line 736, in get_nova_host_of_vm
    assert False, 'Fault seen in nova show %s is:  %s' % (
AssertionError: Fault seen in nova show a1185cc9-10e1-4898-9232-3b5df1515e05 is:  {'code': 500, 'created': '2025-12-22T03:49:28Z', 'message': 'No valid host was found. ', 'details': 'Traceback (most recent call last):\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 1581, in schedule_and_build_instances\n    instance_uuids, return_alternates=True)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 942, in _schedule_instances\n    return_alternates=return_alternates)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations\n    instance_uuids, return_objects, return_alternates)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations\n    return cctxt.call(ctxt, \'select_destinations\', **msg_args)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 192, in call\n    retry=self.retry, transport_options=self.transport_options)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send\n    transport_options=transport_options)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 691, in send\n    transport_options=transport_options)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send\n    raise result\nnova.exception_Remote.NoValidHost_Remote: No valid host was found. \nTraceback (most recent call last):\n\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner\n    return func(*args, **kwargs)\n\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/manager.py", line 209, in select_destinations\n    raise exception.NoValidHost(reason="")\n\nnova.exception.NoValidHost: No valid host was found. \n\n'}



2025-12-22 03:50:06,192 - DEBUG - Skipping xmpp flap check
2025-12-22 03:50:06,192 - INFO - 
2025-12-22 03:50:06,192 - INFO - END TEST : test_bgp_control_node_zone : FAILED[0:00:42]
2025-12-22 03:50:06,192 - INFO - --------------------------------------------------------------------------------
2025-12-22 03:50:06,195 - INFO - ================================================================================
2025-12-22 03:50:06,195 - INFO - STARTING TEST    : test_bgp_control_node_zones_from_single_vnf
2025-12-22 03:50:07,463 - DEBUG - Skipping xmpp flap check
2025-12-22 03:50:07,463 - INFO - Initial checks done. Running the testcase now
2025-12-22 03:50:07,463 - INFO - 
2025-12-22 03:50:07,463 - INFO - executing bgp_control_zone agent restart test
2025-12-22 03:50:07,737 - DEBUG - Response for create_network : {'network': {'id': 'd54e3a68-e39a-4557-8bae-7d4d6c86f0e3', 'name': 'ctest-cnz_vn-43934911', 'tenant_id': '7b526cb0ef57466c84c25bff8d0bce62', 'project_id': '7b526cb0ef57466c84c25bff8d0bce62', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestBGPaasZone-53547593', 'ctest-cnz_vn-43934911'], 'port_security_enabled': True, 'description': ''}}
2025-12-22 03:50:07,985 - DEBUG - Response for create_subnet : {'subnet': {'id': 'c3f08684-280b-411c-9710-ef1bd4fa073f', 'name': '', 'tenant_id': '7b526cb0ef57466c84c25bff8d0bce62', 'network_id': 'd54e3a68-e39a-4557-8bae-7d4d6c86f0e3', 'ip_version': 4, 'cidr': '121.196.5.0/26', 'allocation_pools': [{'start': '121.196.5.2', 'end': '121.196.5.62'}], 'gateway_ip': '121.196.5.1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '121.196.5.2', 'tags': [], 'project_id': '7b526cb0ef57466c84c25bff8d0bce62'}}
2025-12-22 03:50:08,017 - INFO - Created VN ctest-cnz_vn-43934911
2025-12-22 03:50:08,033 - DEBUG - VN ctest-cnz_vn-43934911 UUID is d54e3a68-e39a-4557-8bae-7d4d6c86f0e3
2025-12-22 03:50:09,332 - INFO - VM ([<Server: ctest-TestBGPaasZone-53547593-94137530>]) created on node: (an-jenkins-deploy-platform-ansible-os-4918-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4918-2)
2025-12-22 03:50:10,758 - INFO - VM ([<Server: ctest-TestBGPaasZone-53547593-04273383>]) created on node: (an-jenkins-deploy-platform-ansible-os-4918-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4918-1)
2025-12-22 03:50:12,081 - INFO - VM ([<Server: ctest-TestBGPaasZone-53547593-45766689>]) created on node: (an-jenkins-deploy-platform-ansible-os-4918-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4918-2)
2025-12-22 03:50:12,081 - INFO - Waiting for VM ctest-TestBGPaasZone-53547593-94137530 to be up..
2025-12-22 03:50:12,175 - DEBUG - VM <Server: ctest-TestBGPaasZone-53547593-94137530> is in ACTIVE state now
2025-12-22 03:50:12,176 - INFO - VM name : ctest-TestBGPaasZone-53547593-94137530
2025-12-22 03:50:12,266 - DEBUG - VM ctest-TestBGPaasZone-53547593-94137530 ID is 0c153c7d-9bc7-47d6-8d44-9504ff2d704e
2025-12-22 03:50:12,266 - DEBUG - VM ctest-TestBGPaasZone-53547593-94137530 launched on Node an-jenkins-deploy-platform-ansible-os-4918-2
2025-12-22 03:50:12,378 - DEBUG - Requesting: http://10.0.0.65:8082/virtual-machine/0c153c7d-9bc7-47d6-8d44-9504ff2d704e
2025-12-22 03:50:12,771 - DEBUG - Requesting: http://10.0.0.65:8082/virtual-machine/0c153c7d-9bc7-47d6-8d44-9504ff2d704e
2025-12-22 03:50:12,812 - DEBUG - Requesting: http://10.0.0.65:8082/virtual-machine-interface/d8d87e68-7ad3-492d-abc2-87f02ef223f0
2025-12-22 03:50:16,021 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1025ms')
2025-12-22 03:50:16,021 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-53547593-94137530 failed!
2025-12-22 03:50:16,037 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-53547593:ctest-cnz_vn-43934911 is 121.196.5.1 and allocation pool is NOT set
2025-12-22 03:50:20,113 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1011ms')
2025-12-22 03:50:20,113 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-53547593-94137530 failed!
2025-12-22 03:50:20,131 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-53547593:ctest-cnz_vn-43934911 is 121.196.5.1 and allocation pool is NOT set
2025-12-22 03:50:24,218 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1009ms')
2025-12-22 03:50:24,218 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-53547593-94137530 failed!
2025-12-22 03:50:24,234 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-53547593:ctest-cnz_vn-43934911 is 121.196.5.1 and allocation pool is NOT set
2025-12-22 03:50:28,313 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1012ms')
2025-12-22 03:50:28,314 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-53547593-94137530 failed!
2025-12-22 03:50:28,335 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-53547593:ctest-cnz_vn-43934911 is 121.196.5.1 and allocation pool is NOT set
2025-12-22 03:50:32,410 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1010ms')
2025-12-22 03:50:32,411 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-53547593-94137530 failed!
2025-12-22 03:50:32,425 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-53547593:ctest-cnz_vn-43934911 is 121.196.5.1 and allocation pool is NOT set
2025-12-22 03:50:36,496 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1008ms')
2025-12-22 03:50:36,496 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-53547593-94137530 failed!
2025-12-22 03:50:36,514 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-53547593:ctest-cnz_vn-43934911 is 121.196.5.1 and allocation pool is NOT set
2025-12-22 03:50:40,598 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1020ms')
2025-12-22 03:50:40,599 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-53547593-94137530 failed!
2025-12-22 03:50:40,614 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-53547593:ctest-cnz_vn-43934911 is 121.196.5.1 and allocation pool is NOT set
2025-12-22 03:50:44,695 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1017ms')
2025-12-22 03:50:44,695 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-53547593-94137530 failed!
2025-12-22 03:50:44,710 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-53547593:ctest-cnz_vn-43934911 is 121.196.5.1 and allocation pool is NOT set
2025-12-22 03:50:48,794 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1008ms')
2025-12-22 03:50:48,794 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-53547593-94137530 failed!
2025-12-22 03:50:48,813 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-53547593:ctest-cnz_vn-43934911 is 121.196.5.1 and allocation pool is NOT set
2025-12-22 03:50:52,891 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1014ms')
2025-12-22 03:50:52,891 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-53547593-94137530 failed!
2025-12-22 03:50:52,908 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-53547593:ctest-cnz_vn-43934911 is 121.196.5.1 and allocation pool is NOT set
2025-12-22 03:50:56,986 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1014ms')
2025-12-22 03:50:56,986 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-53547593-94137530 failed!
2025-12-22 03:50:57,003 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-53547593:ctest-cnz_vn-43934911 is 121.196.5.1 and allocation pool is NOT set
2025-12-22 03:51:01,077 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1010ms')
2025-12-22 03:51:01,077 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-53547593-94137530 failed!
2025-12-22 03:51:01,095 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-53547593:ctest-cnz_vn-43934911 is 121.196.5.1 and allocation pool is NOT set
2025-12-22 03:51:05,172 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1013ms')
2025-12-22 03:51:05,172 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-53547593-94137530 failed!
2025-12-22 03:51:05,190 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-53547593:ctest-cnz_vn-43934911 is 121.196.5.1 and allocation pool is NOT set
2025-12-22 03:51:09,268 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1016ms')
2025-12-22 03:51:09,268 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-53547593-94137530 failed!
2025-12-22 03:51:09,288 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-53547593:ctest-cnz_vn-43934911 is 121.196.5.1 and allocation pool is NOT set
2025-12-22 03:51:13,361 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1010ms')
2025-12-22 03:51:13,361 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-53547593-94137530 failed!
2025-12-22 03:51:13,379 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-53547593:ctest-cnz_vn-43934911 is 121.196.5.1 and allocation pool is NOT set
2025-12-22 03:51:17,463 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1020ms')
2025-12-22 03:51:17,464 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-53547593-94137530 failed!
2025-12-22 03:51:17,479 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-53547593:ctest-cnz_vn-43934911 is 121.196.5.1 and allocation pool is NOT set
2025-12-22 03:51:21,557 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1014ms')
2025-12-22 03:51:21,557 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-53547593-94137530 failed!
2025-12-22 03:51:21,571 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-53547593:ctest-cnz_vn-43934911 is 121.196.5.1 and allocation pool is NOT set
2025-12-22 03:51:25,650 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1020ms')
2025-12-22 03:51:25,650 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-53547593-94137530 failed!
2025-12-22 03:51:25,666 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-53547593:ctest-cnz_vn-43934911 is 121.196.5.1 and allocation pool is NOT set
2025-12-22 03:51:29,755 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1023ms')
2025-12-22 03:51:29,755 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-53547593-94137530 failed!
2025-12-22 03:51:29,771 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-53547593:ctest-cnz_vn-43934911 is 121.196.5.1 and allocation pool is NOT set
2025-12-22 03:51:33,843 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1007ms')
2025-12-22 03:51:33,844 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-53547593-94137530 failed!
2025-12-22 03:51:33,858 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-53547593:ctest-cnz_vn-43934911 is 121.196.5.1 and allocation pool is NOT set
2025-12-22 03:51:37,942 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1021ms')
2025-12-22 03:51:37,942 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-53547593-94137530 failed!
2025-12-22 03:51:37,962 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-53547593:ctest-cnz_vn-43934911 is 121.196.5.1 and allocation pool is NOT set
2025-12-22 03:51:42,037 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1013ms')
2025-12-22 03:51:42,037 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-53547593-94137530 failed!
2025-12-22 03:51:42,055 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-53547593:ctest-cnz_vn-43934911 is 121.196.5.1 and allocation pool is NOT set
2025-12-22 03:51:46,135 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1013ms')
2025-12-22 03:51:46,135 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-53547593-94137530 failed!
2025-12-22 03:51:46,151 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-53547593:ctest-cnz_vn-43934911 is 121.196.5.1 and allocation pool is NOT set
2025-12-22 03:51:50,233 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1003ms')
2025-12-22 03:51:50,233 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-53547593-94137530 failed!
2025-12-22 03:51:50,249 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-53547593:ctest-cnz_vn-43934911 is 121.196.5.1 and allocation pool is NOT set
2025-12-22 03:51:52,318 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=2.86 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=2.72 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 2.719/2.789/2.860/0.070 ms')
2025-12-22 03:51:52,318 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-53547593-94137530 passed
2025-12-22 03:51:52,392 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-12-22 03:51:52,392 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-53547593-94137530, IP 121.196.5.3, Port 22
2025-12-22 03:51:52,457 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused
/bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''}
2025-12-22 03:51:52,571 - DEBUG - VM ctest-TestBGPaasZone-53547593-94137530 is NOT ready for SSH connections, VM status: ACTIVE
2025-12-22 03:51:57,572 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-12-22 03:51:57,572 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-53547593-94137530, IP 121.196.5.3, Port 22
2025-12-22 03:51:57,641 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused
/bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''}
2025-12-22 03:51:57,727 - DEBUG - VM ctest-TestBGPaasZone-53547593-94137530 is NOT ready for SSH connections, VM status: ACTIVE
2025-12-22 03:52:02,727 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-12-22 03:52:02,727 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-53547593-94137530, IP 121.196.5.3, Port 22
2025-12-22 03:52:02,796 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused
/bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''}
2025-12-22 03:52:02,903 - DEBUG - VM ctest-TestBGPaasZone-53547593-94137530 is NOT ready for SSH connections, VM status: ACTIVE
2025-12-22 03:52:07,904 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-12-22 03:52:07,904 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-53547593-94137530, IP 121.196.5.3, Port 22
2025-12-22 03:52:07,977 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused
/bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''}
2025-12-22 03:52:08,061 - DEBUG - VM ctest-TestBGPaasZone-53547593-94137530 is NOT ready for SSH connections, VM status: ACTIVE
2025-12-22 03:52:13,062 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-12-22 03:52:13,062 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-53547593-94137530, IP 121.196.5.3, Port 22
2025-12-22 03:52:13,133 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused
/bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''}
2025-12-22 03:52:13,231 - DEBUG - VM ctest-TestBGPaasZone-53547593-94137530 is NOT ready for SSH connections, VM status: ACTIVE
2025-12-22 03:52:18,232 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-12-22 03:52:18,232 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-53547593-94137530, IP 121.196.5.3, Port 22
2025-12-22 03:52:18,301 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused
/bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''}
2025-12-22 03:52:18,384 - DEBUG - VM ctest-TestBGPaasZone-53547593-94137530 is NOT ready for SSH connections, VM status: ACTIVE
2025-12-22 03:52:23,384 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-12-22 03:52:23,384 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-53547593-94137530, IP 121.196.5.3, Port 22
2025-12-22 03:52:23,454 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused
/bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''}
2025-12-22 03:52:23,692 - DEBUG - VM ctest-TestBGPaasZone-53547593-94137530 is NOT ready for SSH connections, VM status: ACTIVE
2025-12-22 03:52:28,692 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-12-22 03:52:28,693 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-53547593-94137530, IP 121.196.5.3, Port 22
2025-12-22 03:52:28,762 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused
/bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''}
2025-12-22 03:52:28,846 - DEBUG - VM ctest-TestBGPaasZone-53547593-94137530 is NOT ready for SSH connections, VM status: ACTIVE
2025-12-22 03:52:33,847 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-12-22 03:52:33,848 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-53547593-94137530, IP 121.196.5.3, Port 22
2025-12-22 03:52:33,913 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused
/bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''}
2025-12-22 03:52:33,992 - DEBUG - VM ctest-TestBGPaasZone-53547593-94137530 is NOT ready for SSH connections, VM status: ACTIVE
2025-12-22 03:52:38,993 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-12-22 03:52:38,993 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-53547593-94137530, IP 121.196.5.3, Port 22
2025-12-22 03:52:39,066 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused
/bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''}
2025-12-22 03:52:39,153 - DEBUG - VM ctest-TestBGPaasZone-53547593-94137530 is NOT ready for SSH connections, VM status: ACTIVE
2025-12-22 03:52:44,154 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-12-22 03:52:44,154 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-53547593-94137530, IP 121.196.5.3, Port 22
2025-12-22 03:52:44,221 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused
/bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''}
2025-12-22 03:52:44,304 - DEBUG - VM ctest-TestBGPaasZone-53547593-94137530 is NOT ready for SSH connections, VM status: ACTIVE
2025-12-22 03:52:49,305 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-12-22 03:52:49,305 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-53547593-94137530, IP 121.196.5.3, Port 22
2025-12-22 03:52:49,378 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused
/bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''}
2025-12-22 03:52:49,466 - DEBUG - VM ctest-TestBGPaasZone-53547593-94137530 is NOT ready for SSH connections, VM status: ACTIVE
2025-12-22 03:52:54,467 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-12-22 03:52:54,467 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-53547593-94137530, IP 121.196.5.3, Port 22
2025-12-22 03:52:54,644 - DEBUG - VM ctest-TestBGPaasZone-53547593-94137530 is ready for SSH connections
2025-12-22 03:52:54,645 - INFO - Waiting for VM ctest-TestBGPaasZone-53547593-04273383 to be up..
2025-12-22 03:52:54,715 - DEBUG - VM <Server: ctest-TestBGPaasZone-53547593-04273383> is in ERROR state now
2025-12-22 03:52:54,715 - ERROR - VM in error state.
2025-12-22 03:52:54,715 - ERROR - VM is not in ACTIVE state
2025-12-22 03:52:54,836 - ERROR - Fatal Nova Exception while getting VM detail
2025-12-22 03:52:54,836 - DEBUG - None
2025-12-22 03:52:54,836 - ERROR - VM ctest-TestBGPaasZone-53547593-04273383 has not booted
2025-12-22 03:52:54,836 - INFO - Deleting VM ctest-TestBGPaasZone-53547593-94137530
2025-12-22 03:52:54,912 - INFO - Skip interface_detach for VM ctest-TestBGPaasZone-53547593-04273383 in state ERROR
2025-12-22 03:52:54,912 - INFO - Deleting VM ctest-TestBGPaasZone-53547593-04273383
2025-12-22 03:52:54,913 - ERROR - VM ctest-TestBGPaasZone-53547593-04273383 has failed to come up
2025-12-22 03:52:54,913 - ERROR - Fault seen in nova show <vm-uuid> is:  {'code': 500, 'created': '2025-12-22T03:50:10Z', 'message': 'No valid host was found. ', 'details': 'Traceback (most recent call last):\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 1581, in schedule_and_build_instances\n    instance_uuids, return_alternates=True)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 942, in _schedule_instances\n    return_alternates=return_alternates)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations\n    instance_uuids, return_objects, return_alternates)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations\n    return cctxt.call(ctxt, \'select_destinations\', **msg_args)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 192, in call\n    retry=self.retry, transport_options=self.transport_options)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send\n    transport_options=transport_options)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 691, in send\n    transport_options=transport_options)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send\n    raise result\nnova.exception_Remote.NoValidHost_Remote: No valid host was found. \nTraceback (most recent call last):\n\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner\n    return func(*args, **kwargs)\n\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/manager.py", line 209, in select_destinations\n    raise exception.NoValidHost(reason="")\n\nnova.exception.NoValidHost: No valid host was found. \n\n'}
2025-12-22 03:52:54,914 - INFO - Skip interface_detach for VM ctest-TestBGPaasZone-53547593-45766689 in state BUILD
2025-12-22 03:52:54,915 - INFO - Deleting VM ctest-TestBGPaasZone-53547593-45766689
2025-12-22 03:52:54,915 - ERROR - VM ctest-TestBGPaasZone-53547593-45766689 has failed to come up
2025-12-22 03:52:54,915 - ERROR - Nova failed to get host of the VM
2025-12-22 03:52:54,915 - ERROR - VM ctest-TestBGPaasZone-53547593-45766689 has failed to come up
2025-12-22 03:52:54,915 - ERROR - Nova failed to get host of the VM
2025-12-22 03:52:54,973 - INFO - Skip interface_detach for VM ctest-TestBGPaasZone-53547593-04273383 in state ERROR
2025-12-22 03:52:54,973 - INFO - Deleting VM ctest-TestBGPaasZone-53547593-04273383
2025-12-22 03:52:54,973 - ERROR - VM ctest-TestBGPaasZone-53547593-04273383 has failed to come up
2025-12-22 03:52:54,973 - ERROR - Fault seen in nova show <vm-uuid> is:  {'code': 500, 'created': '2025-12-22T03:50:10Z', 'message': 'No valid host was found. ', 'details': 'Traceback (most recent call last):\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 1581, in schedule_and_build_instances\n    instance_uuids, return_alternates=True)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 942, in _schedule_instances\n    return_alternates=return_alternates)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations\n    instance_uuids, return_objects, return_alternates)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations\n    return cctxt.call(ctxt, \'select_destinations\', **msg_args)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 192, in call\n    retry=self.retry, transport_options=self.transport_options)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send\n    transport_options=transport_options)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 691, in send\n    transport_options=transport_options)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send\n    raise result\nnova.exception_Remote.NoValidHost_Remote: No valid host was found. \nTraceback (most recent call last):\n\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner\n    return func(*args, **kwargs)\n\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/manager.py", line 209, in select_destinations\n    raise exception.NoValidHost(reason="")\n\nnova.exception.NoValidHost: No valid host was found. \n\n'}
2025-12-22 03:52:54,975 - INFO - Deleting VN ctest-cnz_vn-43934911 
2025-12-22 03:52:55,022 - DEBUG - VN d54e3a68-e39a-4557-8bae-7d4d6c86f0e3 still in use: Unable to complete operation on network d54e3a68-e39a-4557-8bae-7d4d6c86f0e3. There are one or more ports still in use on the network.
Neutron server returns request_ids: ['req-6a7d2746-ec56-4ac3-9c32-5178dfb64968']
2025-12-22 03:52:55,022 - WARNING - Deleting VN ctest-cnz_vn-43934911 failed..Will retry
2025-12-22 03:52:57,295 - DEBUG - Response for deleting network ()
2025-12-22 03:52:58,578 - ERROR - AssertionError
Python 3.9.21: /usr/bin/python3
Mon Dec 22 03:52:54 2025

A problem occurred in a Python script.  Here is the sequence of
function calls leading up to the error, in the order they occurred.

 /contrail-test/tcutils/wrappers.py in wrapper(self=<serial_scripts.bgpaas.test_bgpaas_zone.TestBGPa..._zones_from_single_vnf[sanity] id=0x7fcc2f15b820>, *args=(), **kwargs={})
   78             log.info('Initial checks done. Running the testcase now')
   79             log.info('')
   80             result = function(self, *args, **kwargs)
   81             if self.inputs.upgrade:
   82                 pid = os.getpid()
result = None
function = <function TestBGPaasZone.test_bgp_control_node_zones_from_single_vnf>
self = <serial_scripts.bgpaas.test_bgpaas_zone.TestBGPa..._zones_from_single_vnf[sanity] id=0x7fcc2f15b820>
args = ()
kwargs = {}

 /contrail-test/serial_scripts/bgpaas/test_bgpaas_zone.py in test_bgp_control_node_zones_from_single_vnf(self=<serial_scripts.bgpaas.test_bgpaas_zone.TestBGPa..._zones_from_single_vnf[sanity] id=0x7fcc2f15b820>)
  145         self.logger.info('executing bgp_control_zone agent restart test')
  146         host_ips = []
  147         assert self.basic_control_node_zone_setup('both-zones')
  148         for vm in self.vms:
  149             assert self.verify_bgpaas_in_control_nodes_and_agent(self.bgpaas_fixtures[cnt],vm)
self = <serial_scripts.bgpaas.test_bgpaas_zone.TestBGPa..._zones_from_single_vnf[sanity] id=0x7fcc2f15b820>
self.basic_control_node_zone_setup = <bound method TestBGPaasZone.basic_control_node_...zones_from_single_vnf[sanity] id=0x7fcc2f15b820>>

 /contrail-test/serial_scripts/bgpaas/test_bgpaas_zone.py in basic_control_node_zone_setup(self=<serial_scripts.bgpaas.test_bgpaas_zone.TestBGPa..._zones_from_single_vnf[sanity] id=0x7fcc2f15b820>, ctrl_zone='both-zones')
   31             self.vms.append(self.create_vm(vn_fixture=self.vn, image_name='ubuntu-bird'))
   32         self.client_vm = self.create_vm(vn_fixture=self.vn, image_name='ubuntu-bird')
   33         self.check_vms_booted(self.vms + [self.client_vm])
   34         for bgp_name in self.inputs.bgp_names:
   35             self.cnzs  +=  self.create_control_node_zones("test-zone",[bgp_name])
self = <serial_scripts.bgpaas.test_bgpaas_zone.TestBGPa..._zones_from_single_vnf[sanity] id=0x7fcc2f15b820>
self.check_vms_booted = <bound method GenericTestBase.check_vms_booted o...scripts.bgpaas.test_bgpaas_zone.TestBGPaasZone'>>
self.vms = [<VMFixture: ctest-TestBGPaasZone-53547593-94137530>, <VMFixture: ctest-TestBGPaasZone-53547593-04273383>]
self.client_vm = <VMFixture: ctest-TestBGPaasZone-53547593-45766689>

 /contrail-test/common/base.py in check_vms_booted(cls=<class 'serial_scripts.bgpaas.test_bgpaas_zone.TestBGPaasZone'>, vms_list=[<VMFixture: ctest-TestBGPaasZone-53547593-94137530>, <VMFixture: ctest-TestBGPaasZone-53547593-04273383>, <VMFixture: ctest-TestBGPaasZone-53547593-45766689>], do_assert=True)
  719         if failed and do_assert:
  720             for vm_fixture in vms_list:
  721                 vm_fixture.cleanUp()
  722             assert False, 'One or more vm-boots failed. Check logs'
  723         if failed:
vm_fixture = <VMFixture: ctest-TestBGPaasZone-53547593-04273383>
vm_fixture.cleanUp = <bound method VMFixture.cleanUp of <VMFixture: ctest-TestBGPaasZone-53547593-04273383>>

 /contrail-test/fixtures/vm_test.py in cleanUp(self=<VMFixture: ctest-TestBGPaasZone-53547593-04273383>)
 2050 
 2051     def cleanUp(self):
 2052         self.delete()
 2053         super(VMFixture, self).cleanUp()
 2054 
self = <VMFixture: ctest-TestBGPaasZone-53547593-04273383>
self.delete = <bound method VMFixture.delete of <VMFixture: ctest-TestBGPaasZone-53547593-04273383>>

 /contrail-test/fixtures/vm_test.py in delete(self=<VMFixture: ctest-TestBGPaasZone-53547593-04273383>, verify=False, force=False)
 2080                     self.webui.delete_vm(self)
 2081                 else:
 2082                     self.orch.delete_vm(vm_obj, force=force)
 2083                     self.vm_objs.remove(vm_obj)
 2084                 if self.inputs.ns_agilio_vrouter_data:
self = <VMFixture: ctest-TestBGPaasZone-53547593-04273383>
self.orch = <openstack.OpenstackOrchestrator object>
self.orch.delete_vm = <bound method OpenstackOrchestrator.delete_vm of <openstack.OpenstackOrchestrator object>>
vm_obj = <Server: ctest-TestBGPaasZone-53547593-04273383>
force = False

 /contrail-test/fixtures/openstack.py in delete_vm(self=<openstack.OpenstackOrchestrator object>, vm_obj=<Server: ctest-TestBGPaasZone-53547593-04273383>, **kwargs={'force': False})
  119 
  120    def delete_vm(self, vm_obj, **kwargs):
  121        return self.nova_h.delete_vm(vm_obj, **kwargs)
  122 
  123    def is_vm_deleted(self, vm_obj, **kwargs):
self = <openstack.OpenstackOrchestrator object>
self.nova_h = <nova_test.NovaHelper object>
self.nova_h.delete_vm = <bound method NovaHelper.delete_vm of <nova_test.NovaHelper object>>
vm_obj = <Server: ctest-TestBGPaasZone-53547593-04273383>
kwargs = {'force': False}

 /contrail-test/fixtures/nova_test.py in delete_vm(self=<nova_test.NovaHelper object>, vm_obj=<Server: ctest-TestBGPaasZone-53547593-04273383>, force=False)
  763     def delete_vm(self, vm_obj, force=False):
  764         if not force:
  765             self.get_nova_host_of_vm(vm_obj)
  766         vm_obj.delete()
  767     # end _delete_vm
self = <nova_test.NovaHelper object>
self.get_nova_host_of_vm = <bound method NovaHelper.get_nova_host_of_vm of <nova_test.NovaHelper object>>
vm_obj = <Server: ctest-TestBGPaasZone-53547593-04273383>

 /contrail-test/fixtures/nova_test.py in get_nova_host_of_vm(self=<nova_test.NovaHelper object>, vm_obj=<Server: ctest-TestBGPaasZone-53547593-04273383>)
  734                     self.logger.error('VM %s has failed to come up' % vm_obj.name)
  735                     self.logger.error('Fault seen in nova show <vm-uuid> is:  %s' % vm_obj.__dict__['fault'])
  736                     assert False, 'Fault seen in nova show %s is:  %s' % (
  737                         vm_obj.id, vm_obj.__dict__['fault'])
  738                 else:
vm_obj = <Server: ctest-TestBGPaasZone-53547593-04273383>
vm_obj.id = 'cd1a587d-f5fe-4491-a2ae-faf8b7179f69'
vm_obj.__dict__ = {'OS-DCF:diskConfig': 'MANUAL', 'OS-EXT-AZ:availability_zone': 'nova', 'OS-EXT-SRV-ATTR:host': None, 'OS-EXT-SRV-ATTR:hypervisor_hostname': None, 'OS-EXT-SRV-ATTR:instance_name': 'instance-00000006', 'OS-EXT-STS:power_state': 0, 'OS-EXT-STS:task_state': None, 'OS-EXT-STS:vm_state': 'error', 'OS-SRV-USG:launched_at': None, 'OS-SRV-USG:terminated_at': None, ...}
AssertionError: Fault seen in nova show cd1a587d-f5fe-4491-a2ae-faf8b7179f69 is:  {'code': 500, 'created': '2025-12-22T03:50:10Z', 'message': 'No valid host was found. ', 'details': 'Traceback (most recent call last):\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 1581, in schedule_and_build_instances\n    instance_uuids, return_alternates=True)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 942, in _schedule_instances\n    return_alternates=return_alternates)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations\n    instance_uuids, return_objects, return_alternates)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations\n    return cctxt.call(ctxt, \'select_destinations\', **msg_args)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 192, in call\n    retry=self.retry, transport_options=self.transport_options)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send\n    transport_options=transport_options)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 691, in send\n    transport_options=transport_options)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send\n    raise result\nnova.exception_Remote.NoValidHost_Remote: No valid host was found. \nTraceback (most recent call last):\n\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner\n    return func(*args, **kwargs)\n\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/manager.py", line 209, in select_destinations\n    raise exception.NoValidHost(reason="")\n\nnova.exception.NoValidHost: No valid host was found. \n\n'}
    __cause__ = None
    __class__ = <class 'AssertionError'>
    __context__ = None
    __delattr__ = <method-wrapper '__delattr__' of AssertionError object>
    __dict__ = {}
    __dir__ = <built-in method __dir__ of AssertionError object>
    __doc__ = 'Assertion failed.'
    __eq__ = <method-wrapper '__eq__' of AssertionError object>
    __format__ = <built-in method __format__ of AssertionError object>
    __ge__ = <method-wrapper '__ge__' of AssertionError object>
    __getattribute__ = <method-wrapper '__getattribute__' of AssertionError object>
    __gt__ = <method-wrapper '__gt__' of AssertionError object>
    __hash__ = <method-wrapper '__hash__' of AssertionError object>
    __init__ = <method-wrapper '__init__' of AssertionError object>
    __init_subclass__ = <built-in method __init_subclass__ of type object>
    __le__ = <method-wrapper '__le__' of AssertionError object>
    __lt__ = <method-wrapper '__lt__' of AssertionError object>
    __ne__ = <method-wrapper '__ne__' of AssertionError object>
    __new__ = <built-in method __new__ of type object>
    __reduce__ = <built-in method __reduce__ of AssertionError object>
    __reduce_ex__ = <built-in method __reduce_ex__ of AssertionError object>
    __repr__ = <method-wrapper '__repr__' of AssertionError object>
    __setattr__ = <method-wrapper '__setattr__' of AssertionError object>
    __setstate__ = <built-in method __setstate__ of AssertionError object>
    __sizeof__ = <built-in method __sizeof__ of AssertionError object>
    __str__ = <method-wrapper '__str__' of AssertionError object>
    __subclasshook__ = <built-in method __subclasshook__ of type object>
    __suppress_context__ = False
    __traceback__ = <traceback object>
    args = (r"Fault seen in nova show cd1a587d-f5fe-4491-a2ae-...tion.NoValidHost: No valid host was found. \n\n'}",)
    with_traceback = <built-in method with_traceback of AssertionError object>

The above is a description of an error in a Python program.  Here is
the original traceback:

Traceback (most recent call last):
  File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper
    result = function(self, *args, **kwargs)
  File "/contrail-test/serial_scripts/bgpaas/test_bgpaas_zone.py", line 147, in test_bgp_control_node_zones_from_single_vnf
    assert self.basic_control_node_zone_setup('both-zones')
  File "/contrail-test/serial_scripts/bgpaas/test_bgpaas_zone.py", line 33, in basic_control_node_zone_setup
    self.check_vms_booted(self.vms + [self.client_vm])
  File "/contrail-test/common/base.py", line 721, in check_vms_booted
    vm_fixture.cleanUp()
  File "/contrail-test/fixtures/vm_test.py", line 2052, in cleanUp
    self.delete()
  File "/contrail-test/fixtures/vm_test.py", line 2082, in delete
    self.orch.delete_vm(vm_obj, force=force)
  File "/contrail-test/fixtures/openstack.py", line 121, in delete_vm
    return self.nova_h.delete_vm(vm_obj, **kwargs)
  File "/contrail-test/fixtures/nova_test.py", line 765, in delete_vm
    self.get_nova_host_of_vm(vm_obj)
  File "/contrail-test/fixtures/nova_test.py", line 736, in get_nova_host_of_vm
    assert False, 'Fault seen in nova show %s is:  %s' % (
AssertionError: Fault seen in nova show cd1a587d-f5fe-4491-a2ae-faf8b7179f69 is:  {'code': 500, 'created': '2025-12-22T03:50:10Z', 'message': 'No valid host was found. ', 'details': 'Traceback (most recent call last):\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 1581, in schedule_and_build_instances\n    instance_uuids, return_alternates=True)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 942, in _schedule_instances\n    return_alternates=return_alternates)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations\n    instance_uuids, return_objects, return_alternates)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations\n    return cctxt.call(ctxt, \'select_destinations\', **msg_args)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 192, in call\n    retry=self.retry, transport_options=self.transport_options)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send\n    transport_options=transport_options)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 691, in send\n    transport_options=transport_options)\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send\n    raise result\nnova.exception_Remote.NoValidHost_Remote: No valid host was found. \nTraceback (most recent call last):\n\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner\n    return func(*args, **kwargs)\n\n  File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/manager.py", line 209, in select_destinations\n    raise exception.NoValidHost(reason="")\n\nnova.exception.NoValidHost: No valid host was found. \n\n'}



2025-12-22 03:52:58,579 - DEBUG - Skipping xmpp flap check
2025-12-22 03:52:58,579 - INFO - 
2025-12-22 03:52:58,579 - INFO - END TEST : test_bgp_control_node_zones_from_single_vnf : FAILED[0:02:52]
2025-12-22 03:52:58,579 - INFO - --------------------------------------------------------------------------------
2025-12-22 03:52:59,354 - INFO - Deleted project: ctest-TestBGPaasZone-53547593, ID : 7b526cb0-ef57-466c-84c2-5bff8d0bce62