2025-09-11 03:19:04,377 - INFO - Domain Default found not creating 2025-09-11 03:19:04,525 - INFO - Project ctest-TestBGPaasZone-96156374 not found, creating it 2025-09-11 03:19:05,217 - INFO - Created Project:ctest-TestBGPaasZone-96156374, ID : 635e2a47-ae48-497f-9484-30afb609d66d 2025-09-11 03:19:06,875 - DEBUG - Services list from nova: [, , , ] 2025-09-11 03:19:06,875 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-4079-1', 'an-jenkins-deploy-platform-ansible-os-4079-2']} 2025-09-11 03:19:06,877 - INFO - ================================================================================ 2025-09-11 03:19:06,877 - INFO - STARTING TEST : test_bgp_control_node_zone 2025-09-11 03:19:08,140 - DEBUG - Skipping xmpp flap check 2025-09-11 03:19:08,140 - INFO - Initial checks done. Running the testcase now 2025-09-11 03:19:08,140 - INFO - 2025-09-11 03:19:08,140 - INFO - executing bgp_control_zone test 2025-09-11 03:19:08,861 - DEBUG - Response for create_network : {'network': {'id': 'e7f4137c-b87b-423a-a00c-9b92f59cd65e', 'name': 'ctest-cnz_vn-20373915', 'tenant_id': '635e2a47ae48497f948430afb609d66d', 'project_id': '635e2a47ae48497f948430afb609d66d', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestBGPaasZone-96156374', 'ctest-cnz_vn-20373915'], 'port_security_enabled': True, 'description': ''}} 2025-09-11 03:19:09,113 - DEBUG - Response for create_subnet : {'subnet': {'id': '98b0edae-abe4-419f-ac0a-d4028193024d', 'name': '', 'tenant_id': '635e2a47ae48497f948430afb609d66d', 'network_id': 'e7f4137c-b87b-423a-a00c-9b92f59cd65e', 'ip_version': 4, 'cidr': '199.39.46.192/26', 'allocation_pools': [{'start': '199.39.46.194', 'end': '199.39.46.254'}], 'gateway_ip': '199.39.46.193', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '199.39.46.194', 'tags': [], 'project_id': '635e2a47ae48497f948430afb609d66d'}} 2025-09-11 03:19:09,134 - INFO - Created VN ctest-cnz_vn-20373915 2025-09-11 03:19:09,146 - DEBUG - VN ctest-cnz_vn-20373915 UUID is e7f4137c-b87b-423a-a00c-9b92f59cd65e 2025-09-11 03:19:10,616 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4079-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4079-1) 2025-09-11 03:19:12,294 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4079-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4079-2) 2025-09-11 03:19:14,197 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4079-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4079-1) 2025-09-11 03:19:14,197 - INFO - Waiting for VM ctest-TestBGPaasZone-96156374-09243738 to be up.. 2025-09-11 03:19:14,307 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-09-11 03:19:19,396 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-09-11 03:19:24,484 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-09-11 03:19:29,567 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-09-11 03:19:34,657 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-09-11 03:19:39,741 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-09-11 03:19:44,824 - DEBUG - VM is in ACTIVE state now 2025-09-11 03:19:44,824 - INFO - VM name : ctest-TestBGPaasZone-96156374-09243738 2025-09-11 03:19:44,916 - DEBUG - VM ctest-TestBGPaasZone-96156374-09243738 ID is 7d824a6d-0c2b-41d3-84ff-4ce7d51808f9 2025-09-11 03:19:44,952 - DEBUG - VM ctest-TestBGPaasZone-96156374-09243738 launched on Node an-jenkins-deploy-platform-ansible-os-4079-1 2025-09-11 03:19:45,035 - DEBUG - Requesting: http://10.0.0.49:8082/virtual-machine/7d824a6d-0c2b-41d3-84ff-4ce7d51808f9 2025-09-11 03:19:45,344 - DEBUG - Requesting: http://10.0.0.49:8082/virtual-machine/7d824a6d-0c2b-41d3-84ff-4ce7d51808f9 2025-09-11 03:19:45,378 - DEBUG - Requesting: http://10.0.0.49:8082/virtual-machine-interface/b0a40c03-9afb-499c-aa79-90f42ad2fe14 2025-09-11 03:19:48,553 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1011ms') 2025-09-11 03:19:48,553 - DEBUG - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 failed! 2025-09-11 03:19:48,570 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-96156374:ctest-cnz_vn-20373915 is 199.39.46.193 and allocation pool is NOT set 2025-09-11 03:19:52,654 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1021ms') 2025-09-11 03:19:52,655 - DEBUG - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 failed! 2025-09-11 03:19:52,669 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-96156374:ctest-cnz_vn-20373915 is 199.39.46.193 and allocation pool is NOT set 2025-09-11 03:19:56,750 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1017ms') 2025-09-11 03:19:56,750 - DEBUG - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 failed! 2025-09-11 03:19:56,766 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-96156374:ctest-cnz_vn-20373915 is 199.39.46.193 and allocation pool is NOT set 2025-09-11 03:20:00,848 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1019ms') 2025-09-11 03:20:00,848 - DEBUG - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 failed! 2025-09-11 03:20:00,864 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-96156374:ctest-cnz_vn-20373915 is 199.39.46.193 and allocation pool is NOT set 2025-09-11 03:20:04,941 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1015ms') 2025-09-11 03:20:04,941 - DEBUG - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 failed! 2025-09-11 03:20:04,956 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-96156374:ctest-cnz_vn-20373915 is 199.39.46.193 and allocation pool is NOT set 2025-09-11 03:20:09,033 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1020ms') 2025-09-11 03:20:09,033 - DEBUG - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 failed! 2025-09-11 03:20:09,047 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-96156374:ctest-cnz_vn-20373915 is 199.39.46.193 and allocation pool is NOT set 2025-09-11 03:20:13,130 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1026ms') 2025-09-11 03:20:13,130 - DEBUG - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 failed! 2025-09-11 03:20:13,145 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-96156374:ctest-cnz_vn-20373915 is 199.39.46.193 and allocation pool is NOT set 2025-09-11 03:20:17,232 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1023ms') 2025-09-11 03:20:17,232 - DEBUG - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 failed! 2025-09-11 03:20:17,248 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-96156374:ctest-cnz_vn-20373915 is 199.39.46.193 and allocation pool is NOT set 2025-09-11 03:20:21,322 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1014ms') 2025-09-11 03:20:21,323 - DEBUG - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 failed! 2025-09-11 03:20:21,342 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-96156374:ctest-cnz_vn-20373915 is 199.39.46.193 and allocation pool is NOT set 2025-09-11 03:20:25,419 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1019ms') 2025-09-11 03:20:25,419 - DEBUG - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 failed! 2025-09-11 03:20:25,433 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-96156374:ctest-cnz_vn-20373915 is 199.39.46.193 and allocation pool is NOT set 2025-09-11 03:20:29,522 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1023ms') 2025-09-11 03:20:29,522 - DEBUG - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 failed! 2025-09-11 03:20:29,539 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-96156374:ctest-cnz_vn-20373915 is 199.39.46.193 and allocation pool is NOT set 2025-09-11 03:20:33,613 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1016ms') 2025-09-11 03:20:33,613 - DEBUG - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 failed! 2025-09-11 03:20:33,628 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-96156374:ctest-cnz_vn-20373915 is 199.39.46.193 and allocation pool is NOT set 2025-09-11 03:20:37,710 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1011ms') 2025-09-11 03:20:37,710 - DEBUG - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 failed! 2025-09-11 03:20:37,728 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-96156374:ctest-cnz_vn-20373915 is 199.39.46.193 and allocation pool is NOT set 2025-09-11 03:20:41,808 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1013ms') 2025-09-11 03:20:41,809 - DEBUG - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 failed! 2025-09-11 03:20:41,822 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-96156374:ctest-cnz_vn-20373915 is 199.39.46.193 and allocation pool is NOT set 2025-09-11 03:20:45,900 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1014ms') 2025-09-11 03:20:45,900 - DEBUG - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 failed! 2025-09-11 03:20:45,916 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-96156374:ctest-cnz_vn-20373915 is 199.39.46.193 and allocation pool is NOT set 2025-09-11 03:20:50,002 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1021ms') 2025-09-11 03:20:50,002 - DEBUG - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 failed! 2025-09-11 03:20:50,017 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-96156374:ctest-cnz_vn-20373915 is 199.39.46.193 and allocation pool is NOT set 2025-09-11 03:20:54,122 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1031ms') 2025-09-11 03:20:54,123 - DEBUG - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 failed! 2025-09-11 03:20:54,138 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-96156374:ctest-cnz_vn-20373915 is 199.39.46.193 and allocation pool is NOT set 2025-09-11 03:20:58,223 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1017ms') 2025-09-11 03:20:58,224 - DEBUG - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 failed! 2025-09-11 03:20:58,240 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-96156374:ctest-cnz_vn-20373915 is 199.39.46.193 and allocation pool is NOT set 2025-09-11 03:21:02,316 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1006ms') 2025-09-11 03:21:02,316 - DEBUG - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 failed! 2025-09-11 03:21:02,331 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-96156374:ctest-cnz_vn-20373915 is 199.39.46.193 and allocation pool is NOT set 2025-09-11 03:21:06,412 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1013ms') 2025-09-11 03:21:06,413 - DEBUG - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 failed! 2025-09-11 03:21:06,429 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-96156374:ctest-cnz_vn-20373915 is 199.39.46.193 and allocation pool is NOT set 2025-09-11 03:21:08,500 - DEBUG - (True, 'PING 169.254.0.7 (169.254.0.7) 56(84) bytes of data.\r\n64 bytes from 169.254.0.7: icmp_seq=1 ttl=63 time=4.74 ms\r\n64 bytes from 169.254.0.7: icmp_seq=2 ttl=63 time=1.99 ms\r\n\r\n--- 169.254.0.7 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1002ms\r\nrtt min/avg/max/mdev = 1.988/3.364/4.741/1.376 ms') 2025-09-11 03:21:08,501 - INFO - Ping to Metadata IP 169.254.0.7 of VM ctest-TestBGPaasZone-96156374-09243738 passed 2025-09-11 03:21:08,567 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-09-11 03:21:08,568 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-96156374-09243738, IP 199.39.46.195, Port 22 2025-09-11 03:21:08,644 - DEBUG - Error on ssh to ubuntu@169.254.0.7:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.7/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.7/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.7/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-09-11 03:21:08,731 - DEBUG - VM ctest-TestBGPaasZone-96156374-09243738 is NOT ready for SSH connections, VM status: ACTIVE 2025-09-11 03:21:13,731 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-09-11 03:21:13,731 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-96156374-09243738, IP 199.39.46.195, Port 22 2025-09-11 03:21:13,801 - DEBUG - Error on ssh to ubuntu@169.254.0.7:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.7/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.7/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.7/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-09-11 03:21:13,884 - DEBUG - VM ctest-TestBGPaasZone-96156374-09243738 is NOT ready for SSH connections, VM status: ACTIVE 2025-09-11 03:21:18,884 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-09-11 03:21:18,885 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-96156374-09243738, IP 199.39.46.195, Port 22 2025-09-11 03:21:18,953 - DEBUG - Error on ssh to ubuntu@169.254.0.7:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.7/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.7/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.7/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-09-11 03:21:19,035 - DEBUG - VM ctest-TestBGPaasZone-96156374-09243738 is NOT ready for SSH connections, VM status: ACTIVE 2025-09-11 03:21:24,036 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-09-11 03:21:24,036 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-96156374-09243738, IP 199.39.46.195, Port 22 2025-09-11 03:21:24,095 - DEBUG - Error on ssh to ubuntu@169.254.0.7:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.7/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.7/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.7/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-09-11 03:21:24,189 - DEBUG - VM ctest-TestBGPaasZone-96156374-09243738 is NOT ready for SSH connections, VM status: ACTIVE 2025-09-11 03:21:29,190 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-09-11 03:21:29,190 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-96156374-09243738, IP 199.39.46.195, Port 22 2025-09-11 03:21:29,257 - DEBUG - Error on ssh to ubuntu@169.254.0.7:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.7/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.7/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.7/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-09-11 03:21:29,344 - DEBUG - VM ctest-TestBGPaasZone-96156374-09243738 is NOT ready for SSH connections, VM status: ACTIVE 2025-09-11 03:21:34,345 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-09-11 03:21:34,345 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-96156374-09243738, IP 199.39.46.195, Port 22 2025-09-11 03:21:34,413 - DEBUG - Error on ssh to ubuntu@169.254.0.7:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.7/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.7/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.7/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-09-11 03:21:34,489 - DEBUG - VM ctest-TestBGPaasZone-96156374-09243738 is NOT ready for SSH connections, VM status: ACTIVE 2025-09-11 03:21:39,489 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-09-11 03:21:39,490 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-96156374-09243738, IP 199.39.46.195, Port 22 2025-09-11 03:21:39,557 - DEBUG - Error on ssh to ubuntu@169.254.0.7:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.7/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.7/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.7/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-09-11 03:21:39,645 - DEBUG - VM ctest-TestBGPaasZone-96156374-09243738 is NOT ready for SSH connections, VM status: ACTIVE 2025-09-11 03:21:44,646 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-09-11 03:21:44,647 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-96156374-09243738, IP 199.39.46.195, Port 22 2025-09-11 03:21:44,707 - DEBUG - Error on ssh to ubuntu@169.254.0.7:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.7/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.7/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.7/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-09-11 03:21:44,788 - DEBUG - VM ctest-TestBGPaasZone-96156374-09243738 is NOT ready for SSH connections, VM status: ACTIVE 2025-09-11 03:21:49,788 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-09-11 03:21:49,788 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-96156374-09243738, IP 199.39.46.195, Port 22 2025-09-11 03:21:49,847 - DEBUG - Error on ssh to ubuntu@169.254.0.7:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.7/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.7/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.7/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-09-11 03:21:49,932 - DEBUG - VM ctest-TestBGPaasZone-96156374-09243738 is NOT ready for SSH connections, VM status: ACTIVE 2025-09-11 03:21:54,933 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-09-11 03:21:54,933 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-96156374-09243738, IP 199.39.46.195, Port 22 2025-09-11 03:21:54,991 - DEBUG - Error on ssh to ubuntu@169.254.0.7:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.7/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.7/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.7/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-09-11 03:21:55,066 - DEBUG - VM ctest-TestBGPaasZone-96156374-09243738 is NOT ready for SSH connections, VM status: ACTIVE 2025-09-11 03:22:00,067 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-09-11 03:22:00,067 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-96156374-09243738, IP 199.39.46.195, Port 22 2025-09-11 03:22:00,240 - DEBUG - VM ctest-TestBGPaasZone-96156374-09243738 is ready for SSH connections 2025-09-11 03:22:00,241 - INFO - Waiting for VM ctest-TestBGPaasZone-96156374-45803196 to be up.. 2025-09-11 03:22:00,312 - DEBUG - VM is in ERROR state now 2025-09-11 03:22:00,312 - ERROR - VM in error state. 2025-09-11 03:22:00,312 - ERROR - VM is not in ACTIVE state 2025-09-11 03:22:00,402 - ERROR - Fatal Nova Exception while getting VM detail 2025-09-11 03:22:00,402 - DEBUG - None 2025-09-11 03:22:00,402 - ERROR - VM ctest-TestBGPaasZone-96156374-45803196 has not booted 2025-09-11 03:22:00,402 - INFO - Deleting VM ctest-TestBGPaasZone-96156374-09243738 2025-09-11 03:22:00,478 - INFO - Skip interface_detach for VM ctest-TestBGPaasZone-96156374-45803196 in state ERROR 2025-09-11 03:22:00,479 - INFO - Deleting VM ctest-TestBGPaasZone-96156374-45803196 2025-09-11 03:22:00,479 - ERROR - VM ctest-TestBGPaasZone-96156374-45803196 has failed to come up 2025-09-11 03:22:00,479 - ERROR - Fault seen in nova show is: {'code': 500, 'created': '2025-09-11T03:19:12Z', 'message': 'No valid host was found. ', 'details': 'Traceback (most recent call last):\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 1581, in schedule_and_build_instances\n instance_uuids, return_alternates=True)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 942, in _schedule_instances\n return_alternates=return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations\n instance_uuids, return_objects, return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations\n return cctxt.call(ctxt, \'select_destinations\', **msg_args)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 192, in call\n retry=self.retry, transport_options=self.transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 691, in send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send\n raise result\nnova.exception_Remote.NoValidHost_Remote: No valid host was found. \nTraceback (most recent call last):\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner\n return func(*args, **kwargs)\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/manager.py", line 209, in select_destinations\n raise exception.NoValidHost(reason="")\n\nnova.exception.NoValidHost: No valid host was found. \n\n'} 2025-09-11 03:22:00,483 - INFO - Skip interface_detach for VM ctest-TestBGPaasZone-96156374-91608722 in state BUILD 2025-09-11 03:22:00,483 - INFO - Deleting VM ctest-TestBGPaasZone-96156374-91608722 2025-09-11 03:22:00,484 - ERROR - VM ctest-TestBGPaasZone-96156374-91608722 has failed to come up 2025-09-11 03:22:00,484 - ERROR - Nova failed to get host of the VM 2025-09-11 03:22:00,484 - ERROR - VM ctest-TestBGPaasZone-96156374-91608722 has failed to come up 2025-09-11 03:22:00,484 - ERROR - Nova failed to get host of the VM 2025-09-11 03:22:00,566 - INFO - Skip interface_detach for VM ctest-TestBGPaasZone-96156374-45803196 in state ERROR 2025-09-11 03:22:00,566 - INFO - Deleting VM ctest-TestBGPaasZone-96156374-45803196 2025-09-11 03:22:00,566 - ERROR - VM ctest-TestBGPaasZone-96156374-45803196 has failed to come up 2025-09-11 03:22:00,566 - ERROR - Fault seen in nova show is: {'code': 500, 'created': '2025-09-11T03:19:12Z', 'message': 'No valid host was found. ', 'details': 'Traceback (most recent call last):\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 1581, in schedule_and_build_instances\n instance_uuids, return_alternates=True)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 942, in _schedule_instances\n return_alternates=return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations\n instance_uuids, return_objects, return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations\n return cctxt.call(ctxt, \'select_destinations\', **msg_args)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 192, in call\n retry=self.retry, transport_options=self.transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 691, in send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send\n raise result\nnova.exception_Remote.NoValidHost_Remote: No valid host was found. \nTraceback (most recent call last):\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner\n return func(*args, **kwargs)\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/manager.py", line 209, in select_destinations\n raise exception.NoValidHost(reason="")\n\nnova.exception.NoValidHost: No valid host was found. \n\n'} 2025-09-11 03:22:00,567 - INFO - Deleting VN ctest-cnz_vn-20373915 2025-09-11 03:22:00,612 - DEBUG - VN e7f4137c-b87b-423a-a00c-9b92f59cd65e still in use: Unable to complete operation on network e7f4137c-b87b-423a-a00c-9b92f59cd65e. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-0f5d35ae-b9b2-4703-8f32-48a9e299d053'] 2025-09-11 03:22:00,613 - WARNING - Deleting VN ctest-cnz_vn-20373915 failed..Will retry 2025-09-11 03:22:02,798 - DEBUG - Response for deleting network () 2025-09-11 03:22:04,076 - ERROR - AssertionError Python 3.9.21: /usr/bin/python3 Thu Sep 11 03:22:00 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={}) 78 log.info('Initial checks done. Running the testcase now') 79 log.info('') 80 result = function(self, *args, **kwargs) 81 if self.inputs.upgrade: 82 pid = os.getpid() result = None function = self = args = () kwargs = {} /contrail-test/serial_scripts/bgpaas/test_bgpaas_zone.py in test_bgp_control_node_zone(self=) 57 cnt = 0 58 self.logger.info('executing bgp_control_zone test') 59 assert self.basic_control_node_zone_setup() 60 # update control node zone with different bgp routers 61 self.update_control_node_zones(self.cnzs) self = self.basic_control_node_zone_setup = > /contrail-test/serial_scripts/bgpaas/test_bgpaas_zone.py in basic_control_node_zone_setup(self=, ctrl_zone=None) 31 self.vms.append(self.create_vm(vn_fixture=self.vn, image_name='ubuntu-bird')) 32 self.client_vm = self.create_vm(vn_fixture=self.vn, image_name='ubuntu-bird') 33 self.check_vms_booted(self.vms + [self.client_vm]) 34 for bgp_name in self.inputs.bgp_names: 35 self.cnzs += self.create_control_node_zones("test-zone",[bgp_name]) self = self.check_vms_booted = > self.vms = [, ] self.client_vm = /contrail-test/common/base.py in check_vms_booted(cls=, vms_list=[, , ], do_assert=True) 719 if failed and do_assert: 720 for vm_fixture in vms_list: 721 vm_fixture.cleanUp() 722 assert False, 'One or more vm-boots failed. Check logs' 723 if failed: vm_fixture = vm_fixture.cleanUp = > /contrail-test/fixtures/vm_test.py in cleanUp(self=) 2050 2051 def cleanUp(self): 2052 self.delete() 2053 super(VMFixture, self).cleanUp() 2054 self = self.delete = > /contrail-test/fixtures/vm_test.py in delete(self=, verify=False, force=False) 2080 self.webui.delete_vm(self) 2081 else: 2082 self.orch.delete_vm(vm_obj, force=force) 2083 self.vm_objs.remove(vm_obj) 2084 if self.inputs.ns_agilio_vrouter_data: self = self.orch = self.orch.delete_vm = > vm_obj = force = False /contrail-test/fixtures/openstack.py in delete_vm(self=, vm_obj=, **kwargs={'force': False}) 119 120 def delete_vm(self, vm_obj, **kwargs): 121 return self.nova_h.delete_vm(vm_obj, **kwargs) 122 123 def is_vm_deleted(self, vm_obj, **kwargs): self = self.nova_h = self.nova_h.delete_vm = > vm_obj = kwargs = {'force': False} /contrail-test/fixtures/nova_test.py in delete_vm(self=, vm_obj=, force=False) 763 def delete_vm(self, vm_obj, force=False): 764 if not force: 765 self.get_nova_host_of_vm(vm_obj) 766 vm_obj.delete() 767 # end _delete_vm self = self.get_nova_host_of_vm = > vm_obj = /contrail-test/fixtures/nova_test.py in get_nova_host_of_vm(self=, vm_obj=) 734 self.logger.error('VM %s has failed to come up' % vm_obj.name) 735 self.logger.error('Fault seen in nova show is: %s' % vm_obj.__dict__['fault']) 736 assert False, 'Fault seen in nova show %s is: %s' % ( 737 vm_obj.id, vm_obj.__dict__['fault']) 738 else: vm_obj = vm_obj.id = 'c6f6ecd9-ba12-4220-9c56-fce14bbd1bf8' vm_obj.__dict__ = {'OS-DCF:diskConfig': 'MANUAL', 'OS-EXT-AZ:availability_zone': 'nova', 'OS-EXT-SRV-ATTR:host': None, 'OS-EXT-SRV-ATTR:hypervisor_hostname': None, 'OS-EXT-SRV-ATTR:instance_name': 'instance-0000001a', 'OS-EXT-STS:power_state': 0, 'OS-EXT-STS:task_state': None, 'OS-EXT-STS:vm_state': 'error', 'OS-SRV-USG:launched_at': None, 'OS-SRV-USG:terminated_at': None, ...} AssertionError: Fault seen in nova show c6f6ecd9-ba12-4220-9c56-fce14bbd1bf8 is: {'code': 500, 'created': '2025-09-11T03:19:12Z', 'message': 'No valid host was found. ', 'details': 'Traceback (most recent call last):\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 1581, in schedule_and_build_instances\n instance_uuids, return_alternates=True)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 942, in _schedule_instances\n return_alternates=return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations\n instance_uuids, return_objects, return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations\n return cctxt.call(ctxt, \'select_destinations\', **msg_args)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 192, in call\n retry=self.retry, transport_options=self.transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 691, in send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send\n raise result\nnova.exception_Remote.NoValidHost_Remote: No valid host was found. \nTraceback (most recent call last):\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner\n return func(*args, **kwargs)\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/manager.py", line 209, in select_destinations\n raise exception.NoValidHost(reason="")\n\nnova.exception.NoValidHost: No valid host was found. \n\n'} __cause__ = None __class__ = __context__ = None __delattr__ = __dict__ = {} __dir__ = __doc__ = 'Assertion failed.' __eq__ = __format__ = __ge__ = __getattribute__ = __gt__ = __hash__ = __init__ = __init_subclass__ = __le__ = __lt__ = __ne__ = __new__ = __reduce__ = __reduce_ex__ = __repr__ = __setattr__ = __setstate__ = __sizeof__ = __str__ = __subclasshook__ = __suppress_context__ = False __traceback__ = args = (r"Fault seen in nova show c6f6ecd9-ba12-4220-9c56-...tion.NoValidHost: No valid host was found. \n\n'}",) with_traceback = The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper result = function(self, *args, **kwargs) File "/contrail-test/serial_scripts/bgpaas/test_bgpaas_zone.py", line 59, in test_bgp_control_node_zone assert self.basic_control_node_zone_setup() File "/contrail-test/serial_scripts/bgpaas/test_bgpaas_zone.py", line 33, in basic_control_node_zone_setup self.check_vms_booted(self.vms + [self.client_vm]) File "/contrail-test/common/base.py", line 721, in check_vms_booted vm_fixture.cleanUp() File "/contrail-test/fixtures/vm_test.py", line 2052, in cleanUp self.delete() File "/contrail-test/fixtures/vm_test.py", line 2082, in delete self.orch.delete_vm(vm_obj, force=force) File "/contrail-test/fixtures/openstack.py", line 121, in delete_vm return self.nova_h.delete_vm(vm_obj, **kwargs) File "/contrail-test/fixtures/nova_test.py", line 765, in delete_vm self.get_nova_host_of_vm(vm_obj) File "/contrail-test/fixtures/nova_test.py", line 736, in get_nova_host_of_vm assert False, 'Fault seen in nova show %s is: %s' % ( AssertionError: Fault seen in nova show c6f6ecd9-ba12-4220-9c56-fce14bbd1bf8 is: {'code': 500, 'created': '2025-09-11T03:19:12Z', 'message': 'No valid host was found. ', 'details': 'Traceback (most recent call last):\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 1581, in schedule_and_build_instances\n instance_uuids, return_alternates=True)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 942, in _schedule_instances\n return_alternates=return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations\n instance_uuids, return_objects, return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations\n return cctxt.call(ctxt, \'select_destinations\', **msg_args)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 192, in call\n retry=self.retry, transport_options=self.transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 691, in send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send\n raise result\nnova.exception_Remote.NoValidHost_Remote: No valid host was found. \nTraceback (most recent call last):\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner\n return func(*args, **kwargs)\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/manager.py", line 209, in select_destinations\n raise exception.NoValidHost(reason="")\n\nnova.exception.NoValidHost: No valid host was found. \n\n'} 2025-09-11 03:22:04,076 - DEBUG - Skipping xmpp flap check 2025-09-11 03:22:04,076 - INFO - 2025-09-11 03:22:04,076 - INFO - END TEST : test_bgp_control_node_zone : FAILED[0:02:58] 2025-09-11 03:22:04,076 - INFO - -------------------------------------------------------------------------------- 2025-09-11 03:22:04,079 - INFO - ================================================================================ 2025-09-11 03:22:04,079 - INFO - STARTING TEST : test_bgp_control_node_zones_from_single_vnf 2025-09-11 03:22:05,331 - DEBUG - Skipping xmpp flap check 2025-09-11 03:22:05,331 - INFO - Initial checks done. Running the testcase now 2025-09-11 03:22:05,331 - INFO - 2025-09-11 03:22:05,331 - INFO - executing bgp_control_zone agent restart test 2025-09-11 03:22:05,595 - DEBUG - Response for create_network : {'network': {'id': 'e11d74cb-ff91-46bd-b6d8-1aa9af673435', 'name': 'ctest-cnz_vn-86152394', 'tenant_id': '635e2a47ae48497f948430afb609d66d', 'project_id': '635e2a47ae48497f948430afb609d66d', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestBGPaasZone-96156374', 'ctest-cnz_vn-86152394'], 'port_security_enabled': True, 'description': ''}} 2025-09-11 03:22:05,831 - DEBUG - Response for create_subnet : {'subnet': {'id': '784886d3-5435-433f-b016-1589248ed946', 'name': '', 'tenant_id': '635e2a47ae48497f948430afb609d66d', 'network_id': 'e11d74cb-ff91-46bd-b6d8-1aa9af673435', 'ip_version': 4, 'cidr': '103.91.158.0/26', 'allocation_pools': [{'start': '103.91.158.2', 'end': '103.91.158.62'}], 'gateway_ip': '103.91.158.1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '103.91.158.2', 'tags': [], 'project_id': '635e2a47ae48497f948430afb609d66d'}} 2025-09-11 03:22:05,853 - INFO - Created VN ctest-cnz_vn-86152394 2025-09-11 03:22:05,867 - DEBUG - VN ctest-cnz_vn-86152394 UUID is e11d74cb-ff91-46bd-b6d8-1aa9af673435 2025-09-11 03:22:07,459 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4079-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4079-2) 2025-09-11 03:22:08,829 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4079-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4079-1) 2025-09-11 03:22:10,371 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4079-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4079-2) 2025-09-11 03:22:10,371 - INFO - Waiting for VM ctest-TestBGPaasZone-96156374-43159470 to be up.. 2025-09-11 03:22:10,429 - DEBUG - VM is in ERROR state now 2025-09-11 03:22:10,430 - ERROR - VM in error state. 2025-09-11 03:22:10,430 - ERROR - VM is not in ACTIVE state 2025-09-11 03:22:10,515 - ERROR - Fatal Nova Exception while getting VM detail 2025-09-11 03:22:10,515 - DEBUG - None 2025-09-11 03:22:10,516 - ERROR - VM ctest-TestBGPaasZone-96156374-43159470 has not booted 2025-09-11 03:22:10,516 - INFO - Skip interface_detach for VM ctest-TestBGPaasZone-96156374-43159470 in state ERROR 2025-09-11 03:22:10,516 - INFO - Deleting VM ctest-TestBGPaasZone-96156374-43159470 2025-09-11 03:22:10,516 - ERROR - VM ctest-TestBGPaasZone-96156374-43159470 has failed to come up 2025-09-11 03:22:10,516 - ERROR - Fault seen in nova show is: {'code': 500, 'created': '2025-09-11T03:22:07Z', 'message': 'No valid host was found. ', 'details': 'Traceback (most recent call last):\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 1581, in schedule_and_build_instances\n instance_uuids, return_alternates=True)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 942, in _schedule_instances\n return_alternates=return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations\n instance_uuids, return_objects, return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations\n return cctxt.call(ctxt, \'select_destinations\', **msg_args)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 192, in call\n retry=self.retry, transport_options=self.transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 691, in send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send\n raise result\nnova.exception_Remote.NoValidHost_Remote: No valid host was found. \nTraceback (most recent call last):\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner\n return func(*args, **kwargs)\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/manager.py", line 209, in select_destinations\n raise exception.NoValidHost(reason="")\n\nnova.exception.NoValidHost: No valid host was found. \n\n'} 2025-09-11 03:22:10,519 - INFO - Skip interface_detach for VM ctest-TestBGPaasZone-96156374-05185860 in state ERROR 2025-09-11 03:22:10,519 - INFO - Deleting VM ctest-TestBGPaasZone-96156374-05185860 2025-09-11 03:22:10,519 - ERROR - VM ctest-TestBGPaasZone-96156374-05185860 has failed to come up 2025-09-11 03:22:10,519 - ERROR - Fault seen in nova show is: {'code': 500, 'created': '2025-09-11T03:22:10Z', 'message': 'No valid host was found. ', 'details': 'Traceback (most recent call last):\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 1581, in schedule_and_build_instances\n instance_uuids, return_alternates=True)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 942, in _schedule_instances\n return_alternates=return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations\n instance_uuids, return_objects, return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations\n return cctxt.call(ctxt, \'select_destinations\', **msg_args)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 192, in call\n retry=self.retry, transport_options=self.transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 691, in send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send\n raise result\nnova.exception_Remote.NoValidHost_Remote: No valid host was found. \nTraceback (most recent call last):\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner\n return func(*args, **kwargs)\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/manager.py", line 209, in select_destinations\n raise exception.NoValidHost(reason="")\n\nnova.exception.NoValidHost: No valid host was found. \n\n'} 2025-09-11 03:22:10,519 - INFO - Skip interface_detach for VM ctest-TestBGPaasZone-96156374-73388571 in state BUILD 2025-09-11 03:22:10,520 - INFO - Deleting VM ctest-TestBGPaasZone-96156374-73388571 2025-09-11 03:22:10,520 - ERROR - VM ctest-TestBGPaasZone-96156374-73388571 has failed to come up 2025-09-11 03:22:10,520 - ERROR - Nova failed to get host of the VM 2025-09-11 03:22:10,520 - ERROR - VM ctest-TestBGPaasZone-96156374-73388571 has failed to come up 2025-09-11 03:22:10,520 - ERROR - Nova failed to get host of the VM 2025-09-11 03:22:10,593 - INFO - Skip interface_detach for VM ctest-TestBGPaasZone-96156374-43159470 in state ERROR 2025-09-11 03:22:10,593 - INFO - Deleting VM ctest-TestBGPaasZone-96156374-43159470 2025-09-11 03:22:10,593 - ERROR - VM ctest-TestBGPaasZone-96156374-43159470 has failed to come up 2025-09-11 03:22:10,593 - ERROR - Fault seen in nova show is: {'code': 500, 'created': '2025-09-11T03:22:07Z', 'message': 'No valid host was found. ', 'details': 'Traceback (most recent call last):\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 1581, in schedule_and_build_instances\n instance_uuids, return_alternates=True)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 942, in _schedule_instances\n return_alternates=return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations\n instance_uuids, return_objects, return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations\n return cctxt.call(ctxt, \'select_destinations\', **msg_args)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 192, in call\n retry=self.retry, transport_options=self.transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 691, in send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send\n raise result\nnova.exception_Remote.NoValidHost_Remote: No valid host was found. \nTraceback (most recent call last):\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner\n return func(*args, **kwargs)\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/manager.py", line 209, in select_destinations\n raise exception.NoValidHost(reason="")\n\nnova.exception.NoValidHost: No valid host was found. \n\n'} 2025-09-11 03:22:10,593 - INFO - Deleting VN ctest-cnz_vn-86152394 2025-09-11 03:22:10,639 - DEBUG - VN e11d74cb-ff91-46bd-b6d8-1aa9af673435 still in use: Unable to complete operation on network e11d74cb-ff91-46bd-b6d8-1aa9af673435. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-510bf467-ff87-4103-b253-9276f06f3a98'] 2025-09-11 03:22:10,640 - WARNING - Deleting VN ctest-cnz_vn-86152394 failed..Will retry 2025-09-11 03:22:12,808 - DEBUG - Response for deleting network () 2025-09-11 03:22:14,090 - ERROR - AssertionError Python 3.9.21: /usr/bin/python3 Thu Sep 11 03:22:10 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={}) 78 log.info('Initial checks done. Running the testcase now') 79 log.info('') 80 result = function(self, *args, **kwargs) 81 if self.inputs.upgrade: 82 pid = os.getpid() result = None function = self = args = () kwargs = {} /contrail-test/serial_scripts/bgpaas/test_bgpaas_zone.py in test_bgp_control_node_zones_from_single_vnf(self=) 145 self.logger.info('executing bgp_control_zone agent restart test') 146 host_ips = [] 147 assert self.basic_control_node_zone_setup('both-zones') 148 for vm in self.vms: 149 assert self.verify_bgpaas_in_control_nodes_and_agent(self.bgpaas_fixtures[cnt],vm) self = self.basic_control_node_zone_setup = > /contrail-test/serial_scripts/bgpaas/test_bgpaas_zone.py in basic_control_node_zone_setup(self=, ctrl_zone='both-zones') 31 self.vms.append(self.create_vm(vn_fixture=self.vn, image_name='ubuntu-bird')) 32 self.client_vm = self.create_vm(vn_fixture=self.vn, image_name='ubuntu-bird') 33 self.check_vms_booted(self.vms + [self.client_vm]) 34 for bgp_name in self.inputs.bgp_names: 35 self.cnzs += self.create_control_node_zones("test-zone",[bgp_name]) self = self.check_vms_booted = > self.vms = [, ] self.client_vm = /contrail-test/common/base.py in check_vms_booted(cls=, vms_list=[, , ], do_assert=True) 719 if failed and do_assert: 720 for vm_fixture in vms_list: 721 vm_fixture.cleanUp() 722 assert False, 'One or more vm-boots failed. Check logs' 723 if failed: vm_fixture = vm_fixture.cleanUp = > /contrail-test/fixtures/vm_test.py in cleanUp(self=) 2050 2051 def cleanUp(self): 2052 self.delete() 2053 super(VMFixture, self).cleanUp() 2054 self = self.delete = > /contrail-test/fixtures/vm_test.py in delete(self=, verify=False, force=False) 2080 self.webui.delete_vm(self) 2081 else: 2082 self.orch.delete_vm(vm_obj, force=force) 2083 self.vm_objs.remove(vm_obj) 2084 if self.inputs.ns_agilio_vrouter_data: self = self.orch = self.orch.delete_vm = > vm_obj = force = False /contrail-test/fixtures/openstack.py in delete_vm(self=, vm_obj=, **kwargs={'force': False}) 119 120 def delete_vm(self, vm_obj, **kwargs): 121 return self.nova_h.delete_vm(vm_obj, **kwargs) 122 123 def is_vm_deleted(self, vm_obj, **kwargs): self = self.nova_h = self.nova_h.delete_vm = > vm_obj = kwargs = {'force': False} /contrail-test/fixtures/nova_test.py in delete_vm(self=, vm_obj=, force=False) 763 def delete_vm(self, vm_obj, force=False): 764 if not force: 765 self.get_nova_host_of_vm(vm_obj) 766 vm_obj.delete() 767 # end _delete_vm self = self.get_nova_host_of_vm = > vm_obj = /contrail-test/fixtures/nova_test.py in get_nova_host_of_vm(self=, vm_obj=) 734 self.logger.error('VM %s has failed to come up' % vm_obj.name) 735 self.logger.error('Fault seen in nova show is: %s' % vm_obj.__dict__['fault']) 736 assert False, 'Fault seen in nova show %s is: %s' % ( 737 vm_obj.id, vm_obj.__dict__['fault']) 738 else: vm_obj = vm_obj.id = 'a9e70736-4614-47d5-bd12-ecb6f2acf177' vm_obj.__dict__ = {'OS-DCF:diskConfig': 'MANUAL', 'OS-EXT-AZ:availability_zone': 'nova', 'OS-EXT-SRV-ATTR:host': None, 'OS-EXT-SRV-ATTR:hypervisor_hostname': None, 'OS-EXT-SRV-ATTR:instance_name': 'instance-0000001b', 'OS-EXT-STS:power_state': 0, 'OS-EXT-STS:task_state': None, 'OS-EXT-STS:vm_state': 'error', 'OS-SRV-USG:launched_at': None, 'OS-SRV-USG:terminated_at': None, ...} AssertionError: Fault seen in nova show a9e70736-4614-47d5-bd12-ecb6f2acf177 is: {'code': 500, 'created': '2025-09-11T03:22:07Z', 'message': 'No valid host was found. ', 'details': 'Traceback (most recent call last):\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 1581, in schedule_and_build_instances\n instance_uuids, return_alternates=True)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 942, in _schedule_instances\n return_alternates=return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations\n instance_uuids, return_objects, return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations\n return cctxt.call(ctxt, \'select_destinations\', **msg_args)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 192, in call\n retry=self.retry, transport_options=self.transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 691, in send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send\n raise result\nnova.exception_Remote.NoValidHost_Remote: No valid host was found. \nTraceback (most recent call last):\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner\n return func(*args, **kwargs)\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/manager.py", line 209, in select_destinations\n raise exception.NoValidHost(reason="")\n\nnova.exception.NoValidHost: No valid host was found. \n\n'} __cause__ = None __class__ = __context__ = None __delattr__ = __dict__ = {} __dir__ = __doc__ = 'Assertion failed.' __eq__ = __format__ = __ge__ = __getattribute__ = __gt__ = __hash__ = __init__ = __init_subclass__ = __le__ = __lt__ = __ne__ = __new__ = __reduce__ = __reduce_ex__ = __repr__ = __setattr__ = __setstate__ = __sizeof__ = __str__ = __subclasshook__ = __suppress_context__ = False __traceback__ = args = (r"Fault seen in nova show a9e70736-4614-47d5-bd12-...tion.NoValidHost: No valid host was found. \n\n'}",) with_traceback = The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper result = function(self, *args, **kwargs) File "/contrail-test/serial_scripts/bgpaas/test_bgpaas_zone.py", line 147, in test_bgp_control_node_zones_from_single_vnf assert self.basic_control_node_zone_setup('both-zones') File "/contrail-test/serial_scripts/bgpaas/test_bgpaas_zone.py", line 33, in basic_control_node_zone_setup self.check_vms_booted(self.vms + [self.client_vm]) File "/contrail-test/common/base.py", line 721, in check_vms_booted vm_fixture.cleanUp() File "/contrail-test/fixtures/vm_test.py", line 2052, in cleanUp self.delete() File "/contrail-test/fixtures/vm_test.py", line 2082, in delete self.orch.delete_vm(vm_obj, force=force) File "/contrail-test/fixtures/openstack.py", line 121, in delete_vm return self.nova_h.delete_vm(vm_obj, **kwargs) File "/contrail-test/fixtures/nova_test.py", line 765, in delete_vm self.get_nova_host_of_vm(vm_obj) File "/contrail-test/fixtures/nova_test.py", line 736, in get_nova_host_of_vm assert False, 'Fault seen in nova show %s is: %s' % ( AssertionError: Fault seen in nova show a9e70736-4614-47d5-bd12-ecb6f2acf177 is: {'code': 500, 'created': '2025-09-11T03:22:07Z', 'message': 'No valid host was found. ', 'details': 'Traceback (most recent call last):\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 1581, in schedule_and_build_instances\n instance_uuids, return_alternates=True)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/conductor/manager.py", line 942, in _schedule_instances\n return_alternates=return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations\n instance_uuids, return_objects, return_alternates)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations\n return cctxt.call(ctxt, \'select_destinations\', **msg_args)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", line 192, in call\n retry=self.retry, transport_options=self.transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/transport.py", line 128, in _send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 691, in send\n transport_options=transport_options)\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send\n raise result\nnova.exception_Remote.NoValidHost_Remote: No valid host was found. \nTraceback (most recent call last):\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 241, in inner\n return func(*args, **kwargs)\n\n File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/scheduler/manager.py", line 209, in select_destinations\n raise exception.NoValidHost(reason="")\n\nnova.exception.NoValidHost: No valid host was found. \n\n'} 2025-09-11 03:22:14,091 - DEBUG - Skipping xmpp flap check 2025-09-11 03:22:14,091 - INFO - 2025-09-11 03:22:14,091 - INFO - END TEST : test_bgp_control_node_zones_from_single_vnf : FAILED[0:00:10] 2025-09-11 03:22:14,091 - INFO - -------------------------------------------------------------------------------- 2025-09-11 03:22:14,857 - INFO - Deleted project: ctest-TestBGPaasZone-96156374, ID : 635e2a47-ae48-497f-9484-30afb609d66d