2025-11-24 03:48:26,876 - INFO - Domain Default found not creating 2025-11-24 03:48:27,076 - INFO - Project ctest-TestBGPaasZone-60278317 not found, creating it 2025-11-24 03:48:27,637 - INFO - Created Project:ctest-TestBGPaasZone-60278317, ID : 159f83f9-5789-4418-a07b-6037040262e9 2025-11-24 03:48:29,335 - DEBUG - Services list from nova: [, , , ] 2025-11-24 03:48:29,335 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-4803-2', 'an-jenkins-deploy-platform-ansible-os-4803-1']} 2025-11-24 03:48:29,338 - INFO - ================================================================================ 2025-11-24 03:48:29,338 - INFO - STARTING TEST : test_bgp_control_node_zone 2025-11-24 03:48:30,610 - DEBUG - Skipping xmpp flap check 2025-11-24 03:48:30,610 - INFO - Initial checks done. Running the testcase now 2025-11-24 03:48:30,610 - INFO - 2025-11-24 03:48:30,610 - INFO - executing bgp_control_zone test 2025-11-24 03:48:31,423 - DEBUG - Response for create_network : {'network': {'id': '2efcaa77-1d7e-4792-9bda-4d24f3146b34', 'name': 'ctest-cnz_vn-90130667', 'tenant_id': '159f83f957894418a07b6037040262e9', 'project_id': '159f83f957894418a07b6037040262e9', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestBGPaasZone-60278317', 'ctest-cnz_vn-90130667'], 'port_security_enabled': True, 'description': ''}} 2025-11-24 03:48:31,844 - DEBUG - Response for create_subnet : {'subnet': {'id': 'e8330380-5430-48cb-8697-fe4cf31e027c', 'name': '', 'tenant_id': '159f83f957894418a07b6037040262e9', 'network_id': '2efcaa77-1d7e-4792-9bda-4d24f3146b34', 'ip_version': 4, 'cidr': '166.155.109.64/26', 'allocation_pools': [{'start': '166.155.109.66', 'end': '166.155.109.126'}], 'gateway_ip': '166.155.109.65', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '166.155.109.66', 'tags': [], 'project_id': '159f83f957894418a07b6037040262e9'}} 2025-11-24 03:48:31,879 - INFO - Created VN ctest-cnz_vn-90130667 2025-11-24 03:48:31,892 - DEBUG - VN ctest-cnz_vn-90130667 UUID is 2efcaa77-1d7e-4792-9bda-4d24f3146b34 2025-11-24 03:48:33,187 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4803-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4803-2) 2025-11-24 03:48:34,667 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4803-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4803-1) 2025-11-24 03:48:36,381 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4803-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4803-2) 2025-11-24 03:48:36,381 - INFO - Waiting for VM ctest-TestBGPaasZone-60278317-33757257 to be up.. 2025-11-24 03:48:36,500 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-11-24 03:48:41,605 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-11-24 03:48:46,707 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-11-24 03:48:51,795 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-11-24 03:48:56,891 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-11-24 03:49:01,978 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-11-24 03:49:07,093 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-11-24 03:49:12,199 - DEBUG - VM is in ACTIVE state now 2025-11-24 03:49:12,199 - INFO - VM name : ctest-TestBGPaasZone-60278317-33757257 2025-11-24 03:49:12,309 - DEBUG - VM ctest-TestBGPaasZone-60278317-33757257 ID is e9cfd3f8-978a-46d2-8385-5c5831e97615 2025-11-24 03:49:12,345 - DEBUG - VM ctest-TestBGPaasZone-60278317-33757257 launched on Node an-jenkins-deploy-platform-ansible-os-4803-2 2025-11-24 03:49:12,440 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-machine/e9cfd3f8-978a-46d2-8385-5c5831e97615 2025-11-24 03:49:12,769 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-machine/e9cfd3f8-978a-46d2-8385-5c5831e97615 2025-11-24 03:49:12,815 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-machine-interface/15426f15-8266-4dee-a727-adfd37f5666f 2025-11-24 03:49:16,005 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1014ms') 2025-11-24 03:49:16,005 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-33757257 failed! 2025-11-24 03:49:16,034 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-60278317:ctest-cnz_vn-90130667 is 166.155.109.65 and allocation pool is NOT set 2025-11-24 03:49:20,101 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1000ms') 2025-11-24 03:49:20,101 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-33757257 failed! 2025-11-24 03:49:20,115 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-60278317:ctest-cnz_vn-90130667 is 166.155.109.65 and allocation pool is NOT set 2025-11-24 03:49:24,203 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1017ms') 2025-11-24 03:49:24,203 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-33757257 failed! 2025-11-24 03:49:24,219 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-60278317:ctest-cnz_vn-90130667 is 166.155.109.65 and allocation pool is NOT set 2025-11-24 03:49:28,295 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1014ms') 2025-11-24 03:49:28,295 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-33757257 failed! 2025-11-24 03:49:28,310 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-60278317:ctest-cnz_vn-90130667 is 166.155.109.65 and allocation pool is NOT set 2025-11-24 03:49:32,393 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1018ms') 2025-11-24 03:49:32,393 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-33757257 failed! 2025-11-24 03:49:32,409 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-60278317:ctest-cnz_vn-90130667 is 166.155.109.65 and allocation pool is NOT set 2025-11-24 03:49:36,498 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1014ms') 2025-11-24 03:49:36,498 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-33757257 failed! 2025-11-24 03:49:36,522 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-60278317:ctest-cnz_vn-90130667 is 166.155.109.65 and allocation pool is NOT set 2025-11-24 03:49:40,585 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1003ms') 2025-11-24 03:49:40,585 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-33757257 failed! 2025-11-24 03:49:40,599 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-60278317:ctest-cnz_vn-90130667 is 166.155.109.65 and allocation pool is NOT set 2025-11-24 03:49:44,678 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1019ms') 2025-11-24 03:49:44,678 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-33757257 failed! 2025-11-24 03:49:44,693 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-60278317:ctest-cnz_vn-90130667 is 166.155.109.65 and allocation pool is NOT set 2025-11-24 03:49:48,779 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1023ms') 2025-11-24 03:49:48,779 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-33757257 failed! 2025-11-24 03:49:48,797 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-60278317:ctest-cnz_vn-90130667 is 166.155.109.65 and allocation pool is NOT set 2025-11-24 03:49:52,876 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1009ms') 2025-11-24 03:49:52,876 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-33757257 failed! 2025-11-24 03:49:52,894 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-60278317:ctest-cnz_vn-90130667 is 166.155.109.65 and allocation pool is NOT set 2025-11-24 03:49:56,969 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1011ms') 2025-11-24 03:49:56,969 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-33757257 failed! 2025-11-24 03:49:56,984 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-60278317:ctest-cnz_vn-90130667 is 166.155.109.65 and allocation pool is NOT set 2025-11-24 03:50:01,065 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1019ms') 2025-11-24 03:50:01,065 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-33757257 failed! 2025-11-24 03:50:01,083 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-60278317:ctest-cnz_vn-90130667 is 166.155.109.65 and allocation pool is NOT set 2025-11-24 03:50:05,159 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1013ms') 2025-11-24 03:50:05,159 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-33757257 failed! 2025-11-24 03:50:05,175 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-60278317:ctest-cnz_vn-90130667 is 166.155.109.65 and allocation pool is NOT set 2025-11-24 03:50:09,256 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1014ms') 2025-11-24 03:50:09,257 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-33757257 failed! 2025-11-24 03:50:09,285 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-60278317:ctest-cnz_vn-90130667 is 166.155.109.65 and allocation pool is NOT set 2025-11-24 03:50:13,348 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1006ms') 2025-11-24 03:50:13,348 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-33757257 failed! 2025-11-24 03:50:13,364 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-60278317:ctest-cnz_vn-90130667 is 166.155.109.65 and allocation pool is NOT set 2025-11-24 03:50:17,445 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1023ms') 2025-11-24 03:50:17,445 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-33757257 failed! 2025-11-24 03:50:17,463 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-60278317:ctest-cnz_vn-90130667 is 166.155.109.65 and allocation pool is NOT set 2025-11-24 03:50:21,545 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1014ms') 2025-11-24 03:50:21,545 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-33757257 failed! 2025-11-24 03:50:21,564 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-60278317:ctest-cnz_vn-90130667 is 166.155.109.65 and allocation pool is NOT set 2025-11-24 03:50:25,638 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1014ms') 2025-11-24 03:50:25,639 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-33757257 failed! 2025-11-24 03:50:25,654 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-60278317:ctest-cnz_vn-90130667 is 166.155.109.65 and allocation pool is NOT set 2025-11-24 03:50:29,739 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1021ms') 2025-11-24 03:50:29,739 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-33757257 failed! 2025-11-24 03:50:29,760 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-60278317:ctest-cnz_vn-90130667 is 166.155.109.65 and allocation pool is NOT set 2025-11-24 03:50:33,833 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=3.45 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 1 received, 50% packet loss, time 1009ms\r\nrtt min/avg/max/mdev = 3.454/3.454/3.454/0.000 ms') 2025-11-24 03:50:33,833 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-33757257 passed 2025-11-24 03:50:33,911 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:50:33,912 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-60278317-33757257, IP 166.155.109.67, Port 22 2025-11-24 03:50:33,980 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-24 03:50:34,104 - DEBUG - VM ctest-TestBGPaasZone-60278317-33757257 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-24 03:50:39,105 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:50:39,105 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-60278317-33757257, IP 166.155.109.67, Port 22 2025-11-24 03:50:39,172 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-24 03:50:39,278 - DEBUG - VM ctest-TestBGPaasZone-60278317-33757257 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-24 03:50:44,279 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:50:44,279 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-60278317-33757257, IP 166.155.109.67, Port 22 2025-11-24 03:50:44,352 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-24 03:50:44,444 - DEBUG - VM ctest-TestBGPaasZone-60278317-33757257 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-24 03:50:49,445 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:50:49,445 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-60278317-33757257, IP 166.155.109.67, Port 22 2025-11-24 03:50:49,504 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-24 03:50:49,590 - DEBUG - VM ctest-TestBGPaasZone-60278317-33757257 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-24 03:50:54,590 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:50:54,591 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-60278317-33757257, IP 166.155.109.67, Port 22 2025-11-24 03:50:54,655 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-24 03:50:54,755 - DEBUG - VM ctest-TestBGPaasZone-60278317-33757257 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-24 03:50:59,756 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:50:59,756 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-60278317-33757257, IP 166.155.109.67, Port 22 2025-11-24 03:50:59,829 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-24 03:50:59,916 - DEBUG - VM ctest-TestBGPaasZone-60278317-33757257 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-24 03:51:04,917 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:51:04,917 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-60278317-33757257, IP 166.155.109.67, Port 22 2025-11-24 03:51:04,987 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-24 03:51:05,085 - DEBUG - VM ctest-TestBGPaasZone-60278317-33757257 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-24 03:51:10,085 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:51:10,085 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-60278317-33757257, IP 166.155.109.67, Port 22 2025-11-24 03:51:10,152 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-24 03:51:10,242 - DEBUG - VM ctest-TestBGPaasZone-60278317-33757257 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-24 03:51:15,243 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:51:15,244 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-60278317-33757257, IP 166.155.109.67, Port 22 2025-11-24 03:51:15,316 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-24 03:51:15,418 - DEBUG - VM ctest-TestBGPaasZone-60278317-33757257 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-24 03:51:20,419 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:51:20,419 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-60278317-33757257, IP 166.155.109.67, Port 22 2025-11-24 03:51:20,492 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-24 03:51:20,586 - DEBUG - VM ctest-TestBGPaasZone-60278317-33757257 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-24 03:51:25,587 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:51:25,587 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-60278317-33757257, IP 166.155.109.67, Port 22 2025-11-24 03:51:25,645 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-24 03:51:25,738 - DEBUG - VM ctest-TestBGPaasZone-60278317-33757257 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-24 03:51:30,739 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:51:30,739 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-60278317-33757257, IP 166.155.109.67, Port 22 2025-11-24 03:51:30,923 - DEBUG - VM ctest-TestBGPaasZone-60278317-33757257 is ready for SSH connections 2025-11-24 03:51:30,923 - INFO - Waiting for VM ctest-TestBGPaasZone-60278317-60464411 to be up.. 2025-11-24 03:51:31,028 - DEBUG - VM is in ACTIVE state now 2025-11-24 03:51:31,028 - INFO - VM name : ctest-TestBGPaasZone-60278317-60464411 2025-11-24 03:51:31,126 - DEBUG - VM ctest-TestBGPaasZone-60278317-60464411 ID is 1f2a5470-1088-4403-accf-054298d09381 2025-11-24 03:51:31,126 - DEBUG - VM ctest-TestBGPaasZone-60278317-60464411 launched on Node an-jenkins-deploy-platform-ansible-os-4803-1 2025-11-24 03:51:31,209 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-machine/1f2a5470-1088-4403-accf-054298d09381 2025-11-24 03:51:31,221 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-machine-interface/dc1d4093-0977-4608-8301-602c9eb5bfb8 2025-11-24 03:51:32,382 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=3.65 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=0.655 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 0.655/2.150/3.646/1.495 ms') 2025-11-24 03:51:32,382 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-60464411 passed 2025-11-24 03:51:32,455 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:51:32,455 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-60278317-60464411, IP 166.155.109.68, Port 22 2025-11-24 03:51:32,632 - DEBUG - VM ctest-TestBGPaasZone-60278317-60464411 is ready for SSH connections 2025-11-24 03:51:32,632 - INFO - Waiting for VM ctest-TestBGPaasZone-60278317-92012404 to be up.. 2025-11-24 03:51:32,718 - DEBUG - VM is in ACTIVE state now 2025-11-24 03:51:32,718 - INFO - VM name : ctest-TestBGPaasZone-60278317-92012404 2025-11-24 03:51:32,806 - DEBUG - VM ctest-TestBGPaasZone-60278317-92012404 ID is 59f4d336-dfd6-48ec-a760-bf9603c5d99a 2025-11-24 03:51:32,806 - DEBUG - VM ctest-TestBGPaasZone-60278317-92012404 launched on Node an-jenkins-deploy-platform-ansible-os-4803-2 2025-11-24 03:51:32,895 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-machine/59f4d336-dfd6-48ec-a760-bf9603c5d99a 2025-11-24 03:51:32,905 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-machine-interface/5220ae5b-b56a-4619-9f4d-62114ea42ab9 2025-11-24 03:51:34,076 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n64 bytes from 169.254.0.4: icmp_seq=1 ttl=63 time=2.57 ms\r\n64 bytes from 169.254.0.4: icmp_seq=2 ttl=63 time=0.829 ms\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 0.829/1.701/2.573/0.872 ms') 2025-11-24 03:51:34,077 - INFO - Ping to Metadata IP 169.254.0.4 of VM ctest-TestBGPaasZone-60278317-92012404 passed 2025-11-24 03:51:34,151 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:51:34,151 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-60278317-92012404, IP 166.155.109.69, Port 22 2025-11-24 03:51:34,323 - DEBUG - VM ctest-TestBGPaasZone-60278317-92012404 is ready for SSH connections 2025-11-24 03:51:34,470 - INFO - Created control node zone ctest-test-zone-89713063(13635d0f-edf1-4923-b117-d1afb5c80e73) 2025-11-24 03:51:34,834 - INFO - Created control node zone ctest-test-zone-23214100(6934463f-fd49-438d-a72c-df5183d76ada) 2025-11-24 03:51:35,194 - INFO - Created control node zone ctest-test-zone-72605752(f1c9b4df-a731-49c4-bc94-899a441da084) 2025-11-24 03:51:35,502 - INFO - Creating BGPaaS ctest-TestBGPaasZone-60278317-95214390 2025-11-24 03:51:35,586 - INFO - BGPaaS: ctest-TestBGPaasZone-60278317-95214390(ecf5d4ea-0a67-4c0a-b70e-8a604a010aba) 2025-11-24 03:51:35,586 - DEBUG - Requesting: http://10.0.0.41:8082/bgp-as-a-service/ecf5d4ea-0a67-4c0a-b70e-8a604a010aba 2025-11-24 03:51:35,891 - DEBUG - Requesting: http://10.0.0.41:8082/bgp-as-a-service/ecf5d4ea-0a67-4c0a-b70e-8a604a010aba 2025-11-24 03:51:35,939 - INFO - verify_in_api_server passed for BGPaaS obj ecf5d4ea-0a67-4c0a-b70e-8a604a010aba 2025-11-24 03:51:35,939 - INFO - BGPaaS(ecf5d4ea-0a67-4c0a-b70e-8a604a010aba): verify_on_setup passed 2025-11-24 03:51:35,939 - INFO - We will configure BGP on the VM 2025-11-24 03:51:36,218 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:51:36,218 - INFO - Configuring BGP on ctest-TestBGPaasZone-60278317-33757257 2025-11-24 03:51:36,218 - DEBUG - Running remote_cmd, Cmd : cat > /etc/bird/bird.conf << EOS protocol device { scan time 10; # Scan interfaces every 10 seconds } protocol kernel { persist; # Don't remove routes on bird shutdown scan time 20; # Scan kernel routing table every 20 seconds import all; # Default is import all #export all; # Default is export none } protocol direct { interface "eth*"; } router id 166.155.109.67; protocol bgp bgp_1{ local as 65000; neighbor 166.155.109.65 as 64512; export where source = RTS_STATIC; export all; multihop; #export all; hold time 90; bfd on; source address 166.155.109.67; } protocol bfd { neighbor 166.155.109.65 local 166.155.109.67 multihop on; } EOS , host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.56, gateway password: c0ntrail123 2025-11-24 03:51:36,219 - DEBUG - cat > /etc/bird/bird.conf << EOS protocol device { scan time 10; # Scan interfaces every 10 seconds } protocol kernel { persist; # Don't remove routes on bird shutdown scan time 20; # Scan kernel routing table every 20 seconds import all; # Default is import all #export all; # Default is export none } protocol direct { interface "eth*"; } router id 166.155.109.67; protocol bgp bgp_1{ local as 65000; neighbor 166.155.109.65 as 64512; export where source = RTS_STATIC; export all; multihop; #export all; hold time 90; bfd on; source address 166.155.109.67; } protocol bfd { neighbor 166.155.109.65 local 166.155.109.67 multihop on; } EOS 2025-11-24 03:51:51,522 - DEBUG - None 2025-11-24 03:51:51,523 - DEBUG - Running remote_cmd, Cmd : cat > /etc/bird/bird.conf << EOS protocol device { scan time 10; # Scan interfaces every 10 seconds } protocol kernel { persist; # Don't remove routes on bird shutdown scan time 20; # Scan kernel routing table every 20 seconds import all; # Default is import all #export all; # Default is export none } protocol direct { interface "eth*"; } router id 166.155.109.67; protocol bgp bgp_1{ local as 65000; neighbor 166.155.109.65 as 64512; export where source = RTS_STATIC; export all; multihop; #export all; hold time 90; bfd on; source address 166.155.109.67; } protocol bfd { neighbor 166.155.109.65 local 166.155.109.67 multihop on; } EOS , host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.56, gateway password: c0ntrail123 2025-11-24 03:51:51,523 - DEBUG - cat > /etc/bird/bird.conf << EOS protocol device { scan time 10; # Scan interfaces every 10 seconds } protocol kernel { persist; # Don't remove routes on bird shutdown scan time 20; # Scan kernel routing table every 20 seconds import all; # Default is import all #export all; # Default is export none } protocol direct { interface "eth*"; } router id 166.155.109.67; protocol bgp bgp_1{ local as 65000; neighbor 166.155.109.65 as 64512; export where source = RTS_STATIC; export all; multihop; #export all; hold time 90; bfd on; source address 166.155.109.67; } protocol bfd { neighbor 166.155.109.65 local 166.155.109.67 multihop on; } EOS 2025-11-24 03:51:52,089 - DEBUG - None 2025-11-24 03:51:52,090 - DEBUG - Running remote_cmd, Cmd : service bird restart, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.56, gateway password: c0ntrail123 2025-11-24 03:51:52,090 - DEBUG - service bird restart 2025-11-24 03:51:53,928 - DEBUG - bird stop/waiting bird start/running, process 1439 2025-11-24 03:51:53,929 - INFO - Attaching VMI 15426f15-8266-4dee-a727-adfd37f5666f to BGPaaS ecf5d4ea-0a67-4c0a-b70e-8a604a010aba 2025-11-24 03:51:54,010 - INFO - Attaching the VMI e9cfd3f8-978a-46d2-8385-5c5831e97615 to the BGPaaS ecf5d4ea-0a67-4c0a-b70e-8a604a010aba object 2025-11-24 03:51:54,010 - DEBUG - Running remote_cmd, Cmd : sudo ip addr add 166.155.109.84 dev eth0, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.56, gateway password: c0ntrail123 2025-11-24 03:51:54,010 - DEBUG - sudo ip addr add 166.155.109.84 dev eth0 2025-11-24 03:51:54,792 - DEBUG - None 2025-11-24 03:51:54,862 - INFO - Creating BGPaaS ctest-TestBGPaasZone-60278317-92886897 2025-11-24 03:51:54,946 - INFO - BGPaaS: ctest-TestBGPaasZone-60278317-92886897(8f15dead-a14a-4f69-a315-a94b3d6e0a3e) 2025-11-24 03:51:54,946 - DEBUG - Requesting: http://10.0.0.41:8082/bgp-as-a-service/8f15dead-a14a-4f69-a315-a94b3d6e0a3e 2025-11-24 03:51:54,957 - INFO - verify_in_api_server passed for BGPaaS obj 8f15dead-a14a-4f69-a315-a94b3d6e0a3e 2025-11-24 03:51:54,957 - INFO - BGPaaS(8f15dead-a14a-4f69-a315-a94b3d6e0a3e): verify_on_setup passed 2025-11-24 03:51:54,957 - INFO - We will configure BGP on the VM 2025-11-24 03:51:55,218 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:51:55,218 - INFO - Configuring BGP on ctest-TestBGPaasZone-60278317-60464411 2025-11-24 03:51:55,219 - DEBUG - Running remote_cmd, Cmd : cat > /etc/bird/bird.conf << EOS protocol device { scan time 10; # Scan interfaces every 10 seconds } protocol kernel { persist; # Don't remove routes on bird shutdown scan time 20; # Scan kernel routing table every 20 seconds import all; # Default is import all #export all; # Default is export none } protocol direct { interface "eth*"; } router id 166.155.109.68; protocol bgp bgp_1{ local as 65000; neighbor 166.155.109.66 as 64512; export where source = RTS_STATIC; export all; multihop; #export all; hold time 90; bfd on; source address 166.155.109.68; } protocol bfd { neighbor 166.155.109.66 local 166.155.109.68 multihop on; } EOS , host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.57, gateway password: c0ntrail123 2025-11-24 03:51:55,219 - DEBUG - cat > /etc/bird/bird.conf << EOS protocol device { scan time 10; # Scan interfaces every 10 seconds } protocol kernel { persist; # Don't remove routes on bird shutdown scan time 20; # Scan kernel routing table every 20 seconds import all; # Default is import all #export all; # Default is export none } protocol direct { interface "eth*"; } router id 166.155.109.68; protocol bgp bgp_1{ local as 65000; neighbor 166.155.109.66 as 64512; export where source = RTS_STATIC; export all; multihop; #export all; hold time 90; bfd on; source address 166.155.109.68; } protocol bfd { neighbor 166.155.109.66 local 166.155.109.68 multihop on; } EOS 2025-11-24 03:52:10,975 - DEBUG - None 2025-11-24 03:52:10,975 - DEBUG - Running remote_cmd, Cmd : cat > /etc/bird/bird.conf << EOS protocol device { scan time 10; # Scan interfaces every 10 seconds } protocol kernel { persist; # Don't remove routes on bird shutdown scan time 20; # Scan kernel routing table every 20 seconds import all; # Default is import all #export all; # Default is export none } protocol direct { interface "eth*"; } router id 166.155.109.68; protocol bgp bgp_1{ local as 65000; neighbor 166.155.109.66 as 64512; export where source = RTS_STATIC; export all; multihop; #export all; hold time 90; bfd on; source address 166.155.109.68; } protocol bfd { neighbor 166.155.109.66 local 166.155.109.68 multihop on; } EOS , host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.57, gateway password: c0ntrail123 2025-11-24 03:52:10,975 - DEBUG - cat > /etc/bird/bird.conf << EOS protocol device { scan time 10; # Scan interfaces every 10 seconds } protocol kernel { persist; # Don't remove routes on bird shutdown scan time 20; # Scan kernel routing table every 20 seconds import all; # Default is import all #export all; # Default is export none } protocol direct { interface "eth*"; } router id 166.155.109.68; protocol bgp bgp_1{ local as 65000; neighbor 166.155.109.66 as 64512; export where source = RTS_STATIC; export all; multihop; #export all; hold time 90; bfd on; source address 166.155.109.68; } protocol bfd { neighbor 166.155.109.66 local 166.155.109.68 multihop on; } EOS 2025-11-24 03:52:11,576 - DEBUG - None 2025-11-24 03:52:11,576 - DEBUG - Running remote_cmd, Cmd : service bird restart, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.57, gateway password: c0ntrail123 2025-11-24 03:52:11,576 - DEBUG - service bird restart 2025-11-24 03:52:13,523 - DEBUG - bird stop/waiting bird start/running, process 1442 2025-11-24 03:52:13,524 - INFO - Attaching VMI dc1d4093-0977-4608-8301-602c9eb5bfb8 to BGPaaS 8f15dead-a14a-4f69-a315-a94b3d6e0a3e 2025-11-24 03:52:13,602 - INFO - Attaching the VMI 1f2a5470-1088-4403-accf-054298d09381 to the BGPaaS 8f15dead-a14a-4f69-a315-a94b3d6e0a3e object 2025-11-24 03:52:13,602 - DEBUG - Running remote_cmd, Cmd : sudo ip addr add 166.155.109.84 dev eth0, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.57, gateway password: c0ntrail123 2025-11-24 03:52:13,602 - DEBUG - sudo ip addr add 166.155.109.84 dev eth0 2025-11-24 03:52:14,354 - DEBUG - None 2025-11-24 03:52:14,354 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:52:14,371 - INFO - BGPaaS session seen in control-node 10.20.0.25 2025-11-24 03:52:14,378 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:52:14,398 - ERROR - Not enough BGPaaS sessions seen in control-nodes ['10.20.0.193'] 2025-11-24 03:52:20,421 - ERROR - Not enough BGPaaS sessions seen in control-nodes ['10.20.0.193'] 2025-11-24 03:52:26,443 - INFO - BGPaaS session seen in control-node 10.20.0.193 2025-11-24 03:52:26,451 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 166.155.109.84, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.56, gateway password: c0ntrail123 2025-11-24 03:52:26,451 - DEBUG - ping -s 56 -c 3 -W 1 166.155.109.84 2025-11-24 03:52:38,156 - DEBUG - PING 166.155.109.84 (166.155.109.84) 56(84) bytes of data. 64 bytes from 166.155.109.84: icmp_seq=1 ttl=64 time=10.3 ms 64 bytes from 166.155.109.84: icmp_seq=2 ttl=64 time=1.65 ms 64 bytes from 166.155.109.84: icmp_seq=3 ttl=64 time=1.44 ms --- 166.155.109.84 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2006ms rtt min/avg/max/mdev = 1.444/4.482/10.354/4.153 ms 2025-11-24 03:52:38,156 - INFO - Ping to IP 166.155.109.84 from VM ctest-TestBGPaasZone-60278317-92012404 passed 2025-11-24 03:52:39,058 - DEBUG - Running remote_cmd, Cmd : service bird restart, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.56, gateway password: c0ntrail123 2025-11-24 03:52:39,058 - DEBUG - service bird restart 2025-11-24 03:52:40,942 - DEBUG - bird stop/waiting bird start/running, process 1466 2025-11-24 03:52:40,943 - DEBUG - Running remote_cmd, Cmd : service bird restart, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.57, gateway password: c0ntrail123 2025-11-24 03:52:40,943 - DEBUG - service bird restart 2025-11-24 03:52:42,846 - DEBUG - bird stop/waiting bird start/running, process 1469 2025-11-24 03:52:42,846 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:52:42,846 - DEBUG - Requesting: http://10.20.0.129:8083/Snh_BgpNeighborReq?domain=&ip_address= 2025-11-24 03:52:42,872 - ERROR - Not enough BGPaaS sessions seen in control-nodes ['10.20.0.129'] 2025-11-24 03:52:48,873 - DEBUG - Requesting: http://10.20.0.129:8083/Snh_BgpNeighborReq?domain=&ip_address= 2025-11-24 03:52:48,889 - INFO - BGPaaS session seen in control-node 10.20.0.129 2025-11-24 03:52:48,895 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:52:48,915 - INFO - BGPaaS session seen in control-node 10.20.0.193 2025-11-24 03:52:48,923 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 166.155.109.84, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.56, gateway password: c0ntrail123 2025-11-24 03:52:48,923 - DEBUG - ping -s 56 -c 3 -W 1 166.155.109.84 2025-11-24 03:52:51,434 - DEBUG - PING 166.155.109.84 (166.155.109.84) 56(84) bytes of data. 64 bytes from 166.155.109.84: icmp_seq=1 ttl=64 time=7.34 ms 64 bytes from 166.155.109.84: icmp_seq=2 ttl=64 time=1.10 ms 64 bytes from 166.155.109.84: icmp_seq=3 ttl=64 time=1.04 ms --- 166.155.109.84 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2005ms rtt min/avg/max/mdev = 1.041/3.164/7.343/2.955 ms 2025-11-24 03:52:51,434 - INFO - Ping to IP 166.155.109.84 from VM ctest-TestBGPaasZone-60278317-92012404 passed 2025-11-24 03:52:51,731 - INFO - We will configure BGP on the VM 2025-11-24 03:52:51,946 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:52:51,946 - INFO - Configuring BGP on ctest-TestBGPaasZone-60278317-33757257 2025-11-24 03:52:51,946 - DEBUG - Running remote_cmd, Cmd : cat > /etc/bird/bird.conf << EOS protocol device { scan time 10; # Scan interfaces every 10 seconds } protocol kernel { persist; # Don't remove routes on bird shutdown scan time 20; # Scan kernel routing table every 20 seconds import all; # Default is import all #export all; # Default is export none } protocol direct { interface "eth*"; } router id 166.155.109.67; protocol bgp bgp_1{ local as 65000; neighbor 166.155.109.65 as 64512; export where source = RTS_STATIC; export all; multihop; #export all; hold time 90; bfd on; source address 166.155.109.67; } protocol bfd { neighbor 166.155.109.65 local 166.155.109.67 multihop on; } EOS , host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.56, gateway password: c0ntrail123 2025-11-24 03:52:51,946 - DEBUG - cat > /etc/bird/bird.conf << EOS protocol device { scan time 10; # Scan interfaces every 10 seconds } protocol kernel { persist; # Don't remove routes on bird shutdown scan time 20; # Scan kernel routing table every 20 seconds import all; # Default is import all #export all; # Default is export none } protocol direct { interface "eth*"; } router id 166.155.109.67; protocol bgp bgp_1{ local as 65000; neighbor 166.155.109.65 as 64512; export where source = RTS_STATIC; export all; multihop; #export all; hold time 90; bfd on; source address 166.155.109.67; } protocol bfd { neighbor 166.155.109.65 local 166.155.109.67 multihop on; } EOS 2025-11-24 03:52:52,476 - DEBUG - None 2025-11-24 03:52:52,477 - DEBUG - Running remote_cmd, Cmd : cat > /etc/bird/bird.conf << EOS protocol device { scan time 10; # Scan interfaces every 10 seconds } protocol kernel { persist; # Don't remove routes on bird shutdown scan time 20; # Scan kernel routing table every 20 seconds import all; # Default is import all #export all; # Default is export none } protocol direct { interface "eth*"; } router id 166.155.109.67; protocol bgp bgp_1{ local as 65000; neighbor 166.155.109.65 as 64512; export where source = RTS_STATIC; export all; multihop; #export all; hold time 90; bfd on; source address 166.155.109.67; } protocol bfd { neighbor 166.155.109.65 local 166.155.109.67 multihop on; } EOS , host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.56, gateway password: c0ntrail123 2025-11-24 03:52:52,477 - DEBUG - cat > /etc/bird/bird.conf << EOS protocol device { scan time 10; # Scan interfaces every 10 seconds } protocol kernel { persist; # Don't remove routes on bird shutdown scan time 20; # Scan kernel routing table every 20 seconds import all; # Default is import all #export all; # Default is export none } protocol direct { interface "eth*"; } router id 166.155.109.67; protocol bgp bgp_1{ local as 65000; neighbor 166.155.109.65 as 64512; export where source = RTS_STATIC; export all; multihop; #export all; hold time 90; bfd on; source address 166.155.109.67; } protocol bfd { neighbor 166.155.109.65 local 166.155.109.67 multihop on; } EOS 2025-11-24 03:52:53,008 - DEBUG - None 2025-11-24 03:52:53,008 - DEBUG - Running remote_cmd, Cmd : service bird restart, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.56, gateway password: c0ntrail123 2025-11-24 03:52:53,008 - DEBUG - service bird restart 2025-11-24 03:52:54,858 - DEBUG - bird stop/waiting bird start/running, process 1498 2025-11-24 03:52:54,858 - INFO - Attaching VMI 15426f15-8266-4dee-a727-adfd37f5666f to BGPaaS ecf5d4ea-0a67-4c0a-b70e-8a604a010aba 2025-11-24 03:52:54,880 - INFO - Attaching the VMI e9cfd3f8-978a-46d2-8385-5c5831e97615 to the BGPaaS ecf5d4ea-0a67-4c0a-b70e-8a604a010aba object 2025-11-24 03:52:54,880 - DEBUG - Running remote_cmd, Cmd : sudo ip addr add 166.155.109.84 dev eth0, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.56, gateway password: c0ntrail123 2025-11-24 03:52:54,880 - DEBUG - sudo ip addr add 166.155.109.84 dev eth0 2025-11-24 03:52:55,644 - DEBUG - RTNETLINK answers: File exists 2025-11-24 03:52:55,644 - DEBUG - Running remote_cmd, Cmd : service bird restart, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.56, gateway password: c0ntrail123 2025-11-24 03:52:55,644 - DEBUG - service bird restart 2025-11-24 03:52:57,394 - DEBUG - bird stop/waiting bird start/running, process 1525 2025-11-24 03:52:57,394 - DEBUG - Running remote_cmd, Cmd : service bird restart, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.57, gateway password: c0ntrail123 2025-11-24 03:52:57,394 - DEBUG - service bird restart 2025-11-24 03:52:59,250 - DEBUG - bird stop/waiting bird start/running, process 1491 2025-11-24 03:52:59,250 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:52:59,272 - ERROR - Not enough BGPaaS sessions seen in control-nodes ['10.20.0.25'] 2025-11-24 03:53:05,296 - INFO - BGPaaS session seen in control-node 10.20.0.25 2025-11-24 03:53:05,304 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:53:05,325 - INFO - BGPaaS session seen in control-node 10.20.0.193 2025-11-24 03:53:05,332 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 166.155.109.84, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.56, gateway password: c0ntrail123 2025-11-24 03:53:05,332 - DEBUG - ping -s 56 -c 3 -W 1 166.155.109.84 2025-11-24 03:53:07,843 - DEBUG - PING 166.155.109.84 (166.155.109.84) 56(84) bytes of data. 64 bytes from 166.155.109.84: icmp_seq=1 ttl=64 time=2.03 ms 64 bytes from 166.155.109.84: icmp_seq=2 ttl=64 time=1.27 ms 64 bytes from 166.155.109.84: icmp_seq=3 ttl=64 time=1.12 ms --- 166.155.109.84 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2003ms rtt min/avg/max/mdev = 1.129/1.477/2.032/0.398 ms 2025-11-24 03:53:07,844 - INFO - Ping to IP 166.155.109.84 from VM ctest-TestBGPaasZone-60278317-92012404 passed 2025-11-24 03:53:07,844 - INFO - Detaching VMI 15426f15-8266-4dee-a727-adfd37f5666f from BGPaaS ecf5d4ea-0a67-4c0a-b70e-8a604a010aba 2025-11-24 03:53:07,918 - INFO - Detaching VMI dc1d4093-0977-4608-8301-602c9eb5bfb8 from BGPaaS 8f15dead-a14a-4f69-a315-a94b3d6e0a3e 2025-11-24 03:53:08,010 - INFO - Deleting BGPaaS ctest-TestBGPaasZone-60278317-92886897(8f15dead-a14a-4f69-a315-a94b3d6e0a3e) 2025-11-24 03:53:08,084 - DEBUG - Requesting: http://10.0.0.41:8082/bgp-as-a-service/8f15dead-a14a-4f69-a315-a94b3d6e0a3e 2025-11-24 03:53:08,099 - DEBUG - Response Code: 404 2025-11-24 03:53:08,099 - DEBUG - BGPaaS: 8f15dead-a14a-4f69-a315-a94b3d6e0a3e deleted from api server 2025-11-24 03:53:08,099 - INFO - BGPaaS(8f15dead-a14a-4f69-a315-a94b3d6e0a3e): verify_on_cleanup passed 2025-11-24 03:53:08,099 - INFO - Detaching VMI 15426f15-8266-4dee-a727-adfd37f5666f from BGPaaS ecf5d4ea-0a67-4c0a-b70e-8a604a010aba 2025-11-24 03:53:08,138 - INFO - Deleting BGPaaS ctest-TestBGPaasZone-60278317-95214390(ecf5d4ea-0a67-4c0a-b70e-8a604a010aba) 2025-11-24 03:53:08,230 - DEBUG - Requesting: http://10.0.0.41:8082/bgp-as-a-service/ecf5d4ea-0a67-4c0a-b70e-8a604a010aba 2025-11-24 03:53:08,240 - DEBUG - Response Code: 404 2025-11-24 03:53:08,241 - DEBUG - BGPaaS: ecf5d4ea-0a67-4c0a-b70e-8a604a010aba deleted from api server 2025-11-24 03:53:08,241 - INFO - BGPaaS(ecf5d4ea-0a67-4c0a-b70e-8a604a010aba): verify_on_cleanup passed 2025-11-24 03:53:08,242 - INFO - Deleting Control node zone ctest-test-zone-72605752(f1c9b4df-a731-49c4-bc94-899a441da084) 2025-11-24 03:53:08,423 - INFO - Deleting Control node zone ctest-test-zone-23214100(6934463f-fd49-438d-a72c-df5183d76ada) 2025-11-24 03:53:08,585 - INFO - Deleting Control node zone ctest-test-zone-89713063(13635d0f-edf1-4923-b117-d1afb5c80e73) 2025-11-24 03:53:08,697 - INFO - Deleting VM ctest-TestBGPaasZone-60278317-92012404 2025-11-24 03:53:08,786 - INFO - Deleting VM ctest-TestBGPaasZone-60278317-60464411 2025-11-24 03:53:08,862 - INFO - Deleting VM ctest-TestBGPaasZone-60278317-33757257 2025-11-24 03:53:08,959 - INFO - Deleting VN ctest-cnz_vn-90130667 2025-11-24 03:53:09,011 - DEBUG - VN 2efcaa77-1d7e-4792-9bda-4d24f3146b34 still in use: Unable to complete operation on network 2efcaa77-1d7e-4792-9bda-4d24f3146b34. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-f01ef9c7-e6ba-49ad-966b-ea4e9d281ba0'] 2025-11-24 03:53:09,011 - WARNING - Deleting VN ctest-cnz_vn-90130667 failed..Will retry 2025-11-24 03:53:11,247 - DEBUG - Response for deleting network () 2025-11-24 03:53:12,545 - DEBUG - Skipping xmpp flap check 2025-11-24 03:53:12,546 - INFO - END TEST : test_bgp_control_node_zone : PASSED[0:04:43] 2025-11-24 03:53:12,546 - INFO - -------------------------------------------------------------------------------- 2025-11-24 03:53:12,549 - INFO - ================================================================================ 2025-11-24 03:53:12,549 - INFO - STARTING TEST : test_bgp_control_node_zones_from_single_vnf 2025-11-24 03:53:13,818 - DEBUG - Skipping xmpp flap check 2025-11-24 03:53:13,818 - INFO - Initial checks done. Running the testcase now 2025-11-24 03:53:13,818 - INFO - 2025-11-24 03:53:13,818 - INFO - executing bgp_control_zone agent restart test 2025-11-24 03:53:14,139 - DEBUG - Response for create_network : {'network': {'id': '03a6ffcc-b3de-4e3c-a5cf-cdb0c208e6b1', 'name': 'ctest-cnz_vn-41352727', 'tenant_id': '159f83f957894418a07b6037040262e9', 'project_id': '159f83f957894418a07b6037040262e9', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestBGPaasZone-60278317', 'ctest-cnz_vn-41352727'], 'port_security_enabled': True, 'description': ''}} 2025-11-24 03:53:14,394 - DEBUG - Response for create_subnet : {'subnet': {'id': '6fd7f775-a36d-4a9a-8095-89a1dce38b1f', 'name': '', 'tenant_id': '159f83f957894418a07b6037040262e9', 'network_id': '03a6ffcc-b3de-4e3c-a5cf-cdb0c208e6b1', 'ip_version': 4, 'cidr': '1.24.16.128/26', 'allocation_pools': [{'start': '1.24.16.130', 'end': '1.24.16.190'}], 'gateway_ip': '1.24.16.129', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '1.24.16.130', 'tags': [], 'project_id': '159f83f957894418a07b6037040262e9'}} 2025-11-24 03:53:14,420 - INFO - Created VN ctest-cnz_vn-41352727 2025-11-24 03:53:14,434 - DEBUG - VN ctest-cnz_vn-41352727 UUID is 03a6ffcc-b3de-4e3c-a5cf-cdb0c208e6b1 2025-11-24 03:53:15,857 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4803-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4803-1) 2025-11-24 03:53:17,329 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4803-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4803-2) 2025-11-24 03:53:18,758 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4803-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4803-1) 2025-11-24 03:53:18,758 - INFO - Waiting for VM ctest-TestBGPaasZone-60278317-07761855 to be up.. 2025-11-24 03:53:18,856 - DEBUG - VM is in ACTIVE state now 2025-11-24 03:53:18,857 - INFO - VM name : ctest-TestBGPaasZone-60278317-07761855 2025-11-24 03:53:18,953 - DEBUG - VM ctest-TestBGPaasZone-60278317-07761855 ID is eb0909af-68b5-48e2-9d0c-b7615fc4f0b9 2025-11-24 03:53:18,953 - DEBUG - VM ctest-TestBGPaasZone-60278317-07761855 launched on Node an-jenkins-deploy-platform-ansible-os-4803-1 2025-11-24 03:53:19,054 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-machine/eb0909af-68b5-48e2-9d0c-b7615fc4f0b9 2025-11-24 03:53:19,065 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-machine-interface/e15bebfc-e738-45b1-81f2-f3cbce746b8a 2025-11-24 03:53:22,268 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1006ms') 2025-11-24 03:53:22,268 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-07761855 failed! 2025-11-24 03:53:22,288 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-60278317:ctest-cnz_vn-41352727 is 1.24.16.129 and allocation pool is NOT set 2025-11-24 03:53:26,364 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1004ms') 2025-11-24 03:53:26,364 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-07761855 failed! 2025-11-24 03:53:26,379 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-60278317:ctest-cnz_vn-41352727 is 1.24.16.129 and allocation pool is NOT set 2025-11-24 03:53:30,465 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1013ms') 2025-11-24 03:53:30,465 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-07761855 failed! 2025-11-24 03:53:30,521 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-60278317:ctest-cnz_vn-41352727 is 1.24.16.129 and allocation pool is NOT set 2025-11-24 03:53:34,615 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1035ms') 2025-11-24 03:53:34,616 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-07761855 failed! 2025-11-24 03:53:34,631 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-60278317:ctest-cnz_vn-41352727 is 1.24.16.129 and allocation pool is NOT set 2025-11-24 03:53:38,720 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1021ms') 2025-11-24 03:53:38,721 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-07761855 failed! 2025-11-24 03:53:38,737 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-60278317:ctest-cnz_vn-41352727 is 1.24.16.129 and allocation pool is NOT set 2025-11-24 03:53:42,812 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1009ms') 2025-11-24 03:53:42,812 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-07761855 failed! 2025-11-24 03:53:42,835 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-60278317:ctest-cnz_vn-41352727 is 1.24.16.129 and allocation pool is NOT set 2025-11-24 03:53:46,913 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1006ms') 2025-11-24 03:53:46,913 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-07761855 failed! 2025-11-24 03:53:46,928 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-60278317:ctest-cnz_vn-41352727 is 1.24.16.129 and allocation pool is NOT set 2025-11-24 03:53:51,007 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1014ms') 2025-11-24 03:53:51,007 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-07761855 failed! 2025-11-24 03:53:51,023 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-60278317:ctest-cnz_vn-41352727 is 1.24.16.129 and allocation pool is NOT set 2025-11-24 03:53:55,100 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1015ms') 2025-11-24 03:53:55,100 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-07761855 failed! 2025-11-24 03:53:55,118 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-60278317:ctest-cnz_vn-41352727 is 1.24.16.129 and allocation pool is NOT set 2025-11-24 03:53:59,192 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1019ms') 2025-11-24 03:53:59,192 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-07761855 failed! 2025-11-24 03:53:59,210 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-60278317:ctest-cnz_vn-41352727 is 1.24.16.129 and allocation pool is NOT set 2025-11-24 03:54:03,291 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1019ms') 2025-11-24 03:54:03,291 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-07761855 failed! 2025-11-24 03:54:03,308 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-60278317:ctest-cnz_vn-41352727 is 1.24.16.129 and allocation pool is NOT set 2025-11-24 03:54:07,385 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1016ms') 2025-11-24 03:54:07,385 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-07761855 failed! 2025-11-24 03:54:07,401 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-60278317:ctest-cnz_vn-41352727 is 1.24.16.129 and allocation pool is NOT set 2025-11-24 03:54:11,485 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1020ms') 2025-11-24 03:54:11,485 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-07761855 failed! 2025-11-24 03:54:11,505 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-60278317:ctest-cnz_vn-41352727 is 1.24.16.129 and allocation pool is NOT set 2025-11-24 03:54:15,581 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1013ms') 2025-11-24 03:54:15,581 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-07761855 failed! 2025-11-24 03:54:15,598 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-60278317:ctest-cnz_vn-41352727 is 1.24.16.129 and allocation pool is NOT set 2025-11-24 03:54:19,672 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1018ms') 2025-11-24 03:54:19,672 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-07761855 failed! 2025-11-24 03:54:19,685 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-60278317:ctest-cnz_vn-41352727 is 1.24.16.129 and allocation pool is NOT set 2025-11-24 03:54:23,773 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1021ms') 2025-11-24 03:54:23,773 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-07761855 failed! 2025-11-24 03:54:23,790 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-60278317:ctest-cnz_vn-41352727 is 1.24.16.129 and allocation pool is NOT set 2025-11-24 03:54:27,865 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1018ms') 2025-11-24 03:54:27,866 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-07761855 failed! 2025-11-24 03:54:27,881 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-60278317:ctest-cnz_vn-41352727 is 1.24.16.129 and allocation pool is NOT set 2025-11-24 03:54:31,970 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1020ms') 2025-11-24 03:54:31,970 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-07761855 failed! 2025-11-24 03:54:31,988 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-60278317:ctest-cnz_vn-41352727 is 1.24.16.129 and allocation pool is NOT set 2025-11-24 03:54:36,060 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1010ms') 2025-11-24 03:54:36,060 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-07761855 failed! 2025-11-24 03:54:36,077 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-60278317:ctest-cnz_vn-41352727 is 1.24.16.129 and allocation pool is NOT set 2025-11-24 03:54:40,155 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1013ms') 2025-11-24 03:54:40,155 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-07761855 failed! 2025-11-24 03:54:40,174 - DEBUG - Gateway for vn default-domain:ctest-TestBGPaasZone-60278317:ctest-cnz_vn-41352727 is 1.24.16.129 and allocation pool is NOT set 2025-11-24 03:54:44,250 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=7.04 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 1 received, 50% packet loss, time 1013ms\r\nrtt min/avg/max/mdev = 7.043/7.043/7.043/0.000 ms') 2025-11-24 03:54:44,250 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-07761855 passed 2025-11-24 03:54:44,329 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:54:44,329 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-60278317-07761855, IP 1.24.16.131, Port 22 2025-11-24 03:54:44,386 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-24 03:54:44,562 - DEBUG - VM ctest-TestBGPaasZone-60278317-07761855 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-24 03:54:49,563 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:54:49,563 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-60278317-07761855, IP 1.24.16.131, Port 22 2025-11-24 03:54:49,630 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-24 03:54:49,757 - DEBUG - VM ctest-TestBGPaasZone-60278317-07761855 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-24 03:54:54,757 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:54:54,757 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-60278317-07761855, IP 1.24.16.131, Port 22 2025-11-24 03:54:54,825 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-24 03:54:54,915 - DEBUG - VM ctest-TestBGPaasZone-60278317-07761855 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-24 03:54:59,915 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:54:59,916 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-60278317-07761855, IP 1.24.16.131, Port 22 2025-11-24 03:54:59,985 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-24 03:55:00,073 - DEBUG - VM ctest-TestBGPaasZone-60278317-07761855 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-24 03:55:05,074 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:55:05,074 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-60278317-07761855, IP 1.24.16.131, Port 22 2025-11-24 03:55:05,141 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-24 03:55:05,250 - DEBUG - VM ctest-TestBGPaasZone-60278317-07761855 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-24 03:55:10,251 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:55:10,251 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-60278317-07761855, IP 1.24.16.131, Port 22 2025-11-24 03:55:10,321 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-24 03:55:10,412 - DEBUG - VM ctest-TestBGPaasZone-60278317-07761855 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-24 03:55:15,413 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:55:15,413 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-60278317-07761855, IP 1.24.16.131, Port 22 2025-11-24 03:55:15,472 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-24 03:55:15,610 - DEBUG - VM ctest-TestBGPaasZone-60278317-07761855 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-24 03:55:20,611 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:55:20,611 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-60278317-07761855, IP 1.24.16.131, Port 22 2025-11-24 03:55:20,680 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-24 03:55:20,769 - DEBUG - VM ctest-TestBGPaasZone-60278317-07761855 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-24 03:55:25,770 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:55:25,770 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-60278317-07761855, IP 1.24.16.131, Port 22 2025-11-24 03:55:25,837 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-24 03:55:25,936 - DEBUG - VM ctest-TestBGPaasZone-60278317-07761855 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-24 03:55:30,939 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:55:30,939 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-60278317-07761855, IP 1.24.16.131, Port 22 2025-11-24 03:55:31,009 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-24 03:55:31,120 - DEBUG - VM ctest-TestBGPaasZone-60278317-07761855 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-24 03:55:36,121 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:55:36,121 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-60278317-07761855, IP 1.24.16.131, Port 22 2025-11-24 03:55:36,179 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-24 03:55:36,312 - DEBUG - VM ctest-TestBGPaasZone-60278317-07761855 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-24 03:55:41,313 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:55:41,313 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-60278317-07761855, IP 1.24.16.131, Port 22 2025-11-24 03:55:41,497 - DEBUG - VM ctest-TestBGPaasZone-60278317-07761855 is ready for SSH connections 2025-11-24 03:55:41,497 - INFO - Waiting for VM ctest-TestBGPaasZone-60278317-57148097 to be up.. 2025-11-24 03:55:41,592 - DEBUG - VM is in ACTIVE state now 2025-11-24 03:55:41,592 - INFO - VM name : ctest-TestBGPaasZone-60278317-57148097 2025-11-24 03:55:41,684 - DEBUG - VM ctest-TestBGPaasZone-60278317-57148097 ID is 2d8c92cb-f9e1-4bf0-b78e-c8cfedabed8c 2025-11-24 03:55:41,684 - DEBUG - VM ctest-TestBGPaasZone-60278317-57148097 launched on Node an-jenkins-deploy-platform-ansible-os-4803-2 2025-11-24 03:55:41,791 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-machine/2d8c92cb-f9e1-4bf0-b78e-c8cfedabed8c 2025-11-24 03:55:41,841 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-machine-interface/013b6726-bc34-424c-ae14-94c171914a34 2025-11-24 03:55:43,010 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=4.38 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=0.959 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 0.959/2.669/4.380/1.710 ms') 2025-11-24 03:55:43,010 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBGPaasZone-60278317-57148097 passed 2025-11-24 03:55:43,088 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:55:43,088 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-60278317-57148097, IP 1.24.16.132, Port 22 2025-11-24 03:55:43,255 - DEBUG - VM ctest-TestBGPaasZone-60278317-57148097 is ready for SSH connections 2025-11-24 03:55:43,256 - INFO - Waiting for VM ctest-TestBGPaasZone-60278317-17505159 to be up.. 2025-11-24 03:55:43,351 - DEBUG - VM is in ACTIVE state now 2025-11-24 03:55:43,351 - INFO - VM name : ctest-TestBGPaasZone-60278317-17505159 2025-11-24 03:55:43,453 - DEBUG - VM ctest-TestBGPaasZone-60278317-17505159 ID is ed5ec4bd-979d-404e-ac38-0789273a71ba 2025-11-24 03:55:43,453 - DEBUG - VM ctest-TestBGPaasZone-60278317-17505159 launched on Node an-jenkins-deploy-platform-ansible-os-4803-1 2025-11-24 03:55:43,580 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-machine/ed5ec4bd-979d-404e-ac38-0789273a71ba 2025-11-24 03:55:43,593 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-machine-interface/cfb07c29-f671-4c10-a0e1-362e161d0460 2025-11-24 03:55:44,766 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n64 bytes from 169.254.0.4: icmp_seq=1 ttl=63 time=5.42 ms\r\n64 bytes from 169.254.0.4: icmp_seq=2 ttl=63 time=1.81 ms\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 1.812/3.617/5.422/1.805 ms') 2025-11-24 03:55:44,766 - INFO - Ping to Metadata IP 169.254.0.4 of VM ctest-TestBGPaasZone-60278317-17505159 passed 2025-11-24 03:55:44,840 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:55:44,840 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-60278317-17505159, IP 1.24.16.133, Port 22 2025-11-24 03:55:44,909 - DEBUG - Error on ssh to ubuntu@169.254.0.4:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.4/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.4/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.4/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-11-24 03:55:44,993 - DEBUG - VM ctest-TestBGPaasZone-60278317-17505159 is NOT ready for SSH connections, VM status: ACTIVE 2025-11-24 03:55:49,994 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:55:49,994 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-60278317-17505159, IP 1.24.16.133, Port 22 2025-11-24 03:55:50,164 - DEBUG - VM ctest-TestBGPaasZone-60278317-17505159 is ready for SSH connections 2025-11-24 03:55:50,302 - INFO - Created control node zone ctest-test-zone-22078245(5b401dac-ab92-4dce-a6a0-2e2357797763) 2025-11-24 03:55:50,666 - INFO - Created control node zone ctest-test-zone-38977604(c75d543b-3d0e-4361-a57d-157726b6d7c7) 2025-11-24 03:55:51,030 - INFO - Created control node zone ctest-test-zone-89679406(3d25fcc0-4641-41cc-ab09-9cdedc4f2126) 2025-11-24 03:55:51,318 - INFO - Creating BGPaaS ctest-TestBGPaasZone-60278317-42354465 2025-11-24 03:55:51,394 - INFO - BGPaaS: ctest-TestBGPaasZone-60278317-42354465(c5078227-14b6-4da1-8a03-45f9188f9eb3) 2025-11-24 03:55:51,394 - DEBUG - Requesting: http://10.0.0.41:8082/bgp-as-a-service/c5078227-14b6-4da1-8a03-45f9188f9eb3 2025-11-24 03:55:51,405 - INFO - verify_in_api_server passed for BGPaaS obj c5078227-14b6-4da1-8a03-45f9188f9eb3 2025-11-24 03:55:51,405 - INFO - BGPaaS(c5078227-14b6-4da1-8a03-45f9188f9eb3): verify_on_setup passed 2025-11-24 03:55:51,405 - INFO - We will configure BGP on the VM 2025-11-24 03:55:51,830 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:55:51,830 - INFO - Configuring BGP on ctest-TestBGPaasZone-60278317-07761855 2025-11-24 03:55:51,830 - DEBUG - Running remote_cmd, Cmd : cat > /etc/bird/bird.conf << EOS protocol device { scan time 10; # Scan interfaces every 10 seconds } protocol kernel { persist; # Don't remove routes on bird shutdown scan time 20; # Scan kernel routing table every 20 seconds import all; # Default is import all #export all; # Default is export none } protocol direct { interface "eth*"; } router id 1.24.16.131; protocol bgp bgp_1{ local as 65000; neighbor 1.24.16.129 as 64512; export where source = RTS_STATIC; export all; multihop; #export all; hold time 90; bfd on; source address 1.24.16.131; } protocol bfd { neighbor 1.24.16.129 local 1.24.16.131 multihop on; } EOS , host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.57, gateway password: c0ntrail123 2025-11-24 03:55:51,830 - DEBUG - cat > /etc/bird/bird.conf << EOS protocol device { scan time 10; # Scan interfaces every 10 seconds } protocol kernel { persist; # Don't remove routes on bird shutdown scan time 20; # Scan kernel routing table every 20 seconds import all; # Default is import all #export all; # Default is export none } protocol direct { interface "eth*"; } router id 1.24.16.131; protocol bgp bgp_1{ local as 65000; neighbor 1.24.16.129 as 64512; export where source = RTS_STATIC; export all; multihop; #export all; hold time 90; bfd on; source address 1.24.16.131; } protocol bfd { neighbor 1.24.16.129 local 1.24.16.131 multihop on; } EOS 2025-11-24 03:56:06,620 - DEBUG - None 2025-11-24 03:56:06,620 - DEBUG - Running remote_cmd, Cmd : cat >> /etc/bird/bird.conf << EOS protocol bgp bgp_2{ local as 65000; neighbor 1.24.16.130 as 64512; export where source = RTS_STATIC; export all; multihop; #export all; hold time 90; bfd on; source address 1.24.16.131; } EOS , host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.57, gateway password: c0ntrail123 2025-11-24 03:56:06,620 - DEBUG - cat >> /etc/bird/bird.conf << EOS protocol bgp bgp_2{ local as 65000; neighbor 1.24.16.130 as 64512; export where source = RTS_STATIC; export all; multihop; #export all; hold time 90; bfd on; source address 1.24.16.131; } EOS 2025-11-24 03:56:07,825 - DEBUG - None 2025-11-24 03:56:07,825 - DEBUG - Running remote_cmd, Cmd : service bird restart, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.57, gateway password: c0ntrail123 2025-11-24 03:56:07,825 - DEBUG - service bird restart 2025-11-24 03:56:10,194 - DEBUG - bird stop/waiting bird start/running, process 1445 2025-11-24 03:56:10,194 - INFO - Attaching VMI e15bebfc-e738-45b1-81f2-f3cbce746b8a to BGPaaS c5078227-14b6-4da1-8a03-45f9188f9eb3 2025-11-24 03:56:10,278 - INFO - Attaching the VMI eb0909af-68b5-48e2-9d0c-b7615fc4f0b9 to the BGPaaS c5078227-14b6-4da1-8a03-45f9188f9eb3 object 2025-11-24 03:56:10,278 - DEBUG - Running remote_cmd, Cmd : sudo ip addr add 1.24.16.148 dev eth0, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.57, gateway password: c0ntrail123 2025-11-24 03:56:10,278 - DEBUG - sudo ip addr add 1.24.16.148 dev eth0 2025-11-24 03:56:11,086 - DEBUG - None 2025-11-24 03:56:11,150 - INFO - Creating BGPaaS ctest-TestBGPaasZone-60278317-98753109 2025-11-24 03:56:11,226 - INFO - BGPaaS: ctest-TestBGPaasZone-60278317-98753109(03001c8a-26b6-421d-9ee7-1978888a8c0a) 2025-11-24 03:56:11,226 - DEBUG - Requesting: http://10.0.0.41:8082/bgp-as-a-service/03001c8a-26b6-421d-9ee7-1978888a8c0a 2025-11-24 03:56:11,236 - INFO - verify_in_api_server passed for BGPaaS obj 03001c8a-26b6-421d-9ee7-1978888a8c0a 2025-11-24 03:56:11,237 - INFO - BGPaaS(03001c8a-26b6-421d-9ee7-1978888a8c0a): verify_on_setup passed 2025-11-24 03:56:11,237 - INFO - We will configure BGP on the VM 2025-11-24 03:56:11,678 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:56:11,678 - INFO - Configuring BGP on ctest-TestBGPaasZone-60278317-57148097 2025-11-24 03:56:11,678 - DEBUG - Running remote_cmd, Cmd : cat > /etc/bird/bird.conf << EOS protocol device { scan time 10; # Scan interfaces every 10 seconds } protocol kernel { persist; # Don't remove routes on bird shutdown scan time 20; # Scan kernel routing table every 20 seconds import all; # Default is import all #export all; # Default is export none } protocol direct { interface "eth*"; } router id 1.24.16.132; protocol bgp bgp_1{ local as 65000; neighbor 1.24.16.129 as 64512; export where source = RTS_STATIC; export all; multihop; #export all; hold time 90; bfd on; source address 1.24.16.132; } protocol bfd { neighbor 1.24.16.129 local 1.24.16.132 multihop on; } EOS , host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.56, gateway password: c0ntrail123 2025-11-24 03:56:11,679 - DEBUG - cat > /etc/bird/bird.conf << EOS protocol device { scan time 10; # Scan interfaces every 10 seconds } protocol kernel { persist; # Don't remove routes on bird shutdown scan time 20; # Scan kernel routing table every 20 seconds import all; # Default is import all #export all; # Default is export none } protocol direct { interface "eth*"; } router id 1.24.16.132; protocol bgp bgp_1{ local as 65000; neighbor 1.24.16.129 as 64512; export where source = RTS_STATIC; export all; multihop; #export all; hold time 90; bfd on; source address 1.24.16.132; } protocol bfd { neighbor 1.24.16.129 local 1.24.16.132 multihop on; } EOS 2025-11-24 03:56:17,098 - DEBUG - None 2025-11-24 03:56:17,098 - DEBUG - Running remote_cmd, Cmd : cat >> /etc/bird/bird.conf << EOS protocol bgp bgp_2{ local as 65000; neighbor 1.24.16.130 as 64512; export where source = RTS_STATIC; export all; multihop; #export all; hold time 90; bfd on; source address 1.24.16.132; } EOS , host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.56, gateway password: c0ntrail123 2025-11-24 03:56:17,098 - DEBUG - cat >> /etc/bird/bird.conf << EOS protocol bgp bgp_2{ local as 65000; neighbor 1.24.16.130 as 64512; export where source = RTS_STATIC; export all; multihop; #export all; hold time 90; bfd on; source address 1.24.16.132; } EOS 2025-11-24 03:56:17,658 - DEBUG - None 2025-11-24 03:56:17,658 - DEBUG - Running remote_cmd, Cmd : service bird restart, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.56, gateway password: c0ntrail123 2025-11-24 03:56:17,658 - DEBUG - service bird restart 2025-11-24 03:56:19,532 - DEBUG - bird stop/waiting bird start/running, process 1444 2025-11-24 03:56:19,532 - INFO - Attaching VMI 013b6726-bc34-424c-ae14-94c171914a34 to BGPaaS 03001c8a-26b6-421d-9ee7-1978888a8c0a 2025-11-24 03:56:19,610 - INFO - Attaching the VMI 2d8c92cb-f9e1-4bf0-b78e-c8cfedabed8c to the BGPaaS 03001c8a-26b6-421d-9ee7-1978888a8c0a object 2025-11-24 03:56:19,610 - DEBUG - Running remote_cmd, Cmd : sudo ip addr add 1.24.16.148 dev eth0, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.56, gateway password: c0ntrail123 2025-11-24 03:56:19,610 - DEBUG - sudo ip addr add 1.24.16.148 dev eth0 2025-11-24 03:56:20,348 - DEBUG - None 2025-11-24 03:56:20,348 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:56:20,362 - INFO - BGPaaS session seen in control-node 10.20.0.25 2025-11-24 03:56:20,363 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:56:20,382 - INFO - BGPaaS session seen in control-node 10.20.0.193 2025-11-24 03:56:20,389 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:56:20,403 - ERROR - Not enough BGPaaS sessions seen in control-nodes ['10.20.0.25'] 2025-11-24 03:56:26,421 - ERROR - Not enough BGPaaS sessions seen in control-nodes ['10.20.0.25'] 2025-11-24 03:56:32,440 - INFO - BGPaaS session seen in control-node 10.20.0.25 2025-11-24 03:56:32,440 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:56:32,459 - INFO - BGPaaS session seen in control-node 10.20.0.193 2025-11-24 03:56:32,465 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 1.24.16.148, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.57, gateway password: c0ntrail123 2025-11-24 03:56:32,465 - DEBUG - ping -s 56 -c 3 -W 1 1.24.16.148 2025-11-24 03:56:39,627 - DEBUG - PING 1.24.16.148 (1.24.16.148) 56(84) bytes of data. 64 bytes from 1.24.16.148: icmp_seq=1 ttl=64 time=8.25 ms 64 bytes from 1.24.16.148: icmp_seq=2 ttl=64 time=1.21 ms 64 bytes from 1.24.16.148: icmp_seq=3 ttl=64 time=1.44 ms --- 1.24.16.148 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2004ms rtt min/avg/max/mdev = 1.215/3.640/8.259/3.267 ms 2025-11-24 03:56:39,627 - INFO - Ping to IP 1.24.16.148 from VM ctest-TestBGPaasZone-60278317-17505159 passed 2025-11-24 03:56:39,628 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:56:39,641 - INFO - BGPaaS session seen in control-node 10.20.0.25 2025-11-24 03:56:39,642 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:56:39,663 - INFO - BGPaaS session seen in control-node 10.20.0.193 2025-11-24 03:56:39,670 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:56:39,684 - INFO - BGPaaS session seen in control-node 10.20.0.25 2025-11-24 03:56:39,684 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-11-24 03:56:39,705 - INFO - BGPaaS session seen in control-node 10.20.0.193 2025-11-24 03:56:39,711 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 1.24.16.148, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.57, gateway password: c0ntrail123 2025-11-24 03:56:39,711 - DEBUG - ping -s 56 -c 3 -W 1 1.24.16.148 2025-11-24 03:56:42,235 - DEBUG - PING 1.24.16.148 (1.24.16.148) 56(84) bytes of data. 64 bytes from 1.24.16.148: icmp_seq=1 ttl=64 time=5.11 ms 64 bytes from 1.24.16.148: icmp_seq=2 ttl=64 time=1.58 ms 64 bytes from 1.24.16.148: icmp_seq=3 ttl=64 time=1.67 ms --- 1.24.16.148 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2004ms rtt min/avg/max/mdev = 1.587/2.793/5.119/1.645 ms 2025-11-24 03:56:42,236 - INFO - Ping to IP 1.24.16.148 from VM ctest-TestBGPaasZone-60278317-17505159 passed 2025-11-24 03:56:42,236 - INFO - Detaching VMI 013b6726-bc34-424c-ae14-94c171914a34 from BGPaaS 03001c8a-26b6-421d-9ee7-1978888a8c0a 2025-11-24 03:56:42,318 - INFO - Deleting BGPaaS ctest-TestBGPaasZone-60278317-98753109(03001c8a-26b6-421d-9ee7-1978888a8c0a) 2025-11-24 03:56:42,411 - DEBUG - Requesting: http://10.0.0.41:8082/bgp-as-a-service/03001c8a-26b6-421d-9ee7-1978888a8c0a 2025-11-24 03:56:42,462 - DEBUG - Response Code: 404 2025-11-24 03:56:42,463 - DEBUG - BGPaaS: 03001c8a-26b6-421d-9ee7-1978888a8c0a deleted from api server 2025-11-24 03:56:42,463 - INFO - BGPaaS(03001c8a-26b6-421d-9ee7-1978888a8c0a): verify_on_cleanup passed 2025-11-24 03:56:42,463 - INFO - Detaching VMI e15bebfc-e738-45b1-81f2-f3cbce746b8a from BGPaaS c5078227-14b6-4da1-8a03-45f9188f9eb3 2025-11-24 03:56:42,550 - INFO - Deleting BGPaaS ctest-TestBGPaasZone-60278317-42354465(c5078227-14b6-4da1-8a03-45f9188f9eb3) 2025-11-24 03:56:42,612 - DEBUG - Requesting: http://10.0.0.41:8082/bgp-as-a-service/c5078227-14b6-4da1-8a03-45f9188f9eb3 2025-11-24 03:56:42,619 - DEBUG - Response Code: 404 2025-11-24 03:56:42,619 - DEBUG - BGPaaS: c5078227-14b6-4da1-8a03-45f9188f9eb3 deleted from api server 2025-11-24 03:56:42,619 - INFO - BGPaaS(c5078227-14b6-4da1-8a03-45f9188f9eb3): verify_on_cleanup passed 2025-11-24 03:56:42,620 - INFO - Deleting Control node zone ctest-test-zone-89679406(3d25fcc0-4641-41cc-ab09-9cdedc4f2126) 2025-11-24 03:56:42,775 - INFO - Deleting Control node zone ctest-test-zone-38977604(c75d543b-3d0e-4361-a57d-157726b6d7c7) 2025-11-24 03:56:42,941 - INFO - Deleting Control node zone ctest-test-zone-22078245(5b401dac-ab92-4dce-a6a0-2e2357797763) 2025-11-24 03:56:43,098 - INFO - Deleting VM ctest-TestBGPaasZone-60278317-17505159 2025-11-24 03:56:43,183 - INFO - Deleting VM ctest-TestBGPaasZone-60278317-57148097 2025-11-24 03:56:43,276 - INFO - Deleting VM ctest-TestBGPaasZone-60278317-07761855 2025-11-24 03:56:43,349 - INFO - Deleting VN ctest-cnz_vn-41352727 2025-11-24 03:56:43,397 - DEBUG - VN 03a6ffcc-b3de-4e3c-a5cf-cdb0c208e6b1 still in use: Unable to complete operation on network 03a6ffcc-b3de-4e3c-a5cf-cdb0c208e6b1. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-6a423cbc-c9c9-4f1c-a6b8-969717c4fa2e'] 2025-11-24 03:56:43,397 - WARNING - Deleting VN ctest-cnz_vn-41352727 failed..Will retry 2025-11-24 03:56:45,685 - DEBUG - Response for deleting network () 2025-11-24 03:56:46,946 - DEBUG - Skipping xmpp flap check 2025-11-24 03:56:46,946 - INFO - END TEST : test_bgp_control_node_zones_from_single_vnf : PASSED[0:03:34] 2025-11-24 03:56:46,946 - INFO - -------------------------------------------------------------------------------- 2025-11-24 03:56:47,726 - INFO - Deleted project: ctest-TestBGPaasZone-60278317, ID : 159f83f9-5789-4418-a07b-6037040262e9