is in ACTIVE state now
2026-02-05 04:07:01,560 - INFO - VM name : ctest-TestBGPaasZone-84798581-37377857
2026-02-05 04:07:01,661 - DEBUG - VM ctest-TestBGPaasZone-84798581-37377857 ID is ae681a02-6a62-49bf-9803-4834982e0aca
2026-02-05 04:07:01,661 - DEBUG - VM ctest-TestBGPaasZone-84798581-37377857 launched on Node an-jenkins-deploy-platform-ansible-os-5215-2
2026-02-05 04:07:01,756 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/ae681a02-6a62-49bf-9803-4834982e0aca
2026-02-05 04:07:01,767 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/b187ab67-ba21-4db8-acf2-12b4210fdca9
2026-02-05 04:07:02,940 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n64 bytes from 169.254.0.4: icmp_seq=1 ttl=63 time=3.65 ms\r\n64 bytes from 169.254.0.4: icmp_seq=2 ttl=63 time=1.37 ms\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1002ms\r\nrtt min/avg/max/mdev = 1.373/2.511/3.649/1.138 ms')
2026-02-05 04:07:02,940 - INFO - Ping to Metadata IP 169.254.0.4 of VM ctest-TestBGPaasZone-84798581-37377857 passed
2026-02-05 04:07:03,014 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2026-02-05 04:07:03,015 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-84798581-37377857, IP 137.119.124.133, Port 22
2026-02-05 04:07:03,086 - DEBUG - Error on ssh to ubuntu@169.254.0.4:22, result: /bin/bash: connect: Connection refused
/bin/bash: line 1: /dev/tcp/169.254.0.4/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.4/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.4/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''}
2026-02-05 04:07:03,191 - DEBUG - VM ctest-TestBGPaasZone-84798581-37377857 is NOT ready for SSH connections, VM status: ACTIVE
2026-02-05 04:07:08,191 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2026-02-05 04:07:08,192 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-84798581-37377857, IP 137.119.124.133, Port 22
2026-02-05 04:07:08,257 - DEBUG - Error on ssh to ubuntu@169.254.0.4:22, result: /bin/bash: connect: Connection refused
/bin/bash: line 1: /dev/tcp/169.254.0.4/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.4/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.4/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''}
2026-02-05 04:07:08,359 - DEBUG - VM ctest-TestBGPaasZone-84798581-37377857 is NOT ready for SSH connections, VM status: ACTIVE
2026-02-05 04:07:13,360 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2026-02-05 04:07:13,360 - DEBUG - Waiting to SSH to VM ctest-TestBGPaasZone-84798581-37377857, IP 137.119.124.133, Port 22
2026-02-05 04:07:13,533 - DEBUG - VM ctest-TestBGPaasZone-84798581-37377857 is ready for SSH connections
2026-02-05 04:07:13,676 - INFO - Created control node zone ctest-test-zone-69707665(ad3d90e6-71ca-4002-a64a-eb91ccaab03c)
2026-02-05 04:07:14,044 - INFO - Created control node zone ctest-test-zone-61382938(94f50c2a-82dd-4e3e-975f-0a85534d605b)
2026-02-05 04:07:14,424 - INFO - Created control node zone ctest-test-zone-48173757(0fb9aff7-d256-43f3-ac21-47a3b30b6b1d)
2026-02-05 04:07:14,744 - INFO - Creating BGPaaS ctest-TestBGPaasZone-84798581-83276880
2026-02-05 04:07:14,832 - INFO - BGPaaS: ctest-TestBGPaasZone-84798581-83276880(c2f85db0-76a2-4712-be29-0e8186bb9dcf)
2026-02-05 04:07:14,832 - DEBUG - Requesting: http://10.0.0.27:8082/bgp-as-a-service/c2f85db0-76a2-4712-be29-0e8186bb9dcf
2026-02-05 04:07:14,842 - INFO - verify_in_api_server passed for BGPaaS obj c2f85db0-76a2-4712-be29-0e8186bb9dcf
2026-02-05 04:07:14,843 - INFO - BGPaaS(c2f85db0-76a2-4712-be29-0e8186bb9dcf): verify_on_setup passed
2026-02-05 04:07:14,843 - INFO - We will configure BGP on the VM
2026-02-05 04:07:15,284 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2026-02-05 04:07:15,284 - INFO - Configuring BGP on ctest-TestBGPaasZone-84798581-96216851
2026-02-05 04:07:15,284 - DEBUG - Running remote_cmd, Cmd : cat > /etc/bird/bird.conf << EOS
protocol device {
scan time 10; # Scan interfaces every 10 seconds
}
protocol kernel {
persist; # Don't remove routes on bird shutdown
scan time 20; # Scan kernel routing table every 20 seconds
import all; # Default is import all
#export all; # Default is export none
}
protocol direct {
interface "eth*";
}
router id 137.119.124.131;
protocol bgp bgp_1{
local as 65000;
neighbor 137.119.124.129 as 64512;
export where source = RTS_STATIC;
export all;
multihop;
#export all;
hold time 90;
bfd on;
source address 137.119.124.131;
}
protocol bfd {
neighbor 137.119.124.129 local 137.119.124.131 multihop on;
}
EOS
, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.65, gateway password: c0ntrail123
2026-02-05 04:07:15,284 - DEBUG - cat > /etc/bird/bird.conf << EOS
protocol device {
scan time 10; # Scan interfaces every 10 seconds
}
protocol kernel {
persist; # Don't remove routes on bird shutdown
scan time 20; # Scan kernel routing table every 20 seconds
import all; # Default is import all
#export all; # Default is export none
}
protocol direct {
interface "eth*";
}
router id 137.119.124.131;
protocol bgp bgp_1{
local as 65000;
neighbor 137.119.124.129 as 64512;
export where source = RTS_STATIC;
export all;
multihop;
#export all;
hold time 90;
bfd on;
source address 137.119.124.131;
}
protocol bfd {
neighbor 137.119.124.129 local 137.119.124.131 multihop on;
}
EOS
2026-02-05 04:07:22,496 - DEBUG - None
2026-02-05 04:07:22,496 - DEBUG - Running remote_cmd, Cmd : cat >> /etc/bird/bird.conf << EOS
protocol bgp bgp_2{
local as 65000;
neighbor 137.119.124.130 as 64512;
export where source = RTS_STATIC;
export all;
multihop;
#export all;
hold time 90;
bfd on;
source address 137.119.124.131;
}
EOS
, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.65, gateway password: c0ntrail123
2026-02-05 04:07:22,496 - DEBUG - cat >> /etc/bird/bird.conf << EOS
protocol bgp bgp_2{
local as 65000;
neighbor 137.119.124.130 as 64512;
export where source = RTS_STATIC;
export all;
multihop;
#export all;
hold time 90;
bfd on;
source address 137.119.124.131;
}
EOS
2026-02-05 04:07:23,029 - DEBUG - None
2026-02-05 04:07:23,030 - DEBUG - Running remote_cmd, Cmd : service bird restart, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.65, gateway password: c0ntrail123
2026-02-05 04:07:23,030 - DEBUG - service bird restart
2026-02-05 04:07:24,825 - DEBUG - bird stop/waiting
bird start/running, process 1434
2026-02-05 04:07:24,825 - INFO - Attaching VMI 92ae5a83-a73a-4d37-a353-e60c1dc404bc to BGPaaS c2f85db0-76a2-4712-be29-0e8186bb9dcf
2026-02-05 04:07:24,908 - INFO - Attaching the VMI 4968afac-3357-477a-ad06-59a3a89d38f4 to the BGPaaS c2f85db0-76a2-4712-be29-0e8186bb9dcf object
2026-02-05 04:07:24,908 - DEBUG - Running remote_cmd, Cmd : sudo ip addr add 137.119.124.148 dev eth0, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.65, gateway password: c0ntrail123
2026-02-05 04:07:24,908 - DEBUG - sudo ip addr add 137.119.124.148 dev eth0
2026-02-05 04:07:25,603 - DEBUG - None
2026-02-05 04:07:25,672 - INFO - Creating BGPaaS ctest-TestBGPaasZone-84798581-46266610
2026-02-05 04:07:25,756 - INFO - BGPaaS: ctest-TestBGPaasZone-84798581-46266610(c407824d-ca33-4e95-b9ae-627048bec02c)
2026-02-05 04:07:25,756 - DEBUG - Requesting: http://10.0.0.27:8082/bgp-as-a-service/c407824d-ca33-4e95-b9ae-627048bec02c
2026-02-05 04:07:25,770 - INFO - verify_in_api_server passed for BGPaaS obj c407824d-ca33-4e95-b9ae-627048bec02c
2026-02-05 04:07:25,770 - INFO - BGPaaS(c407824d-ca33-4e95-b9ae-627048bec02c): verify_on_setup passed
2026-02-05 04:07:25,770 - INFO - We will configure BGP on the VM
2026-02-05 04:07:26,204 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2026-02-05 04:07:26,204 - INFO - Configuring BGP on ctest-TestBGPaasZone-84798581-04484377
2026-02-05 04:07:26,204 - DEBUG - Running remote_cmd, Cmd : cat > /etc/bird/bird.conf << EOS
protocol device {
scan time 10; # Scan interfaces every 10 seconds
}
protocol kernel {
persist; # Don't remove routes on bird shutdown
scan time 20; # Scan kernel routing table every 20 seconds
import all; # Default is import all
#export all; # Default is export none
}
protocol direct {
interface "eth*";
}
router id 137.119.124.132;
protocol bgp bgp_1{
local as 65000;
neighbor 137.119.124.129 as 64512;
export where source = RTS_STATIC;
export all;
multihop;
#export all;
hold time 90;
bfd on;
source address 137.119.124.132;
}
protocol bfd {
neighbor 137.119.124.129 local 137.119.124.132 multihop on;
}
EOS
, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123
2026-02-05 04:07:26,204 - DEBUG - cat > /etc/bird/bird.conf << EOS
protocol device {
scan time 10; # Scan interfaces every 10 seconds
}
protocol kernel {
persist; # Don't remove routes on bird shutdown
scan time 20; # Scan kernel routing table every 20 seconds
import all; # Default is import all
#export all; # Default is export none
}
protocol direct {
interface "eth*";
}
router id 137.119.124.132;
protocol bgp bgp_1{
local as 65000;
neighbor 137.119.124.129 as 64512;
export where source = RTS_STATIC;
export all;
multihop;
#export all;
hold time 90;
bfd on;
source address 137.119.124.132;
}
protocol bfd {
neighbor 137.119.124.129 local 137.119.124.132 multihop on;
}
EOS
2026-02-05 04:07:31,937 - DEBUG - None
2026-02-05 04:07:31,938 - DEBUG - Running remote_cmd, Cmd : cat >> /etc/bird/bird.conf << EOS
protocol bgp bgp_2{
local as 65000;
neighbor 137.119.124.130 as 64512;
export where source = RTS_STATIC;
export all;
multihop;
#export all;
hold time 90;
bfd on;
source address 137.119.124.132;
}
EOS
, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123
2026-02-05 04:07:31,938 - DEBUG - cat >> /etc/bird/bird.conf << EOS
protocol bgp bgp_2{
local as 65000;
neighbor 137.119.124.130 as 64512;
export where source = RTS_STATIC;
export all;
multihop;
#export all;
hold time 90;
bfd on;
source address 137.119.124.132;
}
EOS
2026-02-05 04:07:32,473 - DEBUG - None
2026-02-05 04:07:32,473 - DEBUG - Running remote_cmd, Cmd : service bird restart, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123
2026-02-05 04:07:32,473 - DEBUG - service bird restart
2026-02-05 04:07:34,310 - DEBUG - bird stop/waiting
bird start/running, process 1444
2026-02-05 04:07:34,310 - INFO - Attaching VMI 4c1babce-d428-444c-93d2-fb9359ecb738 to BGPaaS c407824d-ca33-4e95-b9ae-627048bec02c
2026-02-05 04:07:34,392 - INFO - Attaching the VMI 268d4d5b-00f9-49db-88b0-41d20431c5d7 to the BGPaaS c407824d-ca33-4e95-b9ae-627048bec02c object
2026-02-05 04:07:34,392 - DEBUG - Running remote_cmd, Cmd : sudo ip addr add 137.119.124.148 dev eth0, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.19, gateway password: c0ntrail123
2026-02-05 04:07:34,392 - DEBUG - sudo ip addr add 137.119.124.148 dev eth0
2026-02-05 04:07:35,119 - DEBUG - None
2026-02-05 04:07:35,119 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2026-02-05 04:07:35,119 - DEBUG - Requesting: http://10.20.0.193:8083/Snh_BgpNeighborReq?domain=&ip_address=
2026-02-05 04:07:35,132 - INFO - BGPaaS session seen in control-node 10.20.0.193
2026-02-05 04:07:35,133 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2026-02-05 04:07:35,154 - INFO - BGPaaS session seen in control-node 10.20.0.209
2026-02-05 04:07:35,164 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2026-02-05 04:07:35,164 - DEBUG - Requesting: http://10.20.0.193:8083/Snh_BgpNeighborReq?domain=&ip_address=
2026-02-05 04:07:35,181 - ERROR - Not enough BGPaaS sessions seen in control-nodes ['10.20.0.193']
2026-02-05 04:07:41,183 - DEBUG - Requesting: http://10.20.0.193:8083/Snh_BgpNeighborReq?domain=&ip_address=
2026-02-05 04:07:41,206 - ERROR - Not enough BGPaaS sessions seen in control-nodes ['10.20.0.193']
2026-02-05 04:07:47,207 - DEBUG - Requesting: http://10.20.0.193:8083/Snh_BgpNeighborReq?domain=&ip_address=
2026-02-05 04:07:47,220 - INFO - BGPaaS session seen in control-node 10.20.0.193
2026-02-05 04:07:47,221 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2026-02-05 04:07:47,237 - INFO - BGPaaS session seen in control-node 10.20.0.209
2026-02-05 04:07:47,243 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 137.119.124.148, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.65, gateway password: c0ntrail123
2026-02-05 04:07:47,243 - DEBUG - ping -s 56 -c 3 -W 1 137.119.124.148
2026-02-05 04:07:54,041 - DEBUG - PING 137.119.124.148 (137.119.124.148) 56(84) bytes of data.
64 bytes from 137.119.124.148: icmp_seq=1 ttl=64 time=6.92 ms
64 bytes from 137.119.124.148: icmp_seq=2 ttl=64 time=1.08 ms
64 bytes from 137.119.124.148: icmp_seq=3 ttl=64 time=1.22 ms
--- 137.119.124.148 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2004ms
rtt min/avg/max/mdev = 1.080/3.074/6.923/2.722 ms
2026-02-05 04:07:54,041 - INFO - Ping to IP 137.119.124.148 from VM ctest-TestBGPaasZone-84798581-37377857 passed
2026-02-05 04:07:54,041 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2026-02-05 04:07:54,042 - DEBUG - Requesting: http://10.20.0.193:8083/Snh_BgpNeighborReq?domain=&ip_address=
2026-02-05 04:07:54,056 - INFO - BGPaaS session seen in control-node 10.20.0.193
2026-02-05 04:07:54,057 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2026-02-05 04:07:54,077 - INFO - BGPaaS session seen in control-node 10.20.0.209
2026-02-05 04:07:54,083 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2026-02-05 04:07:54,083 - DEBUG - Requesting: http://10.20.0.193:8083/Snh_BgpNeighborReq?domain=&ip_address=
2026-02-05 04:07:54,096 - INFO - BGPaaS session seen in control-node 10.20.0.193
2026-02-05 04:07:54,096 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2026-02-05 04:07:54,112 - INFO - BGPaaS session seen in control-node 10.20.0.209
2026-02-05 04:07:54,118 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 137.119.124.148, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.65, gateway password: c0ntrail123
2026-02-05 04:07:54,118 - DEBUG - ping -s 56 -c 3 -W 1 137.119.124.148
2026-02-05 04:07:56,622 - DEBUG - PING 137.119.124.148 (137.119.124.148) 56(84) bytes of data.
64 bytes from 137.119.124.148: icmp_seq=1 ttl=64 time=2.60 ms
64 bytes from 137.119.124.148: icmp_seq=2 ttl=64 time=1.09 ms
64 bytes from 137.119.124.148: icmp_seq=3 ttl=64 time=1.37 ms
--- 137.119.124.148 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2004ms
rtt min/avg/max/mdev = 1.099/1.692/2.607/0.656 ms
2026-02-05 04:07:56,622 - INFO - Ping to IP 137.119.124.148 from VM ctest-TestBGPaasZone-84798581-37377857 passed
2026-02-05 04:07:56,623 - INFO - Detaching VMI 4c1babce-d428-444c-93d2-fb9359ecb738 from BGPaaS c407824d-ca33-4e95-b9ae-627048bec02c
2026-02-05 04:07:56,712 - INFO - Deleting BGPaaS ctest-TestBGPaasZone-84798581-46266610(c407824d-ca33-4e95-b9ae-627048bec02c)
2026-02-05 04:07:56,811 - DEBUG - Requesting: http://10.0.0.27:8082/bgp-as-a-service/c407824d-ca33-4e95-b9ae-627048bec02c
2026-02-05 04:07:56,823 - DEBUG - Response Code: 404
2026-02-05 04:07:56,823 - DEBUG - BGPaaS: c407824d-ca33-4e95-b9ae-627048bec02c deleted from api server
2026-02-05 04:07:56,823 - INFO - BGPaaS(c407824d-ca33-4e95-b9ae-627048bec02c): verify_on_cleanup passed
2026-02-05 04:07:56,823 - INFO - Detaching VMI 92ae5a83-a73a-4d37-a353-e60c1dc404bc from BGPaaS c2f85db0-76a2-4712-be29-0e8186bb9dcf
2026-02-05 04:07:56,928 - INFO - Deleting BGPaaS ctest-TestBGPaasZone-84798581-83276880(c2f85db0-76a2-4712-be29-0e8186bb9dcf)
2026-02-05 04:07:57,025 - DEBUG - Requesting: http://10.0.0.27:8082/bgp-as-a-service/c2f85db0-76a2-4712-be29-0e8186bb9dcf
2026-02-05 04:07:57,032 - DEBUG - Response Code: 404
2026-02-05 04:07:57,033 - DEBUG - BGPaaS: c2f85db0-76a2-4712-be29-0e8186bb9dcf deleted from api server
2026-02-05 04:07:57,033 - INFO - BGPaaS(c2f85db0-76a2-4712-be29-0e8186bb9dcf): verify_on_cleanup passed
2026-02-05 04:07:57,033 - INFO - Deleting Control node zone ctest-test-zone-48173757(0fb9aff7-d256-43f3-ac21-47a3b30b6b1d)
2026-02-05 04:07:57,216 - INFO - Deleting Control node zone ctest-test-zone-61382938(94f50c2a-82dd-4e3e-975f-0a85534d605b)
2026-02-05 04:07:57,374 - INFO - Deleting Control node zone ctest-test-zone-69707665(ad3d90e6-71ca-4002-a64a-eb91ccaab03c)
2026-02-05 04:07:57,534 - INFO - Deleting VM ctest-TestBGPaasZone-84798581-37377857
2026-02-05 04:07:57,629 - INFO - Deleting VM ctest-TestBGPaasZone-84798581-04484377
2026-02-05 04:07:57,703 - INFO - Deleting VM ctest-TestBGPaasZone-84798581-96216851
2026-02-05 04:07:57,782 - INFO - Deleting VN ctest-cnz_vn-92144349
2026-02-05 04:07:57,824 - DEBUG - VN 7a55698d-6d92-4112-8417-8ee737c1be04 still in use: Unable to complete operation on network 7a55698d-6d92-4112-8417-8ee737c1be04. There are one or more ports still in use on the network.
Neutron server returns request_ids: ['req-31edfe04-05cd-428f-b790-ebfd92c1d3f7']
2026-02-05 04:07:57,824 - WARNING - Deleting VN ctest-cnz_vn-92144349 failed..Will retry
2026-02-05 04:08:00,023 - DEBUG - Response for deleting network ()
2026-02-05 04:08:01,323 - DEBUG - Skipping xmpp flap check
2026-02-05 04:08:01,323 - INFO - END TEST : test_bgp_control_node_zones_from_single_vnf : PASSED[0:03:10]
2026-02-05 04:08:01,324 - INFO - --------------------------------------------------------------------------------
2026-02-05 04:08:02,159 - INFO - Deleted project: ctest-TestBGPaasZone-84798581, ID : dabfeb9c-fea6-4f50-9402-beec7acd9d16