is in ACTIVE state now
2026-01-08 03:23:09,519 - INFO - VM name : ctest-DisablePolicyEcmpIpv6-17946902-81944263
2026-01-08 03:23:09,602 - DEBUG - VM ctest-DisablePolicyEcmpIpv6-17946902-81944263 ID is 7c3d6341-d378-427f-b3c9-1b6aee4883e3
2026-01-08 03:23:09,602 - DEBUG - VM ctest-DisablePolicyEcmpIpv6-17946902-81944263 launched on Node an-jenkins-deploy-platform-ansible-os-4976-2
2026-01-08 03:23:09,681 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-machine/7c3d6341-d378-427f-b3c9-1b6aee4883e3
2026-01-08 03:23:09,691 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-machine-interface/fd5392a1-9622-4a74-8bbe-0fc91cb67127
2026-01-08 03:23:10,858 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n64 bytes from 169.254.0.4: icmp_seq=1 ttl=63 time=3.70 ms\r\n64 bytes from 169.254.0.4: icmp_seq=2 ttl=63 time=0.410 ms\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 0.410/2.054/3.698/1.644 ms')
2026-01-08 03:23:10,858 - INFO - Ping to Metadata IP 169.254.0.4 of VM ctest-DisablePolicyEcmpIpv6-17946902-81944263 passed
2026-01-08 03:23:10,936 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2026-01-08 03:23:10,937 - DEBUG - Waiting to SSH to VM ctest-DisablePolicyEcmpIpv6-17946902-81944263, IP 3890:ce13:e144:5f5c:7505:d734:0:6, Port 22
2026-01-08 03:23:11,109 - DEBUG - VM ctest-DisablePolicyEcmpIpv6-17946902-81944263 is ready for SSH connections
2026-01-08 03:23:11,179 - INFO - Policy DISABLED on VMI fd5392a1-9622-4a74-8bbe-0fc91cb67127
2026-01-08 03:23:11,179 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2026-01-08 03:23:11,179 - DEBUG - Running remote_cmd, Cmd : ifconfig -a| grep 3890:ce13:e144:5f5c:7505:d734:0:6 -A2 -B4| grep -i 'hwaddr\|flags' | awk '{print $1}' | cut -d ':' -f 1, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.56, gateway password: c0ntrail123
2026-01-08 03:23:11,179 - DEBUG - ifconfig -a| grep 3890:ce13:e144:5f5c:7505:d734:0:6 -A2 -B4| grep -i 'hwaddr\|flags' | awk '{print $1}' | cut -d ':' -f 1
2026-01-08 03:23:51,079 - DEBUG - eth0
2026-01-08 03:23:51,079 - DEBUG - Running remote_cmd, Cmd : sysctl net.ipv6.conf.eth0.accept_dad=0, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.56, gateway password: c0ntrail123
2026-01-08 03:23:51,080 - DEBUG - sysctl net.ipv6.conf.eth0.accept_dad=0
2026-01-08 03:23:51,716 - DEBUG - net.ipv6.conf.eth0.accept_dad = 0
2026-01-08 03:23:51,939 - INFO - Created InterfaceRouteTable ['default-domain', 'ctest-DisablePolicyEcmpIpv6-17946902', 'ctest-intf-rtb-09314938'](UUID 0f489470-cb59-4731-babf-28b491cb994c), prefixes : ['2619:8b8:8115:ff8c:6bab:58db::/96']
2026-01-08 03:23:52,011 - INFO - Added intf route table 0f489470-cb59-4731-babf-28b491cb994c to port fd5392a1-9622-4a74-8bbe-0fc91cb67127
2026-01-08 03:23:52,011 - DEBUG - Running remote_cmd, Cmd : ifconfig -a| grep -i 'hwaddr\|flags' | awk '{print $1}' | cut -d ':' -f 1, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.56, gateway password: c0ntrail123
2026-01-08 03:23:52,011 - DEBUG - ifconfig -a| grep -i 'hwaddr\|flags' | awk '{print $1}' | cut -d ':' -f 1
2026-01-08 03:23:52,456 - DEBUG - eth0
2026-01-08 03:23:52,456 - DEBUG - Running remote_cmd, Cmd : ifconfig eth0 inet6 add 2619:8b8:8115:ff8c:6bab:58db:9c51:b3e3, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.56, gateway password: c0ntrail123
2026-01-08 03:23:52,456 - DEBUG - ifconfig eth0 inet6 add 2619:8b8:8115:ff8c:6bab:58db:9c51:b3e3
2026-01-08 03:23:52,978 - DEBUG - None
2026-01-08 03:23:52,978 - DEBUG - Running remote_cmd, Cmd : ifconfig -a, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.56, gateway password: c0ntrail123
2026-01-08 03:23:52,978 - DEBUG - ifconfig -a
2026-01-08 03:23:53,561 - DEBUG - eth0 Link encap:Ethernet HWaddr 02:fd:53:92:a1:96
inet addr:25.57.28.6 Bcast:25.57.28.63 Mask:255.255.255.192
inet6 addr: 2619:8b8:8115:ff8c:6bab:58db:9c51:b3e3/128 Scope:Global
inet6 addr: 3890:ce13:e144:5f5c:7505:d734:0:6/64 Scope:Global
inet6 addr: fe80::fd:53ff:fe92:a196/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:588 errors:0 dropped:0 overruns:0 frame:0
TX packets:526 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:52622 (52.6 KB) TX bytes:44680 (44.6 KB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
2026-01-08 03:23:53,561 - INFO - Deleting VM ctest-DisablePolicyEcmpIpv6-17946902-97324464
2026-01-08 03:23:53,731 - INFO - Removed intf route table 8e44bf8d-fd58-467d-afd5-bec357d491df from port 7155c7e7-5dae-41fb-96ef-a2223350beb7
2026-01-08 03:23:53,777 - INFO - Deleted Interface route table 8e44bf8d-fd58-467d-afd5-bec357d491df
2026-01-08 03:23:53,794 - INFO - ECMP routes found in agent 10.0.0.57, for prefix 2619:8b8:8115:ff8c:6bab:58db::/96
2026-01-08 03:23:53,794 - INFO - Tap interface tap245d90d7-67 found in ecmp routes in agent 10.0.0.57
2026-01-08 03:23:53,812 - INFO - ECMP routes found in agent 10.0.0.56, for prefix 2619:8b8:8115:ff8c:6bab:58db::/96
2026-01-08 03:23:53,812 - INFO - Tap interface tapfd5392a1-96 found in ecmp routes in agent 10.0.0.56
2026-01-08 03:23:53,910 - DEBUG - Executing command: sudo tcpdump -nni tap245d90d7-67 -U '(udp and src host 3890:ce13:e144:5f5c:7505:d734:0:3 and dst host 2619:8b8:8115:ff8c:6bab:58db:9c51:b3e3 and dst port 11000)' -w /tmp/tap245d90d7-67_ctest-random-18988330.pcap
2026-01-08 03:23:54,394 - DEBUG - Executing command: sudo tcpdump -nni tapfd5392a1-96 -U '(udp and src host 3890:ce13:e144:5f5c:7505:d734:0:3 and dst host 2619:8b8:8115:ff8c:6bab:58db:9c51:b3e3 and dst port 11000)' -w /tmp/tapfd5392a1-96_ctest-random-27993428.pcap
2026-01-08 03:23:54,801 - DEBUG - Running remote_cmd, Cmd : dd bs=100 count=1 if=/dev/zero of=icmp_error;nc -6 -q 2 -u -p 15805 2619:8b8:8115:ff8c:6bab:58db:9c51:b3e3 11000 < icmp_error, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.56, gateway password: c0ntrail123
2026-01-08 03:23:54,801 - DEBUG - dd bs=100 count=1 if=/dev/zero of=icmp_error;nc -6 -q 2 -u -p 15805 2619:8b8:8115:ff8c:6bab:58db:9c51:b3e3 11000 < icmp_error
2026-01-08 03:23:57,365 - DEBUG - 1+0 records in
1+0 records out
100 bytes (100 B) copied, 0.0014976 s, 66.8 kB/s
2026-01-08 03:23:57,366 - DEBUG - Executing command: sudo tcpdump -nnr /tmp/tap245d90d7-67_ctest-random-18988330.pcap | grep -c length
2026-01-08 03:23:57,379 - DEBUG - STDOUT: 0
2026-01-08 03:23:57,379 - DEBUG - STDERR: reading from file /tmp/tap245d90d7-67_ctest-random-18988330.pcap, link-type EN10MB (Ethernet), snapshot length 262144
2026-01-08 03:23:57,380 - WARNING - No packets are found in tcpdump output file /tmp/tap245d90d7-67_ctest-random-18988330.pcap but expected some packets
2026-01-08 03:23:59,381 - DEBUG - Executing command: sudo tcpdump -nnr /tmp/tap245d90d7-67_ctest-random-18988330.pcap | grep -c length
2026-01-08 03:23:59,397 - DEBUG - STDOUT: 0
2026-01-08 03:23:59,397 - DEBUG - STDERR: reading from file /tmp/tap245d90d7-67_ctest-random-18988330.pcap, link-type EN10MB (Ethernet), snapshot length 262144
2026-01-08 03:23:59,397 - WARNING - No packets are found in tcpdump output file /tmp/tap245d90d7-67_ctest-random-18988330.pcap but expected some packets
2026-01-08 03:24:01,397 - DEBUG - Executing command: sudo tcpdump -nnr /tmp/tap245d90d7-67_ctest-random-18988330.pcap | grep -c length
2026-01-08 03:24:01,413 - DEBUG - STDOUT: 0
2026-01-08 03:24:01,413 - DEBUG - STDERR: reading from file /tmp/tap245d90d7-67_ctest-random-18988330.pcap, link-type EN10MB (Ethernet), snapshot length 262144
2026-01-08 03:24:01,413 - WARNING - No packets are found in tcpdump output file /tmp/tap245d90d7-67_ctest-random-18988330.pcap but expected some packets
2026-01-08 03:24:03,414 - DEBUG - Executing command: sudo tcpdump -nnr /tmp/tap245d90d7-67_ctest-random-18988330.pcap | grep -c length
2026-01-08 03:24:03,429 - DEBUG - STDOUT: 0
2026-01-08 03:24:03,429 - DEBUG - STDERR: reading from file /tmp/tap245d90d7-67_ctest-random-18988330.pcap, link-type EN10MB (Ethernet), snapshot length 262144
2026-01-08 03:24:03,429 - WARNING - No packets are found in tcpdump output file /tmp/tap245d90d7-67_ctest-random-18988330.pcap but expected some packets
2026-01-08 03:24:05,430 - DEBUG - Executing command: sudo tcpdump -nnr /tmp/tap245d90d7-67_ctest-random-18988330.pcap | grep -c length
2026-01-08 03:24:05,445 - DEBUG - STDOUT: 0
2026-01-08 03:24:05,445 - DEBUG - STDERR: reading from file /tmp/tap245d90d7-67_ctest-random-18988330.pcap, link-type EN10MB (Ethernet), snapshot length 262144
2026-01-08 03:24:05,445 - WARNING - No packets are found in tcpdump output file /tmp/tap245d90d7-67_ctest-random-18988330.pcap but expected some packets
2026-01-08 03:24:07,446 - DEBUG - Executing command: sudo tcpdump -nnr /tmp/tap245d90d7-67_ctest-random-18988330.pcap | grep -c length
2026-01-08 03:24:07,461 - DEBUG - STDOUT: 0
2026-01-08 03:24:07,461 - DEBUG - STDERR: reading from file /tmp/tap245d90d7-67_ctest-random-18988330.pcap, link-type EN10MB (Ethernet), snapshot length 262144
2026-01-08 03:24:07,461 - WARNING - No packets are found in tcpdump output file /tmp/tap245d90d7-67_ctest-random-18988330.pcap but expected some packets
2026-01-08 03:24:09,462 - DEBUG - Executing command: sudo tcpdump -nnr /tmp/tap245d90d7-67_ctest-random-18988330.pcap | grep -c length
2026-01-08 03:24:09,479 - DEBUG - STDOUT: 0
2026-01-08 03:24:09,479 - DEBUG - STDERR: reading from file /tmp/tap245d90d7-67_ctest-random-18988330.pcap, link-type EN10MB (Ethernet), snapshot length 262144
2026-01-08 03:24:09,479 - WARNING - No packets are found in tcpdump output file /tmp/tap245d90d7-67_ctest-random-18988330.pcap but expected some packets
2026-01-08 03:24:09,480 - DEBUG - Executing command: sudo tcpdump -nnr /tmp/tapfd5392a1-96_ctest-random-27993428.pcap | grep -c length
2026-01-08 03:24:09,495 - DEBUG - STDOUT: 1
2026-01-08 03:24:09,496 - DEBUG - STDERR: reading from file /tmp/tapfd5392a1-96_ctest-random-27993428.pcap, link-type EN10MB (Ethernet), snapshot length 262144
2026-01-08 03:24:09,496 - INFO - 1 packets are found in tcpdump output as expected
2026-01-08 03:24:09,496 - DEBUG - Executing command: sudo kill $(ps -ef|grep tcpdump | grep pcap| awk '{print $2}')
2026-01-08 03:24:11,540 - INFO - Tcpdump verification on VM 3890:ce13:e144:5f5c:7505:d734:0:6 passed
2026-01-08 03:24:11,834 - DEBUG - Forward and reverse flows for match {'sip': '3890:ce13:e144:5f5c:7505:d734:0:3', 'dip': '2619:8b8:8115:ff8c:6bab:58db:9c51:b3e3', 'sport': '15805', 'dport': '11000', 'proto': '17', 'vrf_id': '2'}: 0, 0
2026-01-08 03:24:11,835 - DEBUG - Flow verification PASSED on node: 10.0.0.56 for VMs - Sender: 3890:ce13:e144:5f5c:7505:d734:0:3, Receiver: 2619:8b8:8115:ff8c:6bab:58db:9c51:b3e3, Flow count expected: 0 0, got: 0 0
2026-01-08 03:24:11,835 - DEBUG - Executing command: sudo kill $(ps -ef|grep tcpdump | grep pcap| awk '{print $2}')
2026-01-08 03:24:13,843 - DEBUG - Executing command: sudo kill $(ps -ef|grep tcpdump | grep pcap| awk '{print $2}')
2026-01-08 03:24:16,152 - DEBUG - Forward and reverse flows for match {'sip': '3890:ce13:e144:5f5c:7505:d734:0:3', 'dip': '2619:8b8:8115:ff8c:6bab:58db:9c51:b3e3', 'sport': '15805', 'dport': '11000', 'proto': '17', 'vrf_id': '2'}: 0, 0
2026-01-08 03:24:16,152 - DEBUG - Flow verification PASSED on node: 10.0.0.56 for VMs - Sender: 3890:ce13:e144:5f5c:7505:d734:0:3, Receiver: 2619:8b8:8115:ff8c:6bab:58db:9c51:b3e3, Flow count expected: 0 0, got: 0 0
2026-01-08 03:24:16,152 - INFO - Traffic verification for ECMP passed
2026-01-08 03:24:16,152 - INFO - Deleting VM ctest-DisablePolicyEcmpIpv6-17946902-81944263
2026-01-08 03:24:16,318 - INFO - Removed intf route table 0f489470-cb59-4731-babf-28b491cb994c from port fd5392a1-9622-4a74-8bbe-0fc91cb67127
2026-01-08 03:24:16,353 - INFO - Deleted Interface route table 0f489470-cb59-4731-babf-28b491cb994c
2026-01-08 03:24:16,354 - INFO - Deleting VM ctest-DisablePolicyEcmpIpv6-17946902-31958061
2026-01-08 03:24:16,507 - INFO - Removed intf route table 594ff20b-7db7-4dd1-aa5a-d30bada36d66 from port 245d90d7-67e8-421d-b0a6-4919ebc6aee8
2026-01-08 03:24:16,546 - INFO - Deleted Interface route table 594ff20b-7db7-4dd1-aa5a-d30bada36d66
2026-01-08 03:24:16,546 - INFO - Deleting VM ctest-DisablePolicyEcmpIpv6-17946902-77623439
2026-01-08 03:24:16,645 - INFO - Deleting VN ctest-vn-61398344
2026-01-08 03:24:16,703 - DEBUG - VN 57d53fcd-610b-4159-b9c0-0bc14ad63fa9 still in use: Unable to complete operation on network 57d53fcd-610b-4159-b9c0-0bc14ad63fa9. There are one or more ports still in use on the network.
Neutron server returns request_ids: ['req-6783ec4e-91d3-4433-b9c0-02daed5aa730']
2026-01-08 03:24:16,703 - WARNING - Deleting VN ctest-vn-61398344 failed..Will retry
2026-01-08 03:24:18,970 - DEBUG - Response for deleting network ()
2026-01-08 03:24:18,970 - DEBUG - Requesting: http://10.0.0.242:8082/routing-instance/7829117f-47df-49a6-862b-801d81ac2d0a
2026-01-08 03:24:18,975 - DEBUG - Response Code: 404
2026-01-08 03:24:18,975 - DEBUG - Requesting: http://10.0.0.242:8082/domains
2026-01-08 03:24:18,983 - DEBUG - Requesting: http://10.0.0.242:8082/domain/01640a83-288f-426f-a2f2-0c0b4b93ac50
2026-01-08 03:24:19,041 - DEBUG - Requesting: http://10.0.0.242:8082/project/d622a118-3fc3-44e3-a045-f6b983e144d2
2026-01-08 03:24:19,152 - INFO - Validated that VN ctest-vn-61398344 is not found in API Server
2026-01-08 03:24:19,162 - DEBUG - VN ctest-vn-61398344 is not present in Agent 10.0.0.57
2026-01-08 03:24:19,176 - DEBUG - VN ctest-vn-61398344 is not present in Agent 10.0.0.56
2026-01-08 03:24:19,176 - INFO - Validated that VN ctest-vn-61398344 is not in any agent
2026-01-08 03:24:19,208 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-vn-61398344 info
2026-01-08 03:24:20,491 - DEBUG - No XMPP flaps were noticed during the test
2026-01-08 03:24:20,491 - INFO - END TEST : test_ecmp_with_static_routes : PASSED[0:04:56]
2026-01-08 03:24:20,491 - INFO - --------------------------------------------------------------------------------
2026-01-08 03:24:21,279 - INFO - Deleted project: ctest-DisablePolicyEcmpIpv6-17946902, ID : d622a118-3fc3-44e3-a045-f6b983e144d2