is in ACTIVE state now
2026-04-27 05:13:00,270 - INFO - VM name : ctest-FatFlowSerialIpv6-45601470-02848325
2026-04-27 05:13:00,409 - DEBUG - VM ctest-FatFlowSerialIpv6-45601470-02848325 ID is 02f968e6-d045-482c-8457-9735eadf76f5
2026-04-27 05:13:00,409 - DEBUG - VM ctest-FatFlowSerialIpv6-45601470-02848325 launched on Node an-jenkins-deploy-platform-ansible-os-5690-2
2026-04-27 05:13:00,544 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/02f968e6-d045-482c-8457-9735eadf76f5
2026-04-27 05:13:00,559 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/95b0afd5-8736-4c7f-91f1-6a0c45359c14
2026-04-27 05:13:01,759 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n64 bytes from 169.254.0.4: icmp_seq=1 ttl=63 time=3.58 ms\r\n64 bytes from 169.254.0.4: icmp_seq=2 ttl=63 time=2.19 ms\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 2.185/2.881/3.578/0.696 ms')
2026-04-27 05:13:01,760 - INFO - Ping to Metadata IP 169.254.0.4 of VM ctest-FatFlowSerialIpv6-45601470-02848325 passed
2026-04-27 05:13:01,843 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2026-04-27 05:13:01,843 - DEBUG - Waiting to SSH to VM ctest-FatFlowSerialIpv6-45601470-02848325, IP 158.142.41.69, Port 22
2026-04-27 05:13:02,025 - DEBUG - VM ctest-FatFlowSerialIpv6-45601470-02848325 is ready for SSH connections
2026-04-27 05:13:02,115 - INFO - Fat flow added on VMI 95b0afd5-8736-4c7f-91f1-6a0c45359c14: {'proto': 'udp', 'port': 53}
2026-04-27 05:13:02,120 - DEBUG - Running remote_cmd, Cmd : rm -f testfile;ls -la;nc -4 -q 2 -w 5 -u -ll -p 53 > testfile, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.65, gateway password: c0ntrail123
2026-04-27 05:13:02,120 - DEBUG - nohup rm -f testfile;ls -la;nc -4 -q 2 -w 5 -u -ll -p 53 > testfile &
2026-04-27 05:13:28,756 - DEBUG - total 132
drwxr-xr-x 11 ubuntu ubuntu 4096 Dec 29 2017 .
drwxr-xr-x 3 root root 4096 Jan 4 2013 ..
-rw------- 1 ubuntu ubuntu 2689 Dec 29 2017 .bash_history
-rw-r--r-- 1 ubuntu ubuntu 220 Apr 3 2012 .bash_logout
-rw-r--r-- 1 ubuntu ubuntu 3486 Apr 3 2012 .bashrc
drwx------ 2 ubuntu ubuntu 4096 Jan 9 2013 .cache
-rwxrwxr-x 1 ubuntu ubuntu 1431 Feb 5 2015 flow_test_pktgen.sh
drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 9 2013 .fontconfig
drwxrwxr-x 2 ubuntu ubuntu 4096 Jan 9 2013 .fvwm
drwxrwxr-x 3 ubuntu ubuntu 4096 Nov 29 2016 .jenkins
-rw------- 1 ubuntu ubuntu 52 Jul 2 2014 .lesshst
drwx------ 4 ubuntu ubuntu 4096 Jan 9 2013 .mozilla
-rwxrwxr-x 1 ubuntu ubuntu 1378 Jun 27 2014 pktgen_new.sh
-rw-r--r-- 1 ubuntu ubuntu 675 Apr 3 2012 .profile
drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 27 2014 scr
drwx------ 2 ubuntu ubuntu 4096 Feb 3 2015 .ssh
-rw-r--r-- 1 ubuntu ubuntu 0 Jan 9 2013 .sudo_as_admin_successful
drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 27 2014 traffic
drwxr-xr-x 4 root root 4096 Jun 27 2014 traffic-1.0
-rw-rw-r-- 1 ubuntu ubuntu 16361 Jan 9 2015 traffic-1.0.tar.gz
-rw------- 1 root root 4695 Dec 17 2017 .viminfo
-rw------- 1 ubuntu ubuntu 152 Jul 2 2014 .Xauthority
-rw------- 1 ubuntu ubuntu 29828 Jan 9 2013 .xsession-errors
2026-04-27 05:13:28,756 - DEBUG - Running remote_cmd, Cmd : nc -4 -q 2 -w 5 -u -ll 53 > testfile, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.65, gateway password: c0ntrail123
2026-04-27 05:13:28,756 - DEBUG - nohup nc -4 -q 2 -w 5 -u -ll 53 > testfile &
2026-04-27 05:13:29,842 - DEBUG - None
2026-04-27 05:13:29,842 - DEBUG - Running remote_cmd, Cmd : dd bs=100 count=1 if=/dev/zero of=testfile;nc -4 -q 2 -w 5 -u -p 10000 158.142.41.69 53 < testfile, host_string: ubuntu@169.254.0.5, password: ubuntugateway: ubuntu@10.0.0.33, gateway password: c0ntrail123
2026-04-27 05:13:29,842 - DEBUG - dd bs=100 count=1 if=/dev/zero of=testfile;nc -4 -q 2 -w 5 -u -p 10000 158.142.41.69 53 < testfile
2026-04-27 05:14:15,078 - DEBUG - 1+0 records in
1+0 records out
100 bytes (100 B) copied, 0.000879107 s, 114 kB/s
2026-04-27 05:14:15,078 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.65, gateway password: c0ntrail123
2026-04-27 05:14:15,078 - DEBUG - ls -l testfile
2026-04-27 05:14:15,527 - DEBUG - -rw-r--r-- 1 root root 100 Apr 27 05:14 testfile
2026-04-27 05:14:15,527 - INFO - File transfer verification for file size 100 passed on the VM ctest-FatFlowSerialIpv6-45601470-02848325
2026-04-27 05:14:15,527 - DEBUG - Running remote_cmd, Cmd : rm -f testfile;ls -la;nc -4 -q 2 -w 5 -u -ll -p 53 > testfile, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.65, gateway password: c0ntrail123
2026-04-27 05:14:15,527 - DEBUG - nohup rm -f testfile;ls -la;nc -4 -q 2 -w 5 -u -ll -p 53 > testfile &
2026-04-27 05:14:16,272 - DEBUG - total 132
drwxr-xr-x 11 ubuntu ubuntu 4096 Apr 27 05:14 .
drwxr-xr-x 3 root root 4096 Jan 4 2013 ..
-rw------- 1 ubuntu ubuntu 2689 Dec 29 2017 .bash_history
-rw-r--r-- 1 ubuntu ubuntu 220 Apr 3 2012 .bash_logout
-rw-r--r-- 1 ubuntu ubuntu 3486 Apr 3 2012 .bashrc
drwx------ 2 ubuntu ubuntu 4096 Jan 9 2013 .cache
-rwxrwxr-x 1 ubuntu ubuntu 1431 Feb 5 2015 flow_test_pktgen.sh
drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 9 2013 .fontconfig
drwxrwxr-x 2 ubuntu ubuntu 4096 Jan 9 2013 .fvwm
drwxrwxr-x 3 ubuntu ubuntu 4096 Nov 29 2016 .jenkins
-rw------- 1 ubuntu ubuntu 52 Jul 2 2014 .lesshst
drwx------ 4 ubuntu ubuntu 4096 Jan 9 2013 .mozilla
-rwxrwxr-x 1 ubuntu ubuntu 1378 Jun 27 2014 pktgen_new.sh
-rw-r--r-- 1 ubuntu ubuntu 675 Apr 3 2012 .profile
drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 27 2014 scr
drwx------ 2 ubuntu ubuntu 4096 Feb 3 2015 .ssh
-rw-r--r-- 1 ubuntu ubuntu 0 Jan 9 2013 .sudo_as_admin_successful
drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 27 2014 traffic
drwxr-xr-x 4 root root 4096 Jun 27 2014 traffic-1.0
-rw-rw-r-- 1 ubuntu ubuntu 16361 Jan 9 2015 traffic-1.0.tar.gz
-rw------- 1 root root 4695 Dec 17 2017 .viminfo
-rw------- 1 ubuntu ubuntu 152 Jul 2 2014 .Xauthority
-rw------- 1 ubuntu ubuntu 29828 Jan 9 2013 .xsession-errors
2026-04-27 05:14:16,273 - DEBUG - Running remote_cmd, Cmd : nc -4 -q 2 -w 5 -u -ll 53 > testfile, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.65, gateway password: c0ntrail123
2026-04-27 05:14:16,273 - DEBUG - nohup nc -4 -q 2 -w 5 -u -ll 53 > testfile &
2026-04-27 05:14:16,837 - DEBUG - None
2026-04-27 05:14:16,837 - DEBUG - Running remote_cmd, Cmd : dd bs=100 count=1 if=/dev/zero of=testfile;nc -4 -q 2 -w 5 -u -p 10000 158.142.41.69 53 < testfile, host_string: ubuntu@169.254.0.6, password: ubuntugateway: ubuntu@10.0.0.33, gateway password: c0ntrail123
2026-04-27 05:14:16,837 - DEBUG - dd bs=100 count=1 if=/dev/zero of=testfile;nc -4 -q 2 -w 5 -u -p 10000 158.142.41.69 53 < testfile
2026-04-27 05:15:00,142 - DEBUG - 1+0 records in
1+0 records out
100 bytes (100 B) copied, 0.000809147 s, 124 kB/s
2026-04-27 05:15:00,142 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.65, gateway password: c0ntrail123
2026-04-27 05:15:00,142 - DEBUG - ls -l testfile
2026-04-27 05:15:00,576 - DEBUG - -rw-r--r-- 1 root root 100 Apr 27 05:14 testfile
2026-04-27 05:15:00,576 - INFO - File transfer verification for file size 100 passed on the VM ctest-FatFlowSerialIpv6-45601470-02848325
2026-04-27 05:15:00,700 - DEBUG - Forward and reverse flows for match {'sip': '158.142.41.67', 'dip': '158.142.41.69', 'sport': '10000', 'dport': '53', 'proto': '17', 'vrf_id': '4'}: 1, 1
2026-04-27 05:15:00,794 - DEBUG - Forward and reverse flows for match {'sip': '158.142.41.67', 'dip': '158.142.41.69', 'sport': '10000', 'dport': '53', 'proto': '17', 'vrf_id': '3'}: 0, 0
2026-04-27 05:15:00,949 - DEBUG - Forward and reverse flows for match {'sip': '158.142.41.68', 'dip': '158.142.41.69', 'sport': '10000', 'dport': '53', 'proto': '17', 'vrf_id': '4'}: 1, 1
2026-04-27 05:15:01,035 - DEBUG - Forward and reverse flows for match {'sip': '158.142.41.68', 'dip': '158.142.41.69', 'sport': '10000', 'dport': '53', 'proto': '17', 'vrf_id': '3'}: 0, 0
2026-04-27 05:15:01,128 - DEBUG - Forward and reverse flows for match {'sip': '158.142.41.67', 'dip': '158.142.41.69', 'sport': '0', 'dport': '53', 'proto': '17', 'vrf_id': '3'}: 1, 1
2026-04-27 05:15:01,128 - INFO - Fat flow verification PASSED on node: 10.0.0.65 for - SIP: 158.142.41.67, DIP: 158.142.41.69, sport: 0, dport: 53, vrf-id: 3 Fat flow expected: 1, got: 1
2026-04-27 05:15:01,257 - DEBUG - Forward and reverse flows for match {'sip': '158.142.41.67', 'dip': '158.142.41.69', 'sport': '0', 'dport': '53', 'proto': '17', 'vrf_id': '4'}: 0, 0
2026-04-27 05:15:01,257 - INFO - Fat flow verification PASSED on node: 10.0.0.33 for - SIP: 158.142.41.67, DIP: 158.142.41.69, sport: 0, dport: 53, vrf-id: 4 Fat flow expected: 0, got: 0
2026-04-27 05:15:01,339 - DEBUG - Forward and reverse flows for match {'sip': '158.142.41.68', 'dip': '158.142.41.69', 'sport': '0', 'dport': '53', 'proto': '17', 'vrf_id': '3'}: 1, 1
2026-04-27 05:15:01,340 - INFO - Fat flow verification PASSED on node: 10.0.0.65 for - SIP: 158.142.41.68, DIP: 158.142.41.69, sport: 0, dport: 53, vrf-id: 3 Fat flow expected: 1, got: 1
2026-04-27 05:15:01,465 - DEBUG - Forward and reverse flows for match {'sip': '158.142.41.68', 'dip': '158.142.41.69', 'sport': '0', 'dport': '53', 'proto': '17', 'vrf_id': '4'}: 0, 0
2026-04-27 05:15:01,465 - INFO - Fat flow verification PASSED on node: 10.0.0.33 for - SIP: 158.142.41.68, DIP: 158.142.41.69, sport: 0, dport: 53, vrf-id: 4 Fat flow expected: 0, got: 0
2026-04-27 05:15:01,465 - INFO - Fat flow verification passed for protocol udp and port 53
2026-04-27 05:15:01,466 - DEBUG - Running remote_cmd, Cmd : rm -f testfile;ls -la;nc -6 -q 2 -w 5 -u -ll -p 53 > testfile, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.65, gateway password: c0ntrail123
2026-04-27 05:15:01,466 - DEBUG - nohup rm -f testfile;ls -la;nc -6 -q 2 -w 5 -u -ll -p 53 > testfile &
2026-04-27 05:15:02,217 - DEBUG - total 132
drwxr-xr-x 11 ubuntu ubuntu 4096 Apr 27 05:15 .
drwxr-xr-x 3 root root 4096 Jan 4 2013 ..
-rw------- 1 ubuntu ubuntu 2689 Dec 29 2017 .bash_history
-rw-r--r-- 1 ubuntu ubuntu 220 Apr 3 2012 .bash_logout
-rw-r--r-- 1 ubuntu ubuntu 3486 Apr 3 2012 .bashrc
drwx------ 2 ubuntu ubuntu 4096 Jan 9 2013 .cache
-rwxrwxr-x 1 ubuntu ubuntu 1431 Feb 5 2015 flow_test_pktgen.sh
drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 9 2013 .fontconfig
drwxrwxr-x 2 ubuntu ubuntu 4096 Jan 9 2013 .fvwm
drwxrwxr-x 3 ubuntu ubuntu 4096 Nov 29 2016 .jenkins
-rw------- 1 ubuntu ubuntu 52 Jul 2 2014 .lesshst
drwx------ 4 ubuntu ubuntu 4096 Jan 9 2013 .mozilla
-rwxrwxr-x 1 ubuntu ubuntu 1378 Jun 27 2014 pktgen_new.sh
-rw-r--r-- 1 ubuntu ubuntu 675 Apr 3 2012 .profile
drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 27 2014 scr
drwx------ 2 ubuntu ubuntu 4096 Feb 3 2015 .ssh
-rw-r--r-- 1 ubuntu ubuntu 0 Jan 9 2013 .sudo_as_admin_successful
drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 27 2014 traffic
drwxr-xr-x 4 root root 4096 Jun 27 2014 traffic-1.0
-rw-rw-r-- 1 ubuntu ubuntu 16361 Jan 9 2015 traffic-1.0.tar.gz
-rw------- 1 root root 4695 Dec 17 2017 .viminfo
-rw------- 1 ubuntu ubuntu 152 Jul 2 2014 .Xauthority
-rw------- 1 ubuntu ubuntu 29828 Jan 9 2013 .xsession-errors
2026-04-27 05:15:02,217 - DEBUG - Running remote_cmd, Cmd : nc -6 -q 2 -w 5 -u -ll 53 > testfile, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.65, gateway password: c0ntrail123
2026-04-27 05:15:02,217 - DEBUG - nohup nc -6 -q 2 -w 5 -u -ll 53 > testfile &
2026-04-27 05:15:02,799 - DEBUG - None
2026-04-27 05:15:02,799 - DEBUG - Running remote_cmd, Cmd : dd bs=100 count=1 if=/dev/zero of=testfile;nc -6 -q 2 -w 5 -u -p 20000 3c33:f235:115d:a000:aac:a088:0:5 53 < testfile, host_string: ubuntu@169.254.0.5, password: ubuntugateway: ubuntu@10.0.0.33, gateway password: c0ntrail123
2026-04-27 05:15:02,799 - DEBUG - dd bs=100 count=1 if=/dev/zero of=testfile;nc -6 -q 2 -w 5 -u -p 20000 3c33:f235:115d:a000:aac:a088:0:5 53 < testfile
2026-04-27 05:15:05,397 - DEBUG - 1+0 records in
1+0 records out
100 bytes (100 B) copied, 0.0010889 s, 91.8 kB/s
2026-04-27 05:15:05,397 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.65, gateway password: c0ntrail123
2026-04-27 05:15:05,397 - DEBUG - ls -l testfile
2026-04-27 05:15:06,140 - DEBUG - -rw-r--r-- 1 root root 100 Apr 27 05:15 testfile
2026-04-27 05:15:06,140 - INFO - File transfer verification for file size 100 passed on the VM ctest-FatFlowSerialIpv6-45601470-02848325
2026-04-27 05:15:06,141 - DEBUG - Running remote_cmd, Cmd : rm -f testfile;ls -la;nc -6 -q 2 -w 5 -u -ll -p 53 > testfile, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.65, gateway password: c0ntrail123
2026-04-27 05:15:06,141 - DEBUG - nohup rm -f testfile;ls -la;nc -6 -q 2 -w 5 -u -ll -p 53 > testfile &
2026-04-27 05:15:06,963 - DEBUG - total 132
drwxr-xr-x 11 ubuntu ubuntu 4096 Apr 27 05:15 .
drwxr-xr-x 3 root root 4096 Jan 4 2013 ..
-rw------- 1 ubuntu ubuntu 2689 Dec 29 2017 .bash_history
-rw-r--r-- 1 ubuntu ubuntu 220 Apr 3 2012 .bash_logout
-rw-r--r-- 1 ubuntu ubuntu 3486 Apr 3 2012 .bashrc
drwx------ 2 ubuntu ubuntu 4096 Jan 9 2013 .cache
-rwxrwxr-x 1 ubuntu ubuntu 1431 Feb 5 2015 flow_test_pktgen.sh
drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 9 2013 .fontconfig
drwxrwxr-x 2 ubuntu ubuntu 4096 Jan 9 2013 .fvwm
drwxrwxr-x 3 ubuntu ubuntu 4096 Nov 29 2016 .jenkins
-rw------- 1 ubuntu ubuntu 52 Jul 2 2014 .lesshst
drwx------ 4 ubuntu ubuntu 4096 Jan 9 2013 .mozilla
-rwxrwxr-x 1 ubuntu ubuntu 1378 Jun 27 2014 pktgen_new.sh
-rw-r--r-- 1 ubuntu ubuntu 675 Apr 3 2012 .profile
drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 27 2014 scr
drwx------ 2 ubuntu ubuntu 4096 Feb 3 2015 .ssh
-rw-r--r-- 1 ubuntu ubuntu 0 Jan 9 2013 .sudo_as_admin_successful
drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 27 2014 traffic
drwxr-xr-x 4 root root 4096 Jun 27 2014 traffic-1.0
-rw-rw-r-- 1 ubuntu ubuntu 16361 Jan 9 2015 traffic-1.0.tar.gz
-rw------- 1 root root 4695 Dec 17 2017 .viminfo
-rw------- 1 ubuntu ubuntu 152 Jul 2 2014 .Xauthority
-rw------- 1 ubuntu ubuntu 29828 Jan 9 2013 .xsession-errors
2026-04-27 05:15:06,963 - DEBUG - Running remote_cmd, Cmd : nc -6 -q 2 -w 5 -u -ll 53 > testfile, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.65, gateway password: c0ntrail123
2026-04-27 05:15:06,963 - DEBUG - nohup nc -6 -q 2 -w 5 -u -ll 53 > testfile &
2026-04-27 05:15:07,521 - DEBUG - None
2026-04-27 05:15:07,521 - DEBUG - Running remote_cmd, Cmd : dd bs=100 count=1 if=/dev/zero of=testfile;nc -6 -q 2 -w 5 -u -p 20001 3c33:f235:115d:a000:aac:a088:0:5 53 < testfile, host_string: ubuntu@169.254.0.5, password: ubuntugateway: ubuntu@10.0.0.33, gateway password: c0ntrail123
2026-04-27 05:15:07,521 - DEBUG - dd bs=100 count=1 if=/dev/zero of=testfile;nc -6 -q 2 -w 5 -u -p 20001 3c33:f235:115d:a000:aac:a088:0:5 53 < testfile
2026-04-27 05:15:10,272 - DEBUG - 1+0 records in
1+0 records out
100 bytes (100 B) copied, 0.00179891 s, 55.6 kB/s
2026-04-27 05:15:10,272 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.65, gateway password: c0ntrail123
2026-04-27 05:15:10,272 - DEBUG - ls -l testfile
2026-04-27 05:15:10,697 - DEBUG - -rw-r--r-- 1 root root 100 Apr 27 05:15 testfile
2026-04-27 05:15:10,697 - INFO - File transfer verification for file size 100 passed on the VM ctest-FatFlowSerialIpv6-45601470-02848325
2026-04-27 05:15:10,698 - DEBUG - Running remote_cmd, Cmd : rm -f testfile;ls -la;nc -6 -q 2 -w 5 -u -ll -p 53 > testfile, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.65, gateway password: c0ntrail123
2026-04-27 05:15:10,698 - DEBUG - nohup rm -f testfile;ls -la;nc -6 -q 2 -w 5 -u -ll -p 53 > testfile &
2026-04-27 05:15:11,421 - DEBUG - total 132
drwxr-xr-x 11 ubuntu ubuntu 4096 Apr 27 05:15 .
drwxr-xr-x 3 root root 4096 Jan 4 2013 ..
-rw------- 1 ubuntu ubuntu 2689 Dec 29 2017 .bash_history
-rw-r--r-- 1 ubuntu ubuntu 220 Apr 3 2012 .bash_logout
-rw-r--r-- 1 ubuntu ubuntu 3486 Apr 3 2012 .bashrc
drwx------ 2 ubuntu ubuntu 4096 Jan 9 2013 .cache
-rwxrwxr-x 1 ubuntu ubuntu 1431 Feb 5 2015 flow_test_pktgen.sh
drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 9 2013 .fontconfig
drwxrwxr-x 2 ubuntu ubuntu 4096 Jan 9 2013 .fvwm
drwxrwxr-x 3 ubuntu ubuntu 4096 Nov 29 2016 .jenkins
-rw------- 1 ubuntu ubuntu 52 Jul 2 2014 .lesshst
drwx------ 4 ubuntu ubuntu 4096 Jan 9 2013 .mozilla
-rwxrwxr-x 1 ubuntu ubuntu 1378 Jun 27 2014 pktgen_new.sh
-rw-r--r-- 1 ubuntu ubuntu 675 Apr 3 2012 .profile
drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 27 2014 scr
drwx------ 2 ubuntu ubuntu 4096 Feb 3 2015 .ssh
-rw-r--r-- 1 ubuntu ubuntu 0 Jan 9 2013 .sudo_as_admin_successful
drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 27 2014 traffic
drwxr-xr-x 4 root root 4096 Jun 27 2014 traffic-1.0
-rw-rw-r-- 1 ubuntu ubuntu 16361 Jan 9 2015 traffic-1.0.tar.gz
-rw------- 1 root root 4695 Dec 17 2017 .viminfo
-rw------- 1 ubuntu ubuntu 152 Jul 2 2014 .Xauthority
-rw------- 1 ubuntu ubuntu 29828 Jan 9 2013 .xsession-errors
2026-04-27 05:15:11,422 - DEBUG - Running remote_cmd, Cmd : nc -6 -q 2 -w 5 -u -ll 53 > testfile, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.65, gateway password: c0ntrail123
2026-04-27 05:15:11,422 - DEBUG - nohup nc -6 -q 2 -w 5 -u -ll 53 > testfile &
2026-04-27 05:15:11,981 - DEBUG - None
2026-04-27 05:15:11,981 - DEBUG - Running remote_cmd, Cmd : dd bs=100 count=1 if=/dev/zero of=testfile;nc -6 -q 2 -w 5 -u -p 20000 3c33:f235:115d:a000:aac:a088:0:5 53 < testfile, host_string: ubuntu@169.254.0.6, password: ubuntugateway: ubuntu@10.0.0.33, gateway password: c0ntrail123
2026-04-27 05:15:11,981 - DEBUG - dd bs=100 count=1 if=/dev/zero of=testfile;nc -6 -q 2 -w 5 -u -p 20000 3c33:f235:115d:a000:aac:a088:0:5 53 < testfile
2026-04-27 05:15:14,586 - DEBUG - 1+0 records in
1+0 records out
100 bytes (100 B) copied, 0.00116825 s, 85.6 kB/s
2026-04-27 05:15:14,586 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.65, gateway password: c0ntrail123
2026-04-27 05:15:14,586 - DEBUG - ls -l testfile
2026-04-27 05:15:15,000 - DEBUG - -rw-r--r-- 1 root root 100 Apr 27 05:15 testfile
2026-04-27 05:15:15,000 - INFO - File transfer verification for file size 100 passed on the VM ctest-FatFlowSerialIpv6-45601470-02848325
2026-04-27 05:15:15,000 - DEBUG - Running remote_cmd, Cmd : rm -f testfile;ls -la;nc -6 -q 2 -w 5 -u -ll -p 53 > testfile, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.65, gateway password: c0ntrail123
2026-04-27 05:15:15,000 - DEBUG - nohup rm -f testfile;ls -la;nc -6 -q 2 -w 5 -u -ll -p 53 > testfile &
2026-04-27 05:15:15,761 - DEBUG - total 132
drwxr-xr-x 11 ubuntu ubuntu 4096 Apr 27 05:15 .
drwxr-xr-x 3 root root 4096 Jan 4 2013 ..
-rw------- 1 ubuntu ubuntu 2689 Dec 29 2017 .bash_history
-rw-r--r-- 1 ubuntu ubuntu 220 Apr 3 2012 .bash_logout
-rw-r--r-- 1 ubuntu ubuntu 3486 Apr 3 2012 .bashrc
drwx------ 2 ubuntu ubuntu 4096 Jan 9 2013 .cache
-rwxrwxr-x 1 ubuntu ubuntu 1431 Feb 5 2015 flow_test_pktgen.sh
drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 9 2013 .fontconfig
drwxrwxr-x 2 ubuntu ubuntu 4096 Jan 9 2013 .fvwm
drwxrwxr-x 3 ubuntu ubuntu 4096 Nov 29 2016 .jenkins
-rw------- 1 ubuntu ubuntu 52 Jul 2 2014 .lesshst
drwx------ 4 ubuntu ubuntu 4096 Jan 9 2013 .mozilla
-rwxrwxr-x 1 ubuntu ubuntu 1378 Jun 27 2014 pktgen_new.sh
-rw-r--r-- 1 ubuntu ubuntu 675 Apr 3 2012 .profile
drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 27 2014 scr
drwx------ 2 ubuntu ubuntu 4096 Feb 3 2015 .ssh
-rw-r--r-- 1 ubuntu ubuntu 0 Jan 9 2013 .sudo_as_admin_successful
drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 27 2014 traffic
drwxr-xr-x 4 root root 4096 Jun 27 2014 traffic-1.0
-rw-rw-r-- 1 ubuntu ubuntu 16361 Jan 9 2015 traffic-1.0.tar.gz
-rw------- 1 root root 4695 Dec 17 2017 .viminfo
-rw------- 1 ubuntu ubuntu 152 Jul 2 2014 .Xauthority
-rw------- 1 ubuntu ubuntu 29828 Jan 9 2013 .xsession-errors
2026-04-27 05:15:15,761 - DEBUG - Running remote_cmd, Cmd : nc -6 -q 2 -w 5 -u -ll 53 > testfile, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.65, gateway password: c0ntrail123
2026-04-27 05:15:15,761 - DEBUG - nohup nc -6 -q 2 -w 5 -u -ll 53 > testfile &
2026-04-27 05:15:16,344 - DEBUG - None
2026-04-27 05:15:16,345 - DEBUG - Running remote_cmd, Cmd : dd bs=100 count=1 if=/dev/zero of=testfile;nc -6 -q 2 -w 5 -u -p 20001 3c33:f235:115d:a000:aac:a088:0:5 53 < testfile, host_string: ubuntu@169.254.0.6, password: ubuntugateway: ubuntu@10.0.0.33, gateway password: c0ntrail123
2026-04-27 05:15:16,345 - DEBUG - dd bs=100 count=1 if=/dev/zero of=testfile;nc -6 -q 2 -w 5 -u -p 20001 3c33:f235:115d:a000:aac:a088:0:5 53 < testfile
2026-04-27 05:15:18,889 - DEBUG - 1+0 records in
1+0 records out
100 bytes (100 B) copied, 0.000984424 s, 102 kB/s
2026-04-27 05:15:18,890 - DEBUG - Running remote_cmd, Cmd : ls -l testfile, host_string: ubuntu@169.254.0.4, password: ubuntugateway: ubuntu@10.0.0.65, gateway password: c0ntrail123
2026-04-27 05:15:18,890 - DEBUG - ls -l testfile
2026-04-27 05:15:19,296 - DEBUG - -rw-r--r-- 1 root root 100 Apr 27 05:15 testfile
2026-04-27 05:15:19,297 - INFO - File transfer verification for file size 100 passed on the VM ctest-FatFlowSerialIpv6-45601470-02848325
2026-04-27 05:15:19,445 - DEBUG - Forward and reverse flows for match {'sip': '3c33:f235:115d:a000:aac:a088:0:3', 'dip': '3c33:f235:115d:a000:aac:a088:0:5', 'sport': '20000', 'dport': '53', 'proto': '17', 'vrf_id': '4'}: 1, 1
2026-04-27 05:15:19,539 - DEBUG - Forward and reverse flows for match {'sip': '3c33:f235:115d:a000:aac:a088:0:3', 'dip': '3c33:f235:115d:a000:aac:a088:0:5', 'sport': '20000', 'dport': '53', 'proto': '17', 'vrf_id': '3'}: 0, 0
2026-04-27 05:15:19,669 - DEBUG - Forward and reverse flows for match {'sip': '3c33:f235:115d:a000:aac:a088:0:3', 'dip': '3c33:f235:115d:a000:aac:a088:0:5', 'sport': '20001', 'dport': '53', 'proto': '17', 'vrf_id': '4'}: 1, 1
2026-04-27 05:15:19,763 - DEBUG - Forward and reverse flows for match {'sip': '3c33:f235:115d:a000:aac:a088:0:3', 'dip': '3c33:f235:115d:a000:aac:a088:0:5', 'sport': '20001', 'dport': '53', 'proto': '17', 'vrf_id': '3'}: 0, 0
2026-04-27 05:15:19,899 - DEBUG - Forward and reverse flows for match {'sip': '3c33:f235:115d:a000:aac:a088:0:4', 'dip': '3c33:f235:115d:a000:aac:a088:0:5', 'sport': '20000', 'dport': '53', 'proto': '17', 'vrf_id': '4'}: 1, 1
2026-04-27 05:15:19,987 - DEBUG - Forward and reverse flows for match {'sip': '3c33:f235:115d:a000:aac:a088:0:4', 'dip': '3c33:f235:115d:a000:aac:a088:0:5', 'sport': '20000', 'dport': '53', 'proto': '17', 'vrf_id': '3'}: 0, 0
2026-04-27 05:15:20,119 - DEBUG - Forward and reverse flows for match {'sip': '3c33:f235:115d:a000:aac:a088:0:4', 'dip': '3c33:f235:115d:a000:aac:a088:0:5', 'sport': '20001', 'dport': '53', 'proto': '17', 'vrf_id': '4'}: 1, 1
2026-04-27 05:15:20,209 - DEBUG - Forward and reverse flows for match {'sip': '3c33:f235:115d:a000:aac:a088:0:4', 'dip': '3c33:f235:115d:a000:aac:a088:0:5', 'sport': '20001', 'dport': '53', 'proto': '17', 'vrf_id': '3'}: 0, 0
2026-04-27 05:15:20,325 - DEBUG - Forward and reverse flows for match {'sip': '3c33:f235:115d:a000:aac:a088:0:3', 'dip': '3c33:f235:115d:a000:aac:a088:0:5', 'sport': '0', 'dport': '53', 'proto': '17', 'vrf_id': '3'}: 1, 1
2026-04-27 05:15:20,325 - INFO - Fat flow verification PASSED on node: 10.0.0.65 for - SIP: 3c33:f235:115d:a000:aac:a088:0:3, DIP: 3c33:f235:115d:a000:aac:a088:0:5, sport: 0, dport: 53, vrf-id: 3 Fat flow expected: 1, got: 1
2026-04-27 05:15:20,452 - DEBUG - Forward and reverse flows for match {'sip': '3c33:f235:115d:a000:aac:a088:0:3', 'dip': '3c33:f235:115d:a000:aac:a088:0:5', 'sport': '0', 'dport': '53', 'proto': '17', 'vrf_id': '4'}: 0, 0
2026-04-27 05:15:20,452 - INFO - Fat flow verification PASSED on node: 10.0.0.33 for - SIP: 3c33:f235:115d:a000:aac:a088:0:3, DIP: 3c33:f235:115d:a000:aac:a088:0:5, sport: 0, dport: 53, vrf-id: 4 Fat flow expected: 0, got: 0
2026-04-27 05:15:20,537 - DEBUG - Forward and reverse flows for match {'sip': '3c33:f235:115d:a000:aac:a088:0:4', 'dip': '3c33:f235:115d:a000:aac:a088:0:5', 'sport': '0', 'dport': '53', 'proto': '17', 'vrf_id': '3'}: 1, 1
2026-04-27 05:15:20,538 - INFO - Fat flow verification PASSED on node: 10.0.0.65 for - SIP: 3c33:f235:115d:a000:aac:a088:0:4, DIP: 3c33:f235:115d:a000:aac:a088:0:5, sport: 0, dport: 53, vrf-id: 3 Fat flow expected: 1, got: 1
2026-04-27 05:15:20,665 - DEBUG - Forward and reverse flows for match {'sip': '3c33:f235:115d:a000:aac:a088:0:4', 'dip': '3c33:f235:115d:a000:aac:a088:0:5', 'sport': '0', 'dport': '53', 'proto': '17', 'vrf_id': '4'}: 0, 0
2026-04-27 05:15:20,666 - INFO - Fat flow verification PASSED on node: 10.0.0.33 for - SIP: 3c33:f235:115d:a000:aac:a088:0:4, DIP: 3c33:f235:115d:a000:aac:a088:0:5, sport: 0, dport: 53, vrf-id: 4 Fat flow expected: 0, got: 0
2026-04-27 05:15:20,666 - INFO - Fat flow verification passed for protocol udp and port 53
2026-04-27 05:15:20,667 - INFO - Deleting VM ctest-FatFlowSerialIpv6-45601470-02848325
2026-04-27 05:15:20,784 - INFO - Deleting VM ctest-FatFlowSerialIpv6-45601470-65546380
2026-04-27 05:15:20,915 - INFO - Deleting VM ctest-FatFlowSerialIpv6-45601470-21800809
2026-04-27 05:15:21,035 - INFO - Deleting VN ctest-vn-39620653
2026-04-27 05:15:21,099 - DEBUG - VN 29dcdadb-d053-47f1-af47-7c2436e28c02 still in use: Unable to complete operation on network 29dcdadb-d053-47f1-af47-7c2436e28c02. There are one or more ports still in use on the network.
Neutron server returns request_ids: ['req-2b7b9114-deae-47db-a4d9-58f471fbadb9']
2026-04-27 05:15:21,100 - WARNING - Deleting VN ctest-vn-39620653 failed..Will retry
2026-04-27 05:15:23,437 - DEBUG - Response for deleting network ()
2026-04-27 05:15:23,437 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/e213ba45-4b79-41d1-9e2d-756ba32bc56d
2026-04-27 05:15:23,448 - DEBUG - Response Code: 404
2026-04-27 05:15:23,448 - DEBUG - Requesting: http://10.0.0.41:8082/domains
2026-04-27 05:15:23,456 - DEBUG - Requesting: http://10.0.0.41:8082/domain/3cfd3207-f874-4fca-b30c-5a5349ff35bf
2026-04-27 05:15:23,524 - DEBUG - Requesting: http://10.0.0.41:8082/project/3f7d3e8f-603a-440c-96b4-04708c95bbcf
2026-04-27 05:15:23,660 - INFO - Validated that VN ctest-vn-39620653 is not found in API Server
2026-04-27 05:15:23,672 - DEBUG - VN ctest-vn-39620653 is not present in Agent 10.0.0.33
2026-04-27 05:15:23,685 - DEBUG - VN ctest-vn-39620653 is not present in Agent 10.0.0.65
2026-04-27 05:15:23,685 - INFO - Validated that VN ctest-vn-39620653 is not in any agent
2026-04-27 05:15:23,732 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-vn-39620653 info
2026-04-27 05:15:25,055 - DEBUG - Skipping xmpp flap check
2026-04-27 05:15:25,055 - INFO - END TEST : test_fat_flow_intra_vn_inter_node : PASSED[0:04:04]
2026-04-27 05:15:25,056 - INFO - --------------------------------------------------------------------------------
2026-04-27 05:15:25,928 - INFO - Deleted project: ctest-FatFlowSerialIpv6-45601470, ID : 3f7d3e8f-603a-440c-96b4-04708c95bbcf