2025-12-29 02:54:59,180 - INFO - Domain Default found not creating 2025-12-29 02:54:59,359 - INFO - Project ctest-TestQos-20364845 not found, creating it 2025-12-29 02:54:59,873 - INFO - Created Project:ctest-TestQos-20364845, ID : 69ec548c-2f6b-4f89-abbf-f7128ad33e9d 2025-12-29 02:55:02,360 - DEBUG - Response for create_network : {'network': {'id': '681d71d4-cc33-4e8d-99a6-1524b1f56109', 'name': 'ctest-vn-39392062', 'tenant_id': '69ec548c2f6b4f89abbff7128ad33e9d', 'project_id': '69ec548c2f6b4f89abbff7128ad33e9d', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestQos-20364845', 'ctest-vn-39392062'], 'port_security_enabled': True, 'description': ''}} 2025-12-29 02:55:02,703 - DEBUG - Response for create_subnet : {'subnet': {'id': 'd5c7a161-c2cb-4b2e-a558-5613c3a203bf', 'name': '', 'tenant_id': '69ec548c2f6b4f89abbff7128ad33e9d', 'network_id': '681d71d4-cc33-4e8d-99a6-1524b1f56109', 'ip_version': 4, 'cidr': '70.71.152.128/26', 'allocation_pools': [{'start': '70.71.152.130', 'end': '70.71.152.190'}], 'gateway_ip': '70.71.152.129', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '70.71.152.130', 'tags': [], 'project_id': '69ec548c2f6b4f89abbff7128ad33e9d'}} 2025-12-29 02:55:02,961 - DEBUG - Response for create_subnet : {'subnet': {'id': '0f3bb0dc-a154-4789-b59b-d64db2bf6637', 'name': '', 'tenant_id': '69ec548c2f6b4f89abbff7128ad33e9d', 'network_id': '681d71d4-cc33-4e8d-99a6-1524b1f56109', 'ip_version': 6, 'cidr': '3a98:b798:9bb5:9425:126e:7261::/96', 'allocation_pools': [{'start': '3a98:b798:9bb5:9425:126e:7261:0:2', 'end': '3a98:b798:9bb5:9425:126e:7261:ffff:fffe'}], 'gateway_ip': '3a98:b798:9bb5:9425:126e:7261:0:1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '3a98:b798:9bb5:9425:126e:7261:0:2', 'tags': [], 'project_id': '69ec548c2f6b4f89abbff7128ad33e9d'}} 2025-12-29 02:55:02,986 - INFO - Created VN ctest-vn-39392062 2025-12-29 02:55:02,997 - DEBUG - VN ctest-vn-39392062 UUID is 681d71d4-cc33-4e8d-99a6-1524b1f56109 2025-12-29 02:55:03,427 - DEBUG - Response for create_network : {'network': {'id': '1b976e5a-18ee-4574-b8e4-6d9edd9a849d', 'name': 'ctest-vn-93647829', 'tenant_id': '69ec548c2f6b4f89abbff7128ad33e9d', 'project_id': '69ec548c2f6b4f89abbff7128ad33e9d', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestQos-20364845', 'ctest-vn-93647829'], 'port_security_enabled': True, 'description': ''}} 2025-12-29 02:55:03,680 - DEBUG - Response for create_subnet : {'subnet': {'id': 'ae8dbb7c-0645-4fe1-bfa4-78fd4e7a8504', 'name': '', 'tenant_id': '69ec548c2f6b4f89abbff7128ad33e9d', 'network_id': '1b976e5a-18ee-4574-b8e4-6d9edd9a849d', 'ip_version': 4, 'cidr': '71.101.207.192/26', 'allocation_pools': [{'start': '71.101.207.194', 'end': '71.101.207.254'}], 'gateway_ip': '71.101.207.193', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '71.101.207.194', 'tags': [], 'project_id': '69ec548c2f6b4f89abbff7128ad33e9d'}} 2025-12-29 02:55:03,900 - DEBUG - Response for create_subnet : {'subnet': {'id': '93c1197d-50bb-4a79-94a8-15c6a18172da', 'name': '', 'tenant_id': '69ec548c2f6b4f89abbff7128ad33e9d', 'network_id': '1b976e5a-18ee-4574-b8e4-6d9edd9a849d', 'ip_version': 6, 'cidr': '3f8f:4e27:bc50:9a9e:c124:877b::/96', 'allocation_pools': [{'start': '3f8f:4e27:bc50:9a9e:c124:877b:0:2', 'end': '3f8f:4e27:bc50:9a9e:c124:877b:ffff:fffe'}], 'gateway_ip': '3f8f:4e27:bc50:9a9e:c124:877b:0:1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '3f8f:4e27:bc50:9a9e:c124:877b:0:2', 'tags': [], 'project_id': '69ec548c2f6b4f89abbff7128ad33e9d'}} 2025-12-29 02:55:03,930 - INFO - Created VN ctest-vn-93647829 2025-12-29 02:55:03,941 - DEBUG - VN ctest-vn-93647829 UUID is 1b976e5a-18ee-4574-b8e4-6d9edd9a849d 2025-12-29 02:55:04,109 - DEBUG - Services list from nova: [, , , ] 2025-12-29 02:55:05,291 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4934-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4934-1) 2025-12-29 02:55:06,725 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4934-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4934-2) 2025-12-29 02:55:08,283 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4934-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4934-2) 2025-12-29 02:55:08,283 - INFO - Waiting for VM ctest-TestQos-20364845-62335717 to be up.. 2025-12-29 02:55:08,376 - DEBUG - VM is in ACTIVE state now 2025-12-29 02:55:08,377 - INFO - VM name : ctest-TestQos-20364845-62335717 2025-12-29 02:55:08,496 - DEBUG - VM ctest-TestQos-20364845-62335717 ID is 37781cfb-ad89-4a64-be24-5b120a057b3b 2025-12-29 02:55:08,527 - DEBUG - VM ctest-TestQos-20364845-62335717 launched on Node an-jenkins-deploy-platform-ansible-os-4934-1 2025-12-29 02:55:08,619 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-machine/37781cfb-ad89-4a64-be24-5b120a057b3b 2025-12-29 02:55:09,197 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-machine/37781cfb-ad89-4a64-be24-5b120a057b3b 2025-12-29 02:55:09,281 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-machine-interface/0aae0dc3-e8e8-418a-a035-eb315be2f0f1 2025-12-29 02:55:12,458 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1018ms') 2025-12-29 02:55:12,458 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestQos-20364845-62335717 failed! 2025-12-29 02:55:12,471 - DEBUG - Gateway for vn default-domain:ctest-TestQos-20364845:ctest-vn-39392062 is 70.71.152.129 and allocation pool is NOT set 2025-12-29 02:55:12,471 - DEBUG - Gateway for vn default-domain:ctest-TestQos-20364845:ctest-vn-39392062 is 3a98:b798:9bb5:9425:126e:7261:0:1 and allocation pool is NOT set 2025-12-29 02:55:16,564 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1026ms') 2025-12-29 02:55:16,564 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestQos-20364845-62335717 failed! 2025-12-29 02:55:16,579 - DEBUG - Gateway for vn default-domain:ctest-TestQos-20364845:ctest-vn-39392062 is 70.71.152.129 and allocation pool is NOT set 2025-12-29 02:55:16,579 - DEBUG - Gateway for vn default-domain:ctest-TestQos-20364845:ctest-vn-39392062 is 3a98:b798:9bb5:9425:126e:7261:0:1 and allocation pool is NOT set 2025-12-29 02:55:20,656 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1012ms') 2025-12-29 02:55:20,656 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestQos-20364845-62335717 failed! 2025-12-29 02:55:20,669 - DEBUG - Gateway for vn default-domain:ctest-TestQos-20364845:ctest-vn-39392062 is 70.71.152.129 and allocation pool is NOT set 2025-12-29 02:55:20,669 - DEBUG - Gateway for vn default-domain:ctest-TestQos-20364845:ctest-vn-39392062 is 3a98:b798:9bb5:9425:126e:7261:0:1 and allocation pool is NOT set 2025-12-29 02:55:24,756 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1022ms') 2025-12-29 02:55:24,756 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestQos-20364845-62335717 failed! 2025-12-29 02:55:24,768 - DEBUG - Gateway for vn default-domain:ctest-TestQos-20364845:ctest-vn-39392062 is 70.71.152.129 and allocation pool is NOT set 2025-12-29 02:55:24,768 - DEBUG - Gateway for vn default-domain:ctest-TestQos-20364845:ctest-vn-39392062 is 3a98:b798:9bb5:9425:126e:7261:0:1 and allocation pool is NOT set 2025-12-29 02:55:28,844 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1014ms') 2025-12-29 02:55:28,844 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestQos-20364845-62335717 failed! 2025-12-29 02:55:28,857 - DEBUG - Gateway for vn default-domain:ctest-TestQos-20364845:ctest-vn-39392062 is 70.71.152.129 and allocation pool is NOT set 2025-12-29 02:55:28,857 - DEBUG - Gateway for vn default-domain:ctest-TestQos-20364845:ctest-vn-39392062 is 3a98:b798:9bb5:9425:126e:7261:0:1 and allocation pool is NOT set 2025-12-29 02:55:32,946 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1025ms') 2025-12-29 02:55:32,946 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestQos-20364845-62335717 failed! 2025-12-29 02:55:32,966 - DEBUG - Gateway for vn default-domain:ctest-TestQos-20364845:ctest-vn-39392062 is 70.71.152.129 and allocation pool is NOT set 2025-12-29 02:55:32,966 - DEBUG - Gateway for vn default-domain:ctest-TestQos-20364845:ctest-vn-39392062 is 3a98:b798:9bb5:9425:126e:7261:0:1 and allocation pool is NOT set 2025-12-29 02:55:37,043 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1010ms') 2025-12-29 02:55:37,043 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestQos-20364845-62335717 failed! 2025-12-29 02:55:37,057 - DEBUG - Gateway for vn default-domain:ctest-TestQos-20364845:ctest-vn-39392062 is 70.71.152.129 and allocation pool is NOT set 2025-12-29 02:55:37,057 - DEBUG - Gateway for vn default-domain:ctest-TestQos-20364845:ctest-vn-39392062 is 3a98:b798:9bb5:9425:126e:7261:0:1 and allocation pool is NOT set 2025-12-29 02:55:41,130 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1010ms') 2025-12-29 02:55:41,130 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestQos-20364845-62335717 failed! 2025-12-29 02:55:41,145 - DEBUG - Gateway for vn default-domain:ctest-TestQos-20364845:ctest-vn-39392062 is 70.71.152.129 and allocation pool is NOT set 2025-12-29 02:55:41,145 - DEBUG - Gateway for vn default-domain:ctest-TestQos-20364845:ctest-vn-39392062 is 3a98:b798:9bb5:9425:126e:7261:0:1 and allocation pool is NOT set 2025-12-29 02:55:45,227 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1021ms') 2025-12-29 02:55:45,227 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestQos-20364845-62335717 failed! 2025-12-29 02:55:45,240 - DEBUG - Gateway for vn default-domain:ctest-TestQos-20364845:ctest-vn-39392062 is 70.71.152.129 and allocation pool is NOT set 2025-12-29 02:55:45,240 - DEBUG - Gateway for vn default-domain:ctest-TestQos-20364845:ctest-vn-39392062 is 3a98:b798:9bb5:9425:126e:7261:0:1 and allocation pool is NOT set 2025-12-29 02:55:49,332 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1026ms') 2025-12-29 02:55:49,332 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestQos-20364845-62335717 failed! 2025-12-29 02:55:49,346 - DEBUG - Gateway for vn default-domain:ctest-TestQos-20364845:ctest-vn-39392062 is 70.71.152.129 and allocation pool is NOT set 2025-12-29 02:55:49,347 - DEBUG - Gateway for vn default-domain:ctest-TestQos-20364845:ctest-vn-39392062 is 3a98:b798:9bb5:9425:126e:7261:0:1 and allocation pool is NOT set 2025-12-29 02:55:51,410 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=10.4 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=0.726 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1002ms\r\nrtt min/avg/max/mdev = 0.726/5.587/10.448/4.861 ms') 2025-12-29 02:55:51,410 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-TestQos-20364845-62335717 passed 2025-12-29 02:55:51,481 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-12-29 02:55:51,481 - DEBUG - Waiting to SSH to VM ctest-TestQos-20364845-62335717, IP 70.71.152.131, Port 22 2025-12-29 02:55:51,547 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-12-29 02:55:51,632 - DEBUG - VM ctest-TestQos-20364845-62335717 is NOT ready for SSH connections, VM status: ACTIVE 2025-12-29 02:55:56,633 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-12-29 02:55:56,633 - DEBUG - Waiting to SSH to VM ctest-TestQos-20364845-62335717, IP 70.71.152.131, Port 22 2025-12-29 02:55:56,700 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-12-29 02:55:56,777 - DEBUG - VM ctest-TestQos-20364845-62335717 is NOT ready for SSH connections, VM status: ACTIVE 2025-12-29 02:56:01,778 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-12-29 02:56:01,778 - DEBUG - Waiting to SSH to VM ctest-TestQos-20364845-62335717, IP 70.71.152.131, Port 22 2025-12-29 02:56:01,843 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-12-29 02:56:01,925 - DEBUG - VM ctest-TestQos-20364845-62335717 is NOT ready for SSH connections, VM status: ACTIVE 2025-12-29 02:56:06,926 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-12-29 02:56:06,926 - DEBUG - Waiting to SSH to VM ctest-TestQos-20364845-62335717, IP 70.71.152.131, Port 22 2025-12-29 02:56:06,995 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-12-29 02:56:07,081 - DEBUG - VM ctest-TestQos-20364845-62335717 is NOT ready for SSH connections, VM status: ACTIVE 2025-12-29 02:56:12,082 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-12-29 02:56:12,082 - DEBUG - Waiting to SSH to VM ctest-TestQos-20364845-62335717, IP 70.71.152.131, Port 22 2025-12-29 02:56:12,150 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-12-29 02:56:12,232 - DEBUG - VM ctest-TestQos-20364845-62335717 is NOT ready for SSH connections, VM status: ACTIVE 2025-12-29 02:56:17,233 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-12-29 02:56:17,233 - DEBUG - Waiting to SSH to VM ctest-TestQos-20364845-62335717, IP 70.71.152.131, Port 22 2025-12-29 02:56:17,399 - DEBUG - VM ctest-TestQos-20364845-62335717 is ready for SSH connections 2025-12-29 02:56:17,399 - INFO - Waiting for VM ctest-TestQos-20364845-10325055 to be up.. 2025-12-29 02:56:17,473 - DEBUG - VM is in ACTIVE state now 2025-12-29 02:56:17,473 - INFO - VM name : ctest-TestQos-20364845-10325055 2025-12-29 02:56:17,546 - DEBUG - VM ctest-TestQos-20364845-10325055 ID is 134a1f93-2fa6-4142-b257-f05b6ab0d457 2025-12-29 02:56:17,546 - DEBUG - VM ctest-TestQos-20364845-10325055 launched on Node an-jenkins-deploy-platform-ansible-os-4934-2 2025-12-29 02:56:17,616 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-machine/134a1f93-2fa6-4142-b257-f05b6ab0d457 2025-12-29 02:56:17,626 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-machine-interface/2bf7a626-7803-42bf-9cbb-e5cbcbdf5186 2025-12-29 02:56:18,795 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=5.30 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=1.17 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 1.170/3.234/5.299/2.064 ms') 2025-12-29 02:56:18,795 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-TestQos-20364845-10325055 passed 2025-12-29 02:56:18,866 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-12-29 02:56:18,866 - DEBUG - Waiting to SSH to VM ctest-TestQos-20364845-10325055, IP 70.71.152.132, Port 22 2025-12-29 02:56:19,022 - DEBUG - VM ctest-TestQos-20364845-10325055 is ready for SSH connections 2025-12-29 02:56:19,022 - INFO - Waiting for VM ctest-TestQos-20364845-03099817 to be up.. 2025-12-29 02:56:19,106 - DEBUG - VM is in ACTIVE state now 2025-12-29 02:56:19,106 - INFO - VM name : ctest-TestQos-20364845-03099817 2025-12-29 02:56:19,188 - DEBUG - VM ctest-TestQos-20364845-03099817 ID is 14591095-1cf2-4d3b-859f-54daf00d853d 2025-12-29 02:56:19,188 - DEBUG - VM ctest-TestQos-20364845-03099817 launched on Node an-jenkins-deploy-platform-ansible-os-4934-2 2025-12-29 02:56:19,267 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-machine/14591095-1cf2-4d3b-859f-54daf00d853d 2025-12-29 02:56:19,277 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-machine-interface/f040fc16-5df1-447b-b333-0203848f8e6f 2025-12-29 02:56:20,428 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n64 bytes from 169.254.0.4: icmp_seq=1 ttl=63 time=3.75 ms\r\n64 bytes from 169.254.0.4: icmp_seq=2 ttl=63 time=1.28 ms\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1002ms\r\nrtt min/avg/max/mdev = 1.284/2.517/3.750/1.233 ms') 2025-12-29 02:56:20,428 - INFO - Ping to Metadata IP 169.254.0.4 of VM ctest-TestQos-20364845-03099817 passed 2025-12-29 02:56:20,492 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-12-29 02:56:20,492 - DEBUG - Waiting to SSH to VM ctest-TestQos-20364845-03099817, IP 71.101.207.195, Port 22 2025-12-29 02:56:20,653 - DEBUG - VM ctest-TestQos-20364845-03099817 is ready for SSH connections 2025-12-29 02:56:20,656 - INFO - ================================================================================ 2025-12-29 02:56:20,656 - INFO - STARTING TEST : test_qos_remark_dscp_on_vmi 2025-12-29 02:56:20,656 - INFO - TEST DESCRIPTION : Create a qos config for remarking DSCP 1 to 10 Have VMs A, B Apply the qos config to VM A Validate that packets from A to B have DSCP marked correctly 2025-12-29 02:56:21,921 - DEBUG - Nothing to compare xmpp stats {'10.0.0.56': {'10.20.0.145': '0', '10.20.0.18': '0'}, '10.0.0.240': {'10.20.0.252': '0', '10.20.0.18': '0'}} with 2025-12-29 02:56:21,922 - INFO - Initial checks done. Running the testcase now 2025-12-29 02:56:21,922 - INFO - 2025-12-29 02:56:21,928 - DEBUG - FC Dict is {'fc_id': 0, 'dscp': 10, 'dot1p': 1, 'exp': 1, 'connections': } 2025-12-29 02:56:22,192 - INFO - Created FC ['default-global-system-config', 'default-global-qos-config', 'ctest-fc-50904664'], UUID df78cfa0-8d93-4840-9e78-a8556e5fed8b 2025-12-29 02:56:22,537 - INFO - Created QosConfig ['default-domain', 'ctest-TestQos-20364845', 'ctest-qos_config-02718163'], UUID: 2bb82974-bde3-4c6b-b27e-4d2da0c73eb6 2025-12-29 02:56:22,584 - INFO - Applying qos-config on VM 0aae0dc3-e8e8-418a-a035-eb315be2f0f1 2025-12-29 02:56:22,817 - DEBUG - VM is in ACTIVE state now 2025-12-29 02:56:23,038 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-machine/37781cfb-ad89-4a64-be24-5b120a057b3b 2025-12-29 02:56:23,047 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-machine-interface/0aae0dc3-e8e8-418a-a035-eb315be2f0f1 2025-12-29 02:56:23,060 - DEBUG - Requesting: http://10.0.0.27:8082/instance-ip/138815ae-c314-4932-becc-bec89c4d26c9 2025-12-29 02:56:23,071 - DEBUG - Requesting: http://10.0.0.27:8082/instance-ip/c0e2a052-86fc-4fab-8f11-63a04a384f0a 2025-12-29 02:56:23,258 - DEBUG - VM is in ACTIVE state now 2025-12-29 02:56:23,479 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-machine/134a1f93-2fa6-4142-b257-f05b6ab0d457 2025-12-29 02:56:23,486 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-machine-interface/2bf7a626-7803-42bf-9cbb-e5cbcbdf5186 2025-12-29 02:56:23,497 - DEBUG - Requesting: http://10.0.0.27:8082/instance-ip/6f230a8a-19ea-4b89-bbf3-df14576fe979 2025-12-29 02:56:23,506 - DEBUG - Requesting: http://10.0.0.27:8082/instance-ip/980283d0-7d01-42a4-bfc2-0ac0b5d3e12f 2025-12-29 02:56:23,547 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-12-29 02:56:23,547 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-12-29 02:56:23,547 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-12-29 02:56:23,547 - INFO - Starting hping3 on ctest-TestQos-20364845-62335717, args: --destport 20000 --baseport 10000 --count 20000 --interval 1 --udp --tos 4 --keep --numeric 2025-12-29 02:56:23,548 - DEBUG - Hping3 cmd : hping3 --destport 20000 --baseport 10000 --count 20000 --interval 1 --udp --tos 4 --keep --numeric 70.71.152.132 1>/tmp/hping_ctest-random-04065261.log 2>/tmp/hping_ctest-random-04065261.result 2025-12-29 02:56:23,548 - DEBUG - Running remote_cmd, Cmd : hping3 --destport 20000 --baseport 10000 --count 20000 --interval 1 --udp --tos 4 --keep --numeric 70.71.152.132 1>/tmp/hping_ctest-random-04065261.log 2>/tmp/hping_ctest-random-04065261.result, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.56, gateway password: c0ntrail123 2025-12-29 02:56:23,548 - DEBUG - nohup hping3 --destport 20000 --baseport 10000 --count 20000 --interval 1 --udp --tos 4 --keep --numeric 70.71.152.132 1>/tmp/hping_ctest-random-04065261.log 2>/tmp/hping_ctest-random-04065261.result & echo $! > /tmp/hping_ctest-random-04065261.pid 2025-12-29 02:56:39,743 - DEBUG - None 2025-12-29 02:56:44,956 - DEBUG - Forward flow: {'index': '377236', 'rflow': '287956', 'sip': '70.71.152.131', 'sport': '10000', 'dip': '70.71.152.132', 'dport': '20000', 'proto': '17', 'vrf_id': '2', 'action': 'FORWARD', 'flags': ' ACTIVE | RFLOW_VALID ', 'd_vrf_id': '0', 'bytes': '252', 'pkts': '6', 'insight': '0', 'nhid': '30', 'underlay_udp_sport': '58728', 'ttl': '0', 'qos_id': '0', 'gen_id': '1', 'tcp_seq': '0', 'oflow_bytes': '0', 'oflow_packets': '0', 'underlay_gw_index': '-1'} 2025-12-29 02:56:44,957 - DEBUG - Reverse flow: {'index': '287956', 'rflow': '377236', 'sip': '70.71.152.132', 'sport': '20000', 'dip': '70.71.152.131', 'dport': '10000', 'proto': '17', 'vrf_id': '2', 'action': 'FORWARD', 'flags': ' ACTIVE | RFLOW_VALID ', 'd_vrf_id': '0', 'bytes': '350', 'pkts': '5', 'insight': '0', 'nhid': '30', 'underlay_udp_sport': '64670', 'ttl': '0', 'qos_id': '0', 'gen_id': '1', 'tcp_seq': '0', 'oflow_bytes': '0', 'oflow_packets': '0', 'underlay_gw_index': '-1'} 2025-12-29 02:56:44,957 - DEBUG - The filter pattern is ['udp', 'and', 'src port 58728'] 2025-12-29 02:56:44,957 - DEBUG - The filter string is '(udp and src port 58728)' 2025-12-29 02:56:45,054 - DEBUG - Executing command: sudo tcpdump -nni ens6 -U -vvxx '(udp and src port 58728)' -w /tmp/ens6_ctest-random-14243955.pcap 2025-12-29 02:56:50,163 - DEBUG - Executing command: sudo kill $(ps -ef|grep tcpdump | grep pcap| awk '{print $2}') 2025-12-29 02:56:52,166 - DEBUG - Ensuring hping3 instance with result file /tmp/hping_ctest-random-04065261.result on ctest-TestQos-20364845-62335717 is stopped 2025-12-29 02:56:52,166 - DEBUG - Running remote_cmd, Cmd : cat /tmp/hping_ctest-random-04065261.pid | xargs kill , host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.56, gateway password: c0ntrail123 2025-12-29 02:56:52,167 - DEBUG - cat /tmp/hping_ctest-random-04065261.pid | xargs kill 2025-12-29 02:56:54,138 - DEBUG - None 2025-12-29 02:56:54,138 - DEBUG - Running remote_cmd, Cmd : cat /tmp/hping_ctest-random-04065261.result, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.56, gateway password: c0ntrail123 2025-12-29 02:56:54,138 - DEBUG - cat /tmp/hping_ctest-random-04065261.result 2025-12-29 02:56:54,879 - DEBUG - --- 70.71.152.132 hping statistic --- 15 packets transmitted, 15 packets received, 0% packet loss round-trip min/avg/max = 14.4/7012.0/14015.9 ms 2025-12-29 02:56:54,879 - DEBUG - Running remote_cmd, Cmd : cat /tmp/hping_ctest-random-04065261.log, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.56, gateway password: c0ntrail123 2025-12-29 02:56:54,879 - DEBUG - cat /tmp/hping_ctest-random-04065261.log 2025-12-29 02:56:55,305 - DEBUG - HPING 70.71.152.132 (eth0 70.71.152.132): udp mode set, 28 headers + 0 data bytes ICMP Port Unreachable from ip=70.71.152.132 status=0 port=10000 seq=0 ICMP Port Unreachable from ip=70.71.152.132 status=1 port=10000 seq=0 ICMP Port Unreachable from ip=70.71.152.132 status=1 port=10000 seq=0 ICMP Port Unreachable from ip=70.71.152.132 status=1 port=10000 seq=0 ICMP Port Unreachable from ip=70.71.152.132 status=1 port=10000 seq=0 ICMP Port Unreachable from ip=70.71.152.132 status=1 port=10000 seq=0 ICMP Port Unreachable from ip=70.71.152.132 status=1 port=10000 seq=0 ICMP Port Unreachable from ip=70.71.152.132 status=1 port=10000 seq=0 ICMP Port Unreachable from ip=70.71.152.132 status=1 port=10000 seq=0 ICMP Port Unreachable from ip=70.71.152.132 status=1 port=10000 seq=0 ICMP Port Unreachable from ip=70.71.152.132 status=1 port=10000 seq=0 ICMP Port Unreachable from ip=70.71.152.132 status=1 port=10000 seq=0 ICMP Port Unreachable from ip=70.71.152.132 status=1 port=10000 seq=0 ICMP Port Unreachable from ip=70.71.152.132 status=1 port=10000 seq=0 ICMP Port Unreachable from ip=70.71.152.132 status=1 port=10000 seq=0 2025-12-29 02:56:55,306 - DEBUG - Hping3 stats: {'sent': '15', 'received': '15', 'loss': '0', 'rtt_min': '14.4', 'rtt_avg': '7012.0', 'rtt_max': '14015.9'} 2025-12-29 02:56:55,306 - DEBUG - Executing command: sudo tcpdump -nnr /tmp/ens6_ctest-random-14243955.pcap | wc -l 2025-12-29 02:56:55,328 - DEBUG - STDOUT: 16 2025-12-29 02:56:55,329 - DEBUG - STDERR: reading from file /tmp/ens6_ctest-random-14243955.pcap, link-type EN10MB (Ethernet), snapshot length 262144 2025-12-29 02:56:55,329 - DEBUG - 16 packets are found in tcpdump output file /tmp/ens6_ctest-random-14243955.pcap but expected 1, which is fine 2025-12-29 02:56:55,329 - INFO - 16 packets are found in tcpdump output as expected 2025-12-29 02:56:55,329 - DEBUG - Executing command: sudo kill $(ps -ef|grep tcpdump | grep pcap| awk '{print $2}') 2025-12-29 02:56:57,393 - DEBUG - ['/contrail-test/ens6_ctest-random-14243955.pcap'] 2025-12-29 02:56:57,394 - DEBUG - Verifying for packet number 0 2025-12-29 02:56:57,394 - DEBUG - Validated DSCP marking of 10 2025-12-29 02:56:57,394 - DEBUG - Verifying for packet number 1 2025-12-29 02:56:57,394 - DEBUG - Validated DSCP marking of 10 2025-12-29 02:56:57,394 - DEBUG - Verifying for packet number 2 2025-12-29 02:56:57,394 - DEBUG - Validated DSCP marking of 10 2025-12-29 02:56:57,394 - DEBUG - Verifying for packet number 3 2025-12-29 02:56:57,394 - DEBUG - Validated DSCP marking of 10 2025-12-29 02:56:57,394 - INFO - Packet QoS marking validation passed 2025-12-29 02:56:57,394 - DEBUG - Executing command: sudo tcpdump -nnr /tmp/ens6_ctest-random-14243955.pcap | wc -l 2025-12-29 02:56:57,412 - DEBUG - STDOUT: 16 2025-12-29 02:56:57,412 - DEBUG - STDERR: reading from file /tmp/ens6_ctest-random-14243955.pcap, link-type EN10MB (Ethernet), snapshot length 262144 2025-12-29 02:56:57,412 - DEBUG - 16 packets are found in tcpdump output file /tmp/ens6_ctest-random-14243955.pcap but expected 1, which is fine 2025-12-29 02:56:57,412 - INFO - 16 packets are found in tcpdump output as expected 2025-12-29 02:56:57,412 - DEBUG - Executing command: sudo kill $(ps -ef|grep tcpdump | grep pcap| awk '{print $2}') 2025-12-29 02:56:59,470 - DEBUG - ['/contrail-test/ens6_ctest-random-14243955.pcap'] 2025-12-29 02:56:59,471 - DEBUG - Verifying for packet number 0 2025-12-29 02:56:59,471 - DEBUG - Interface ens6 does not seem to be a tagged intf. Skipping dot1p check 2025-12-29 02:56:59,471 - DEBUG - Verifying for packet number 1 2025-12-29 02:56:59,471 - DEBUG - Interface ens6 does not seem to be a tagged intf. Skipping dot1p check 2025-12-29 02:56:59,471 - DEBUG - Verifying for packet number 2 2025-12-29 02:56:59,471 - DEBUG - Interface ens6 does not seem to be a tagged intf. Skipping dot1p check 2025-12-29 02:56:59,471 - DEBUG - Verifying for packet number 3 2025-12-29 02:56:59,471 - DEBUG - Interface ens6 does not seem to be a tagged intf. Skipping dot1p check 2025-12-29 02:56:59,471 - INFO - Packet QoS marking validation passed 2025-12-29 02:56:59,579 - INFO - Removing qos-config on VM 0aae0dc3-e8e8-418a-a035-eb315be2f0f1 2025-12-29 02:56:59,644 - INFO - Deleting Qos config ['default-domain', 'ctest-TestQos-20364845', 'ctest-qos_config-02718163'], UUID: 2bb82974-bde3-4c6b-b27e-4d2da0c73eb6 2025-12-29 02:56:59,728 - INFO - Deleting FC ['default-global-system-config', 'default-global-qos-config', 'ctest-fc-50904664'], UUID: df78cfa0-8d93-4840-9e78-a8556e5fed8b 2025-12-29 02:57:01,050 - DEBUG - No XMPP flaps were noticed during the test 2025-12-29 02:57:01,050 - INFO - END TEST : test_qos_remark_dscp_on_vmi : PASSED[0:00:41] 2025-12-29 02:57:01,050 - INFO - -------------------------------------------------------------------------------- 2025-12-29 02:57:01,052 - INFO - Deleting VM ctest-TestQos-20364845-03099817 2025-12-29 02:57:01,131 - INFO - Deleting VM ctest-TestQos-20364845-10325055 2025-12-29 02:57:01,196 - INFO - Deleting VM ctest-TestQos-20364845-62335717 2025-12-29 02:57:01,277 - INFO - Deleting VN ctest-vn-93647829 2025-12-29 02:57:01,340 - DEBUG - VN 1b976e5a-18ee-4574-b8e4-6d9edd9a849d still in use: Unable to complete operation on network 1b976e5a-18ee-4574-b8e4-6d9edd9a849d. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-1910226e-2dd2-4bea-8f12-aad1ad5be828'] 2025-12-29 02:57:01,340 - WARNING - Deleting VN ctest-vn-93647829 failed..Will retry 2025-12-29 02:57:03,533 - DEBUG - Response for deleting network () 2025-12-29 02:57:03,533 - INFO - Deleting VN ctest-vn-39392062 2025-12-29 02:57:03,727 - DEBUG - Response for deleting network () 2025-12-29 02:57:04,548 - INFO - Deleted project: ctest-TestQos-20364845, ID : 69ec548c-2f6b-4f89-abbf-f7128ad33e9d