2026-04-27 03:21:38,602 - INFO - Domain Default found not creating 2026-04-27 03:21:38,855 - INFO - Project ctest-TestQos-23217535 not found, creating it 2026-04-27 03:21:39,576 - INFO - Created Project:ctest-TestQos-23217535, ID : 3ddb8241-a3b1-4197-88ce-80756743ebab 2026-04-27 03:21:43,022 - DEBUG - Response for create_network : {'network': {'id': 'a1169fba-cc51-46a3-a743-cc35bd51bbd6', 'name': 'ctest-vn-02737557', 'tenant_id': '3ddb8241a3b1419788ce80756743ebab', 'project_id': '3ddb8241a3b1419788ce80756743ebab', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestQos-23217535', 'ctest-vn-02737557'], 'port_security_enabled': True, 'description': ''}} 2026-04-27 03:21:43,363 - DEBUG - Response for create_subnet : {'subnet': {'id': '144c9794-5ef3-439a-96ea-c7efe90182cf', 'name': '', 'tenant_id': '3ddb8241a3b1419788ce80756743ebab', 'network_id': 'a1169fba-cc51-46a3-a743-cc35bd51bbd6', 'ip_version': 4, 'cidr': '158.27.187.128/26', 'allocation_pools': [{'start': '158.27.187.130', 'end': '158.27.187.190'}], 'gateway_ip': '158.27.187.129', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '158.27.187.130', 'tags': [], 'project_id': '3ddb8241a3b1419788ce80756743ebab'}} 2026-04-27 03:21:43,672 - DEBUG - Response for create_subnet : {'subnet': {'id': '4823f2de-f2ae-4284-b97b-874b775bfc57', 'name': '', 'tenant_id': '3ddb8241a3b1419788ce80756743ebab', 'network_id': 'a1169fba-cc51-46a3-a743-cc35bd51bbd6', 'ip_version': 6, 'cidr': '2f65:1228:4e29:7e60:9df1:e430::/96', 'allocation_pools': [{'start': '2f65:1228:4e29:7e60:9df1:e430:0:2', 'end': '2f65:1228:4e29:7e60:9df1:e430:ffff:fffe'}], 'gateway_ip': '2f65:1228:4e29:7e60:9df1:e430:0:1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '2f65:1228:4e29:7e60:9df1:e430:0:2', 'tags': [], 'project_id': '3ddb8241a3b1419788ce80756743ebab'}} 2026-04-27 03:21:43,709 - INFO - Created VN ctest-vn-02737557 2026-04-27 03:21:43,726 - DEBUG - VN ctest-vn-02737557 UUID is a1169fba-cc51-46a3-a743-cc35bd51bbd6 2026-04-27 03:21:44,334 - DEBUG - Response for create_network : {'network': {'id': 'bfec5062-57ab-4529-8c61-2bb14f996dad', 'name': 'ctest-vn-30316986', 'tenant_id': '3ddb8241a3b1419788ce80756743ebab', 'project_id': '3ddb8241a3b1419788ce80756743ebab', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestQos-23217535', 'ctest-vn-30316986'], 'port_security_enabled': True, 'description': ''}} 2026-04-27 03:21:44,716 - DEBUG - Response for create_subnet : {'subnet': {'id': 'c7da814e-2d22-4d82-b676-d97a29be996f', 'name': '', 'tenant_id': '3ddb8241a3b1419788ce80756743ebab', 'network_id': 'bfec5062-57ab-4529-8c61-2bb14f996dad', 'ip_version': 4, 'cidr': '106.83.38.192/26', 'allocation_pools': [{'start': '106.83.38.194', 'end': '106.83.38.254'}], 'gateway_ip': '106.83.38.193', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '106.83.38.194', 'tags': [], 'project_id': '3ddb8241a3b1419788ce80756743ebab'}} 2026-04-27 03:21:45,017 - DEBUG - Response for create_subnet : {'subnet': {'id': 'fb36cf90-67f2-449b-9660-01d4379286a9', 'name': '', 'tenant_id': '3ddb8241a3b1419788ce80756743ebab', 'network_id': 'bfec5062-57ab-4529-8c61-2bb14f996dad', 'ip_version': 6, 'cidr': '3893:fe:fdd9:77c2:c837:5c23::/96', 'allocation_pools': [{'start': '3893:fe:fdd9:77c2:c837:5c23:0:2', 'end': '3893:fe:fdd9:77c2:c837:5c23:ffff:fffe'}], 'gateway_ip': '3893:fe:fdd9:77c2:c837:5c23:0:1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '3893:fe:fdd9:77c2:c837:5c23:0:2', 'tags': [], 'project_id': '3ddb8241a3b1419788ce80756743ebab'}} 2026-04-27 03:21:45,064 - INFO - Created VN ctest-vn-30316986 2026-04-27 03:21:45,095 - DEBUG - VN ctest-vn-30316986 UUID is bfec5062-57ab-4529-8c61-2bb14f996dad 2026-04-27 03:21:45,360 - DEBUG - Services list from nova: [, , , ] 2026-04-27 03:21:46,733 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-5690-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-5690-1) 2026-04-27 03:21:48,332 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-5690-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-5690-2) 2026-04-27 03:21:50,141 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-5690-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-5690-2) 2026-04-27 03:21:50,142 - INFO - Waiting for VM ctest-TestQos-23217535-05775201 to be up.. 2026-04-27 03:21:50,277 - DEBUG - VM is in ACTIVE state now 2026-04-27 03:21:50,277 - INFO - VM name : ctest-TestQos-23217535-05775201 2026-04-27 03:21:50,426 - DEBUG - VM ctest-TestQos-23217535-05775201 ID is 0a2e6917-9e4c-4f17-9e63-f3c41c65f6fe 2026-04-27 03:21:50,470 - DEBUG - VM ctest-TestQos-23217535-05775201 launched on Node an-jenkins-deploy-platform-ansible-os-5690-1 2026-04-27 03:21:50,650 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/0a2e6917-9e4c-4f17-9e63-f3c41c65f6fe 2026-04-27 03:21:51,272 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/0a2e6917-9e4c-4f17-9e63-f3c41c65f6fe 2026-04-27 03:21:51,445 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/76c3a6b2-a88f-42e7-a3ae-8221d4c3fb4e 2026-04-27 03:21:54,654 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1006ms') 2026-04-27 03:21:54,654 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestQos-23217535-05775201 failed! 2026-04-27 03:21:54,671 - DEBUG - Gateway for vn default-domain:ctest-TestQos-23217535:ctest-vn-02737557 is 158.27.187.129 and allocation pool is NOT set 2026-04-27 03:21:54,671 - DEBUG - Gateway for vn default-domain:ctest-TestQos-23217535:ctest-vn-02737557 is 2f65:1228:4e29:7e60:9df1:e430:0:1 and allocation pool is NOT set 2026-04-27 03:21:58,756 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1019ms') 2026-04-27 03:21:58,756 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestQos-23217535-05775201 failed! 2026-04-27 03:21:58,776 - DEBUG - Gateway for vn default-domain:ctest-TestQos-23217535:ctest-vn-02737557 is 158.27.187.129 and allocation pool is NOT set 2026-04-27 03:21:58,776 - DEBUG - Gateway for vn default-domain:ctest-TestQos-23217535:ctest-vn-02737557 is 2f65:1228:4e29:7e60:9df1:e430:0:1 and allocation pool is NOT set 2026-04-27 03:22:02,847 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1003ms') 2026-04-27 03:22:02,847 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestQos-23217535-05775201 failed! 2026-04-27 03:22:02,870 - DEBUG - Gateway for vn default-domain:ctest-TestQos-23217535:ctest-vn-02737557 is 158.27.187.129 and allocation pool is NOT set 2026-04-27 03:22:02,870 - DEBUG - Gateway for vn default-domain:ctest-TestQos-23217535:ctest-vn-02737557 is 2f65:1228:4e29:7e60:9df1:e430:0:1 and allocation pool is NOT set 2026-04-27 03:22:06,940 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1010ms') 2026-04-27 03:22:06,940 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestQos-23217535-05775201 failed! 2026-04-27 03:22:06,960 - DEBUG - Gateway for vn default-domain:ctest-TestQos-23217535:ctest-vn-02737557 is 158.27.187.129 and allocation pool is NOT set 2026-04-27 03:22:06,961 - DEBUG - Gateway for vn default-domain:ctest-TestQos-23217535:ctest-vn-02737557 is 2f65:1228:4e29:7e60:9df1:e430:0:1 and allocation pool is NOT set 2026-04-27 03:22:11,032 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1015ms') 2026-04-27 03:22:11,033 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestQos-23217535-05775201 failed! 2026-04-27 03:22:11,052 - DEBUG - Gateway for vn default-domain:ctest-TestQos-23217535:ctest-vn-02737557 is 158.27.187.129 and allocation pool is NOT set 2026-04-27 03:22:11,053 - DEBUG - Gateway for vn default-domain:ctest-TestQos-23217535:ctest-vn-02737557 is 2f65:1228:4e29:7e60:9df1:e430:0:1 and allocation pool is NOT set 2026-04-27 03:22:15,137 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1023ms') 2026-04-27 03:22:15,137 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestQos-23217535-05775201 failed! 2026-04-27 03:22:15,156 - DEBUG - Gateway for vn default-domain:ctest-TestQos-23217535:ctest-vn-02737557 is 158.27.187.129 and allocation pool is NOT set 2026-04-27 03:22:15,157 - DEBUG - Gateway for vn default-domain:ctest-TestQos-23217535:ctest-vn-02737557 is 2f65:1228:4e29:7e60:9df1:e430:0:1 and allocation pool is NOT set 2026-04-27 03:22:19,231 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1007ms') 2026-04-27 03:22:19,232 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestQos-23217535-05775201 failed! 2026-04-27 03:22:19,253 - DEBUG - Gateway for vn default-domain:ctest-TestQos-23217535:ctest-vn-02737557 is 158.27.187.129 and allocation pool is NOT set 2026-04-27 03:22:19,254 - DEBUG - Gateway for vn default-domain:ctest-TestQos-23217535:ctest-vn-02737557 is 2f65:1228:4e29:7e60:9df1:e430:0:1 and allocation pool is NOT set 2026-04-27 03:22:23,329 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1011ms') 2026-04-27 03:22:23,329 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestQos-23217535-05775201 failed! 2026-04-27 03:22:23,351 - DEBUG - Gateway for vn default-domain:ctest-TestQos-23217535:ctest-vn-02737557 is 158.27.187.129 and allocation pool is NOT set 2026-04-27 03:22:23,351 - DEBUG - Gateway for vn default-domain:ctest-TestQos-23217535:ctest-vn-02737557 is 2f65:1228:4e29:7e60:9df1:e430:0:1 and allocation pool is NOT set 2026-04-27 03:22:27,421 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=9.65 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 1 received, 50% packet loss, time 1011ms\r\nrtt min/avg/max/mdev = 9.653/9.653/9.653/0.000 ms') 2026-04-27 03:22:27,421 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-TestQos-23217535-05775201 passed 2026-04-27 03:22:27,498 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2026-04-27 03:22:27,498 - DEBUG - Waiting to SSH to VM ctest-TestQos-23217535-05775201, IP 158.27.187.131, Port 22 2026-04-27 03:22:27,567 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2026-04-27 03:22:27,669 - DEBUG - VM ctest-TestQos-23217535-05775201 is NOT ready for SSH connections, VM status: ACTIVE 2026-04-27 03:22:32,671 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2026-04-27 03:22:32,671 - DEBUG - Waiting to SSH to VM ctest-TestQos-23217535-05775201, IP 158.27.187.131, Port 22 2026-04-27 03:22:32,740 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2026-04-27 03:22:32,851 - DEBUG - VM ctest-TestQos-23217535-05775201 is NOT ready for SSH connections, VM status: ACTIVE 2026-04-27 03:22:37,852 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2026-04-27 03:22:37,852 - DEBUG - Waiting to SSH to VM ctest-TestQos-23217535-05775201, IP 158.27.187.131, Port 22 2026-04-27 03:22:37,919 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2026-04-27 03:22:38,025 - DEBUG - VM ctest-TestQos-23217535-05775201 is NOT ready for SSH connections, VM status: ACTIVE 2026-04-27 03:22:43,027 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2026-04-27 03:22:43,027 - DEBUG - Waiting to SSH to VM ctest-TestQos-23217535-05775201, IP 158.27.187.131, Port 22 2026-04-27 03:22:43,098 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2026-04-27 03:22:43,228 - DEBUG - VM ctest-TestQos-23217535-05775201 is NOT ready for SSH connections, VM status: ACTIVE 2026-04-27 03:22:48,229 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2026-04-27 03:22:48,229 - DEBUG - Waiting to SSH to VM ctest-TestQos-23217535-05775201, IP 158.27.187.131, Port 22 2026-04-27 03:22:48,396 - DEBUG - VM ctest-TestQos-23217535-05775201 is ready for SSH connections 2026-04-27 03:22:48,396 - INFO - Waiting for VM ctest-TestQos-23217535-00835800 to be up.. 2026-04-27 03:22:48,521 - DEBUG - VM is in ACTIVE state now 2026-04-27 03:22:48,521 - INFO - VM name : ctest-TestQos-23217535-00835800 2026-04-27 03:22:48,624 - DEBUG - VM ctest-TestQos-23217535-00835800 ID is 7339268a-4ead-4a74-ab1a-3a9417a482ba 2026-04-27 03:22:48,625 - DEBUG - VM ctest-TestQos-23217535-00835800 launched on Node an-jenkins-deploy-platform-ansible-os-5690-2 2026-04-27 03:22:48,735 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/7339268a-4ead-4a74-ab1a-3a9417a482ba 2026-04-27 03:22:48,748 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/2600b23c-9853-4459-8bef-58052a27334d 2026-04-27 03:22:49,935 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=8.72 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=3.70 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1002ms\r\nrtt min/avg/max/mdev = 3.704/6.211/8.718/2.507 ms') 2026-04-27 03:22:49,935 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-TestQos-23217535-00835800 passed 2026-04-27 03:22:50,015 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2026-04-27 03:22:50,015 - DEBUG - Waiting to SSH to VM ctest-TestQos-23217535-00835800, IP 158.27.187.132, Port 22 2026-04-27 03:22:50,084 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2026-04-27 03:22:50,228 - DEBUG - VM ctest-TestQos-23217535-00835800 is NOT ready for SSH connections, VM status: ACTIVE 2026-04-27 03:22:55,229 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2026-04-27 03:22:55,229 - DEBUG - Waiting to SSH to VM ctest-TestQos-23217535-00835800, IP 158.27.187.132, Port 22 2026-04-27 03:22:55,295 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2026-04-27 03:22:55,404 - DEBUG - VM ctest-TestQos-23217535-00835800 is NOT ready for SSH connections, VM status: ACTIVE 2026-04-27 03:23:00,405 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2026-04-27 03:23:00,405 - DEBUG - Waiting to SSH to VM ctest-TestQos-23217535-00835800, IP 158.27.187.132, Port 22 2026-04-27 03:23:00,472 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2026-04-27 03:23:00,629 - DEBUG - VM ctest-TestQos-23217535-00835800 is NOT ready for SSH connections, VM status: ACTIVE 2026-04-27 03:23:05,630 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2026-04-27 03:23:05,630 - DEBUG - Waiting to SSH to VM ctest-TestQos-23217535-00835800, IP 158.27.187.132, Port 22 2026-04-27 03:23:05,696 - DEBUG - Error on ssh to ubuntu@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2026-04-27 03:23:05,850 - DEBUG - VM ctest-TestQos-23217535-00835800 is NOT ready for SSH connections, VM status: ACTIVE 2026-04-27 03:23:10,851 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2026-04-27 03:23:10,851 - DEBUG - Waiting to SSH to VM ctest-TestQos-23217535-00835800, IP 158.27.187.132, Port 22 2026-04-27 03:23:11,017 - DEBUG - VM ctest-TestQos-23217535-00835800 is ready for SSH connections 2026-04-27 03:23:11,017 - INFO - Waiting for VM ctest-TestQos-23217535-89858904 to be up.. 2026-04-27 03:23:11,133 - DEBUG - VM is in ACTIVE state now 2026-04-27 03:23:11,133 - INFO - VM name : ctest-TestQos-23217535-89858904 2026-04-27 03:23:11,255 - DEBUG - VM ctest-TestQos-23217535-89858904 ID is 514925d7-0e61-4131-91fa-53763c710211 2026-04-27 03:23:11,255 - DEBUG - VM ctest-TestQos-23217535-89858904 launched on Node an-jenkins-deploy-platform-ansible-os-5690-2 2026-04-27 03:23:11,369 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/514925d7-0e61-4131-91fa-53763c710211 2026-04-27 03:23:11,384 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/1e4b7348-f851-4059-a21c-adc6810772fa 2026-04-27 03:23:12,569 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n64 bytes from 169.254.0.4: icmp_seq=1 ttl=63 time=2.42 ms\r\n64 bytes from 169.254.0.4: icmp_seq=2 ttl=63 time=3.56 ms\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 2.417/2.989/3.562/0.572 ms') 2026-04-27 03:23:12,569 - INFO - Ping to Metadata IP 169.254.0.4 of VM ctest-TestQos-23217535-89858904 passed 2026-04-27 03:23:12,648 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2026-04-27 03:23:12,648 - DEBUG - Waiting to SSH to VM ctest-TestQos-23217535-89858904, IP 106.83.38.195, Port 22 2026-04-27 03:23:12,716 - DEBUG - Error on ssh to ubuntu@169.254.0.4:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.4/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.4/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.4/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2026-04-27 03:23:12,822 - DEBUG - VM ctest-TestQos-23217535-89858904 is NOT ready for SSH connections, VM status: ACTIVE 2026-04-27 03:23:17,822 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2026-04-27 03:23:17,823 - DEBUG - Waiting to SSH to VM ctest-TestQos-23217535-89858904, IP 106.83.38.195, Port 22 2026-04-27 03:23:17,987 - DEBUG - VM ctest-TestQos-23217535-89858904 is ready for SSH connections 2026-04-27 03:23:17,992 - INFO - ================================================================================ 2026-04-27 03:23:17,992 - INFO - STARTING TEST : test_qos_remark_dscp_on_vmi 2026-04-27 03:23:17,992 - INFO - TEST DESCRIPTION : Create a qos config for remarking DSCP 1 to 10 Have VMs A, B Apply the qos config to VM A Validate that packets from A to B have DSCP marked correctly 2026-04-27 03:23:19,327 - DEBUG - Nothing to compare xmpp stats {'10.0.0.33': {'10.20.0.193': '0', '10.20.0.18': '0'}, '10.0.0.65': {'10.20.0.25': '0', '10.20.0.18': '0'}} with 2026-04-27 03:23:19,328 - INFO - Initial checks done. Running the testcase now 2026-04-27 03:23:19,328 - INFO - 2026-04-27 03:23:19,336 - DEBUG - FC Dict is {'fc_id': 0, 'dscp': 10, 'dot1p': 1, 'exp': 1, 'connections': } 2026-04-27 03:23:19,635 - INFO - Created FC ['default-global-system-config', 'default-global-qos-config', 'ctest-fc-71322067'], UUID 67bb1ae7-78eb-4d00-8019-fe2381588c20 2026-04-27 03:23:20,039 - INFO - Created QosConfig ['default-domain', 'ctest-TestQos-23217535', 'ctest-qos_config-73613948'], UUID: 6dccb06c-6eac-4632-8b2d-6bc03e5dc6bf 2026-04-27 03:23:20,091 - INFO - Applying qos-config on VM 76c3a6b2-a88f-42e7-a3ae-8221d4c3fb4e 2026-04-27 03:23:20,425 - DEBUG - VM is in ACTIVE state now 2026-04-27 03:23:20,744 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/0a2e6917-9e4c-4f17-9e63-f3c41c65f6fe 2026-04-27 03:23:20,757 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/76c3a6b2-a88f-42e7-a3ae-8221d4c3fb4e 2026-04-27 03:23:20,772 - DEBUG - Requesting: http://10.0.0.23:8082/instance-ip/8b83560d-7427-4fde-abe8-ce1aa6ba1529 2026-04-27 03:23:20,788 - DEBUG - Requesting: http://10.0.0.23:8082/instance-ip/d1a21092-67c7-4f75-a43b-f0bee80ee2c2 2026-04-27 03:23:21,060 - DEBUG - VM is in ACTIVE state now 2026-04-27 03:23:21,438 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/7339268a-4ead-4a74-ab1a-3a9417a482ba 2026-04-27 03:23:21,454 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/2600b23c-9853-4459-8bef-58052a27334d 2026-04-27 03:23:21,475 - DEBUG - Requesting: http://10.0.0.23:8082/instance-ip/3fed8f2a-e1be-4217-bb5f-dc4037ce0350 2026-04-27 03:23:21,492 - DEBUG - Requesting: http://10.0.0.23:8082/instance-ip/78422f4a-c0f4-4415-b1d1-7ef714d7e2d8 2026-04-27 03:23:21,544 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2026-04-27 03:23:21,544 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2026-04-27 03:23:21,544 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2026-04-27 03:23:21,544 - INFO - Starting hping3 on ctest-TestQos-23217535-05775201, args: --destport 20000 --baseport 10000 --count 20000 --interval 1 --udp --tos 4 --keep --numeric 2026-04-27 03:23:21,544 - DEBUG - Hping3 cmd : hping3 --destport 20000 --baseport 10000 --count 20000 --interval 1 --udp --tos 4 --keep --numeric 158.27.187.132 1>/tmp/hping_ctest-random-79733419.log 2>/tmp/hping_ctest-random-79733419.result 2026-04-27 03:23:21,547 - DEBUG - Running remote_cmd, Cmd : hping3 --destport 20000 --baseport 10000 --count 20000 --interval 1 --udp --tos 4 --keep --numeric 158.27.187.132 1>/tmp/hping_ctest-random-79733419.log 2>/tmp/hping_ctest-random-79733419.result, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.33, gateway password: c0ntrail123 2026-04-27 03:23:21,547 - DEBUG - nohup hping3 --destport 20000 --baseport 10000 --count 20000 --interval 1 --udp --tos 4 --keep --numeric 158.27.187.132 1>/tmp/hping_ctest-random-79733419.log 2>/tmp/hping_ctest-random-79733419.result & echo $! > /tmp/hping_ctest-random-79733419.pid 2026-04-27 03:24:00,534 - DEBUG - None 2026-04-27 03:24:05,749 - DEBUG - Forward flow: {'index': '30484', 'rflow': '128796', 'sip': '158.27.187.131', 'sport': '10000', 'dip': '158.27.187.132', 'dport': '20000', 'proto': '17', 'vrf_id': '2', 'action': 'FORWARD', 'flags': ' ACTIVE | RFLOW_VALID ', 'd_vrf_id': '0', 'bytes': '210', 'pkts': '5', 'insight': '0', 'nhid': '31', 'underlay_udp_sport': '55644', 'ttl': '0', 'qos_id': '0', 'gen_id': '1', 'tcp_seq': '0', 'oflow_bytes': '0', 'oflow_packets': '0', 'underlay_gw_index': '-1'} 2026-04-27 03:24:05,750 - DEBUG - Reverse flow: {'index': '128796', 'rflow': '30484', 'sip': '158.27.187.132', 'sport': '20000', 'dip': '158.27.187.131', 'dport': '10000', 'proto': '17', 'vrf_id': '2', 'action': 'FORWARD', 'flags': ' ACTIVE | RFLOW_VALID ', 'd_vrf_id': '0', 'bytes': '350', 'pkts': '5', 'insight': '0', 'nhid': '31', 'underlay_udp_sport': '55746', 'ttl': '0', 'qos_id': '0', 'gen_id': '1', 'tcp_seq': '0', 'oflow_bytes': '0', 'oflow_packets': '0', 'underlay_gw_index': '-1'} 2026-04-27 03:24:05,750 - DEBUG - The filter pattern is ['udp', 'and', 'src port 55644'] 2026-04-27 03:24:05,750 - DEBUG - The filter string is '(udp and src port 55644)' 2026-04-27 03:24:05,833 - DEBUG - Executing command: sudo tcpdump -nni ens6 -U -vvxx '(udp and src port 55644)' -w /tmp/ens6_ctest-random-18926124.pcap 2026-04-27 03:24:10,934 - DEBUG - Executing command: sudo kill $(ps -ef|grep tcpdump | grep pcap| awk '{print $2}') 2026-04-27 03:24:12,937 - DEBUG - Ensuring hping3 instance with result file /tmp/hping_ctest-random-79733419.result on ctest-TestQos-23217535-05775201 is stopped 2026-04-27 03:24:12,937 - DEBUG - Running remote_cmd, Cmd : cat /tmp/hping_ctest-random-79733419.pid | xargs kill , host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.33, gateway password: c0ntrail123 2026-04-27 03:24:12,938 - DEBUG - cat /tmp/hping_ctest-random-79733419.pid | xargs kill 2026-04-27 03:24:13,755 - DEBUG - None 2026-04-27 03:24:13,755 - DEBUG - Running remote_cmd, Cmd : cat /tmp/hping_ctest-random-79733419.result, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.33, gateway password: c0ntrail123 2026-04-27 03:24:13,755 - DEBUG - cat /tmp/hping_ctest-random-79733419.result 2026-04-27 03:24:14,078 - DEBUG - --- 158.27.187.132 hping statistic --- 14 packets transmitted, 14 packets received, 0% packet loss round-trip min/avg/max = 18.5/6510.9/13013.2 ms 2026-04-27 03:24:14,078 - DEBUG - Running remote_cmd, Cmd : cat /tmp/hping_ctest-random-79733419.log, host_string: ubuntu@169.254.0.3, password: ubuntugateway: ubuntu@10.0.0.33, gateway password: c0ntrail123 2026-04-27 03:24:14,078 - DEBUG - cat /tmp/hping_ctest-random-79733419.log 2026-04-27 03:24:14,420 - DEBUG - HPING 158.27.187.132 (eth0 158.27.187.132): udp mode set, 28 headers + 0 data bytes ICMP Port Unreachable from ip=158.27.187.132 status=0 port=10000 seq=0 ICMP Port Unreachable from ip=158.27.187.132 status=1 port=10000 seq=0 ICMP Port Unreachable from ip=158.27.187.132 status=1 port=10000 seq=0 ICMP Port Unreachable from ip=158.27.187.132 status=1 port=10000 seq=0 ICMP Port Unreachable from ip=158.27.187.132 status=1 port=10000 seq=0 ICMP Port Unreachable from ip=158.27.187.132 status=1 port=10000 seq=0 ICMP Port Unreachable from ip=158.27.187.132 status=1 port=10000 seq=0 ICMP Port Unreachable from ip=158.27.187.132 status=1 port=10000 seq=0 ICMP Port Unreachable from ip=158.27.187.132 status=1 port=10000 seq=0 ICMP Port Unreachable from ip=158.27.187.132 status=1 port=10000 seq=0 ICMP Port Unreachable from ip=158.27.187.132 status=1 port=10000 seq=0 ICMP Port Unreachable from ip=158.27.187.132 status=1 port=10000 seq=0 ICMP Port Unreachable from ip=158.27.187.132 status=1 port=10000 seq=0 ICMP Port Unreachable from ip=158.27.187.132 status=1 port=10000 seq=0 2026-04-27 03:24:14,421 - DEBUG - Hping3 stats: {'sent': '14', 'received': '14', 'loss': '0', 'rtt_min': '18.5', 'rtt_avg': '6510.9', 'rtt_max': '13013.2'} 2026-04-27 03:24:14,421 - DEBUG - Executing command: sudo tcpdump -nnr /tmp/ens6_ctest-random-18926124.pcap | wc -l 2026-04-27 03:24:14,437 - DEBUG - STDOUT: 16 2026-04-27 03:24:14,437 - DEBUG - STDERR: reading from file /tmp/ens6_ctest-random-18926124.pcap, link-type EN10MB (Ethernet), snapshot length 262144 2026-04-27 03:24:14,437 - DEBUG - 16 packets are found in tcpdump output file /tmp/ens6_ctest-random-18926124.pcap but expected 1, which is fine 2026-04-27 03:24:14,437 - INFO - 16 packets are found in tcpdump output as expected 2026-04-27 03:24:14,438 - DEBUG - Executing command: sudo kill $(ps -ef|grep tcpdump | grep pcap| awk '{print $2}') 2026-04-27 03:24:16,508 - DEBUG - ['/contrail-test/ens6_ctest-random-18926124.pcap'] 2026-04-27 03:24:16,508 - DEBUG - Verifying for packet number 0 2026-04-27 03:24:16,508 - DEBUG - Validated DSCP marking of 10 2026-04-27 03:24:16,508 - DEBUG - Verifying for packet number 1 2026-04-27 03:24:16,508 - DEBUG - Validated DSCP marking of 10 2026-04-27 03:24:16,508 - DEBUG - Verifying for packet number 2 2026-04-27 03:24:16,508 - DEBUG - Validated DSCP marking of 10 2026-04-27 03:24:16,508 - DEBUG - Verifying for packet number 3 2026-04-27 03:24:16,508 - DEBUG - Validated DSCP marking of 10 2026-04-27 03:24:16,509 - INFO - Packet QoS marking validation passed 2026-04-27 03:24:16,509 - DEBUG - Executing command: sudo tcpdump -nnr /tmp/ens6_ctest-random-18926124.pcap | wc -l 2026-04-27 03:24:16,523 - DEBUG - STDOUT: 16 2026-04-27 03:24:16,523 - DEBUG - STDERR: reading from file /tmp/ens6_ctest-random-18926124.pcap, link-type EN10MB (Ethernet), snapshot length 262144 2026-04-27 03:24:16,523 - DEBUG - 16 packets are found in tcpdump output file /tmp/ens6_ctest-random-18926124.pcap but expected 1, which is fine 2026-04-27 03:24:16,523 - INFO - 16 packets are found in tcpdump output as expected 2026-04-27 03:24:16,523 - DEBUG - Executing command: sudo kill $(ps -ef|grep tcpdump | grep pcap| awk '{print $2}') 2026-04-27 03:24:18,581 - DEBUG - ['/contrail-test/ens6_ctest-random-18926124.pcap'] 2026-04-27 03:24:18,582 - DEBUG - Verifying for packet number 0 2026-04-27 03:24:18,582 - DEBUG - Interface ens6 does not seem to be a tagged intf. Skipping dot1p check 2026-04-27 03:24:18,582 - DEBUG - Verifying for packet number 1 2026-04-27 03:24:18,582 - DEBUG - Interface ens6 does not seem to be a tagged intf. Skipping dot1p check 2026-04-27 03:24:18,582 - DEBUG - Verifying for packet number 2 2026-04-27 03:24:18,582 - DEBUG - Interface ens6 does not seem to be a tagged intf. Skipping dot1p check 2026-04-27 03:24:18,582 - DEBUG - Verifying for packet number 3 2026-04-27 03:24:18,582 - DEBUG - Interface ens6 does not seem to be a tagged intf. Skipping dot1p check 2026-04-27 03:24:18,582 - INFO - Packet QoS marking validation passed 2026-04-27 03:24:18,692 - INFO - Removing qos-config on VM 76c3a6b2-a88f-42e7-a3ae-8221d4c3fb4e 2026-04-27 03:24:18,775 - INFO - Deleting Qos config ['default-domain', 'ctest-TestQos-23217535', 'ctest-qos_config-73613948'], UUID: 6dccb06c-6eac-4632-8b2d-6bc03e5dc6bf 2026-04-27 03:24:18,887 - INFO - Deleting FC ['default-global-system-config', 'default-global-qos-config', 'ctest-fc-71322067'], UUID: 67bb1ae7-78eb-4d00-8019-fe2381588c20 2026-04-27 03:24:20,265 - DEBUG - No XMPP flaps were noticed during the test 2026-04-27 03:24:20,265 - INFO - END TEST : test_qos_remark_dscp_on_vmi : PASSED[0:01:03] 2026-04-27 03:24:20,265 - INFO - -------------------------------------------------------------------------------- 2026-04-27 03:24:20,268 - INFO - Deleting VM ctest-TestQos-23217535-89858904 2026-04-27 03:24:20,356 - INFO - Deleting VM ctest-TestQos-23217535-00835800 2026-04-27 03:24:20,445 - INFO - Deleting VM ctest-TestQos-23217535-05775201 2026-04-27 03:24:20,537 - INFO - Deleting VN ctest-vn-30316986 2026-04-27 03:24:20,599 - DEBUG - VN bfec5062-57ab-4529-8c61-2bb14f996dad still in use: Unable to complete operation on network bfec5062-57ab-4529-8c61-2bb14f996dad. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-d999e0b4-176c-4d63-bc31-0ffba4393a60'] 2026-04-27 03:24:20,600 - WARNING - Deleting VN ctest-vn-30316986 failed..Will retry 2026-04-27 03:24:22,935 - DEBUG - Response for deleting network () 2026-04-27 03:24:22,935 - INFO - Deleting VN ctest-vn-02737557 2026-04-27 03:24:23,249 - DEBUG - Response for deleting network () 2026-04-27 03:24:24,139 - INFO - Deleted project: ctest-TestQos-23217535, ID : 3ddb8241-a3b1-4197-88ce-80756743ebab