2025-03-27 13:44:59,706 - INFO - Domain Default found not creating 2025-03-27 13:44:59,859 - INFO - Project ctest-TestBasicVMVN-34967799 not found, creating it 2025-03-27 13:45:00,321 - INFO - Created Project:ctest-TestBasicVMVN-34967799, ID : fcf045f7-4e61-4b41-9ba6-3302354d4797 2025-03-27 13:45:01,795 - INFO - ================================================================================ 2025-03-27 13:45:01,795 - INFO - STARTING TEST : test_generic_link_local_service 2025-03-27 13:45:01,795 - INFO - TEST DESCRIPTION : Description: Test to validate generic linklocal service - running nova list from vm. 1.Create generic link local service to be able to wget to jenkins 2.Create a vm 3.Try wget to jenkins - passes if successful else fails Maintainer: sandipd@juniper.net 2025-03-27 13:45:02,049 - DEBUG - Nothing to compare xmpp stats {'10.0.0.28': {'10.20.0.11': '0'}} with 2025-03-27 13:45:02,049 - INFO - Initial checks done. Running the testcase now 2025-03-27 13:45:02,050 - INFO - 2025-03-27 13:45:02,617 - DEBUG - Response for create_network : {'network': {'id': '86826a67-3beb-404d-9911-9075af4a1ed6', 'name': 'ctest-vn2_metadata-65177067', 'tenant_id': 'fcf045f74e614b419ba63302354d4797', 'project_id': 'fcf045f74e614b419ba63302354d4797', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestBasicVMVN-34967799', 'ctest-vn2_metadata-65177067'], 'port_security_enabled': True, 'description': ''}} 2025-03-27 13:45:02,828 - DEBUG - Response for create_subnet : {'subnet': {'id': 'e1fc6f58-aab9-481c-98ef-aaef789b49f1', 'name': '', 'tenant_id': 'fcf045f74e614b419ba63302354d4797', 'network_id': '86826a67-3beb-404d-9911-9075af4a1ed6', 'ip_version': 4, 'cidr': '11.1.1.0/24', 'allocation_pools': [{'start': '11.1.1.2', 'end': '11.1.1.254'}], 'gateway_ip': '11.1.1.1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '11.1.1.2', 'tags': [], 'project_id': 'fcf045f74e614b419ba63302354d4797'}} 2025-03-27 13:45:02,847 - INFO - Created VN ctest-vn2_metadata-65177067 2025-03-27 13:45:02,899 - DEBUG - VN ctest-vn2_metadata-65177067 UUID is 86826a67-3beb-404d-9911-9075af4a1ed6 2025-03-27 13:45:03,095 - DEBUG - Services list from nova: [, , ] 2025-03-27 13:45:04,150 - INFO - VM ([]) created on node: (cn-jenkins-deploy-platform-ansible-os-2972-1), Zone: (nova:cn-jenkins-deploy-platform-ansible-os-2972-1) 2025-03-27 13:45:04,255 - DEBUG - Link local service introspect added 2025-03-27 13:45:04,296 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-03-27 13:45:09,385 - DEBUG - VM is in ACTIVE state now 2025-03-27 13:45:09,525 - INFO - Waiting for VM ctest-TestBasicVMVN-34967799-65323588 to be up.. 2025-03-27 13:45:09,525 - INFO - VM name : ctest-TestBasicVMVN-34967799-65323588 2025-03-27 13:45:09,600 - DEBUG - VM ctest-TestBasicVMVN-34967799-65323588 ID is b41cd8d1-02eb-4295-abd9-7a5cb825f828 2025-03-27 13:45:09,600 - DEBUG - VM ctest-TestBasicVMVN-34967799-65323588 launched on Node cn-jenkins-deploy-platform-ansible-os-2972-1 2025-03-27 13:45:09,676 - DEBUG - Requesting: http://10.0.0.28:8082/virtual-machine/b41cd8d1-02eb-4295-abd9-7a5cb825f828 2025-03-27 13:45:09,973 - DEBUG - Requesting: http://10.0.0.28:8082/virtual-machine/b41cd8d1-02eb-4295-abd9-7a5cb825f828 2025-03-27 13:45:10,008 - DEBUG - Requesting: http://10.0.0.28:8082/virtual-machine-interface/da404f08-30da-4fad-876a-1f8ec1800284 2025-03-27 13:45:13,290 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1030ms') 2025-03-27 13:45:13,291 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBasicVMVN-34967799-65323588 failed! 2025-03-27 13:45:13,351 - DEBUG - Gateway for vn default-domain:ctest-TestBasicVMVN-34967799:ctest-vn2_metadata-65177067 is 11.1.1.1 and allocation pool is NOT set 2025-03-27 13:45:15,414 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=3.34 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=0.569 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 0.569/1.956/3.344/1.387 ms') 2025-03-27 13:45:15,415 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBasicVMVN-34967799-65323588 passed 2025-03-27 13:45:15,571 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-03-27 13:45:15,571 - DEBUG - Waiting to SSH to VM ctest-TestBasicVMVN-34967799-65323588, IP 11.1.1.3, Port 22 2025-03-27 13:45:15,730 - DEBUG - VM ctest-TestBasicVMVN-34967799-65323588 is ready for SSH connections 2025-03-27 13:45:15,730 - INFO - Retry 0 2025-03-27 13:45:15,730 - DEBUG - Running remote_cmd, Cmd : wget http://169.254.1.2:8083 --spider && echo "Successful", host_string: cirros@169.254.0.3, password: cubswin:)gateway: ubuntu@10.0.0.28, gateway password: c0ntrail123 2025-03-27 13:45:15,731 - DEBUG - wget http://169.254.1.2:8083 --spider && echo "Successful" 2025-03-27 13:45:17,343 - DEBUG - Connecting to 169.254.1.2:8083 (169.254.1.2:8083) Successful 2025-03-27 13:45:17,343 - INFO - Generic metadata worked 2025-03-27 13:45:17,595 - DEBUG - Link local service introspect removed 2025-03-27 13:45:17,596 - INFO - Deleting VM ctest-TestBasicVMVN-34967799-65323588 2025-03-27 13:45:17,667 - INFO - Deleting VN ctest-vn2_metadata-65177067 2025-03-27 13:45:17,706 - DEBUG - VN 86826a67-3beb-404d-9911-9075af4a1ed6 still in use: Unable to complete operation on network 86826a67-3beb-404d-9911-9075af4a1ed6. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-d2ebe69d-cb9e-439f-b025-59d5b0456993'] 2025-03-27 13:45:17,706 - WARNING - Deleting VN ctest-vn2_metadata-65177067 failed..Will retry 2025-03-27 13:45:19,847 - DEBUG - Response for deleting network () 2025-03-27 13:45:20,105 - DEBUG - No XMPP flaps were noticed during the test 2025-03-27 13:45:20,105 - INFO - -------------------------------------------------------------------------------- 2025-03-27 13:45:20,108 - INFO - ================================================================================ 2025-03-27 13:45:20,108 - INFO - STARTING TEST : test_metadata_service 2025-03-27 13:45:20,108 - INFO - TEST DESCRIPTION : Description: Test to validate metadata service on VM creation. 1.Verify from global-vrouter-config if metadata configures or not - fails otherwise 2.Create a shell script which writes 'hello world ' in a file in /tmp and save the script on the nova api node 3.Create a vm with userdata pointing to that script - script should get executed during vm boot up 4.Go to the vm and verify if the file with 'hello world ' written saved in /tmp of the vm - fails otherwise Maintainer: sandipd@juniper.net 2025-03-27 13:45:20,393 - DEBUG - Nothing to compare xmpp stats {'10.0.0.28': {'10.20.0.11': '0'}} with 2025-03-27 13:45:20,393 - INFO - Initial checks done. Running the testcase now 2025-03-27 13:45:20,393 - INFO - 2025-03-27 13:45:20,393 - DEBUG - Requesting: http://10.0.0.28:8082/global-vrouter-configs 2025-03-27 13:45:20,400 - DEBUG - Requesting: http://10.0.0.28:8082/global-vrouter-config/bfab356d-8305-45bf-ac06-0ff1dcca768d 2025-03-27 13:45:20,587 - DEBUG - Response for create_network : {'network': {'id': '9194a283-b824-48ae-afa1-ac426ac63552', 'name': 'ctest-vn-25786443', 'tenant_id': 'fcf045f74e614b419ba63302354d4797', 'project_id': 'fcf045f74e614b419ba63302354d4797', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestBasicVMVN-34967799', 'ctest-vn-25786443'], 'port_security_enabled': True, 'description': ''}} 2025-03-27 13:45:20,743 - DEBUG - Response for create_subnet : {'subnet': {'id': 'b53295e6-fb6d-4187-a6e6-f7a55fdedc04', 'name': '', 'tenant_id': 'fcf045f74e614b419ba63302354d4797', 'network_id': '9194a283-b824-48ae-afa1-ac426ac63552', 'ip_version': 4, 'cidr': '108.40.155.0/26', 'allocation_pools': [{'start': '108.40.155.2', 'end': '108.40.155.62'}], 'gateway_ip': '108.40.155.1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '108.40.155.2', 'tags': [], 'project_id': 'fcf045f74e614b419ba63302354d4797'}} 2025-03-27 13:45:20,767 - INFO - Created VN ctest-vn-25786443 2025-03-27 13:45:20,819 - DEBUG - VN ctest-vn-25786443 UUID is 9194a283-b824-48ae-afa1-ac426ac63552 2025-03-27 13:45:21,994 - INFO - VM ([]) created on node: (cn-jenkins-deploy-platform-ansible-os-2972-1), Zone: (nova:cn-jenkins-deploy-platform-ansible-os-2972-1) 2025-03-27 13:45:21,994 - INFO - Waiting for VM ctest-TestBasicVMVN-34967799-36557467 to be up.. 2025-03-27 13:45:22,038 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-03-27 13:45:27,148 - DEBUG - VM is in ACTIVE state now 2025-03-27 13:45:27,148 - INFO - VM name : ctest-TestBasicVMVN-34967799-36557467 2025-03-27 13:45:27,225 - DEBUG - VM ctest-TestBasicVMVN-34967799-36557467 ID is 9a28dc90-fd72-498b-8cce-b728eca1028f 2025-03-27 13:45:27,225 - DEBUG - VM ctest-TestBasicVMVN-34967799-36557467 launched on Node cn-jenkins-deploy-platform-ansible-os-2972-1 2025-03-27 13:45:27,300 - DEBUG - Requesting: http://10.0.0.28:8082/virtual-machine/9a28dc90-fd72-498b-8cce-b728eca1028f 2025-03-27 13:45:27,309 - DEBUG - Requesting: http://10.0.0.28:8082/virtual-machine-interface/d0541330-11e8-4afc-9954-3d582e40f2ee 2025-03-27 13:45:30,577 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1023ms') 2025-03-27 13:45:30,577 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBasicVMVN-34967799-36557467 failed! 2025-03-27 13:45:30,631 - DEBUG - Gateway for vn default-domain:ctest-TestBasicVMVN-34967799:ctest-vn-25786443 is 108.40.155.1 and allocation pool is NOT set 2025-03-27 13:45:32,688 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=9.88 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=0.475 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 0.475/5.177/9.880/4.702 ms') 2025-03-27 13:45:32,688 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBasicVMVN-34967799-36557467 passed 2025-03-27 13:45:32,855 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-03-27 13:45:32,856 - DEBUG - Waiting to SSH to VM ctest-TestBasicVMVN-34967799-36557467, IP 108.40.155.3, Port 22 2025-03-27 13:45:33,011 - DEBUG - VM ctest-TestBasicVMVN-34967799-36557467 is ready for SSH connections 2025-03-27 13:45:33,011 - DEBUG - Running remote_cmd, Cmd : cat /tmp/output.txt, host_string: cirros@169.254.0.3, password: cubswin:)gateway: ubuntu@10.0.0.28, gateway password: c0ntrail123 2025-03-27 13:45:33,012 - DEBUG - cat /tmp/output.txt 2025-03-27 13:45:34,710 - DEBUG - Hello World. The time is now Thu, 27 Mar 2025 07:45:33 -0600! 2025-03-27 13:45:34,711 - DEBUG - ret : {'cat /tmp/output.txt': 'Hello World. The time is now Thu, 27 Mar 2025 07:45:33 -0600!'} 2025-03-27 13:45:34,711 - INFO - metadata_script.txt got executed in the vm 2025-03-27 13:45:34,711 - INFO - Deleting VM ctest-TestBasicVMVN-34967799-36557467 2025-03-27 13:45:34,781 - INFO - Deleting VN ctest-vn-25786443 2025-03-27 13:45:34,823 - DEBUG - VN 9194a283-b824-48ae-afa1-ac426ac63552 still in use: Unable to complete operation on network 9194a283-b824-48ae-afa1-ac426ac63552. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-66c97a45-ce29-43b4-85b5-798f6bc4bf8a'] 2025-03-27 13:45:34,823 - WARNING - Deleting VN ctest-vn-25786443 failed..Will retry 2025-03-27 13:45:36,930 - DEBUG - Response for deleting network () 2025-03-27 13:45:37,189 - DEBUG - No XMPP flaps were noticed during the test
2025-03-27 13:45:37,189 - INFO - END TEST : test_metadata_service : PASSED[0:00:17]
2025-03-27 13:45:37,189 - INFO - -------------------------------------------------------------------------------- 2025-03-27 13:45:37,192 - INFO - ================================================================================ 2025-03-27 13:45:37,192 - INFO - STARTING TEST : test_ping_within_vn_two_vms_two_different_subnets 2025-03-27 13:45:37,192 - INFO - TEST DESCRIPTION : Description: Validate Ping between 2 VMs in the same VN, 2 VMs in different VN subnets. Test steps: 1. Create 1 IPAM's. 2. Create 1 VN with 2 subnets and launch 2 VMs in them. 3. Ping between the VMs in the same VN should go thru fine. 4. Ping to the subnet broadcast and all-broadcast address. Pass criteria: VM in the same subnet will respond to both the pings, while the VM in a different VN should respond only to the all-broadcast address. Maintainer : ganeshahv@juniper.net 2025-03-27 13:45:37,477 - DEBUG - Nothing to compare xmpp stats {'10.0.0.28': {'10.20.0.11': '0'}} with 2025-03-27 13:45:37,477 - INFO - Initial checks done. Running the testcase now 2025-03-27 13:45:37,477 - INFO - 2025-03-27 13:45:37,971 - DEBUG - Response for create_network : {'network': {'id': 'ddfd8547-9f33-4d22-93ec-6799fc3edc77', 'name': 'ctest-vn-31236606', 'tenant_id': 'fcf045f74e614b419ba63302354d4797', 'project_id': 'fcf045f74e614b419ba63302354d4797', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestBasicVMVN-34967799', 'ctest-vn-31236606'], 'port_security_enabled': True, 'description': ''}} 2025-03-27 13:45:38,116 - DEBUG - Response for create_subnet : {'subnet': {'id': '01c65906-10c7-4af1-aa20-13a5e561e785', 'name': '', 'tenant_id': 'fcf045f74e614b419ba63302354d4797', 'network_id': 'ddfd8547-9f33-4d22-93ec-6799fc3edc77', 'ip_version': 4, 'cidr': '31.1.1.0/29', 'allocation_pools': [{'start': '31.1.1.2', 'end': '31.1.1.6'}], 'gateway_ip': '31.1.1.1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '31.1.1.2', 'tags': [], 'project_id': 'fcf045f74e614b419ba63302354d4797'}} 2025-03-27 13:45:38,305 - DEBUG - Response for create_subnet : {'subnet': {'id': 'b92488fb-ca47-4c46-bfe4-47211fb223f7', 'name': '', 'tenant_id': 'fcf045f74e614b419ba63302354d4797', 'network_id': 'ddfd8547-9f33-4d22-93ec-6799fc3edc77', 'ip_version': 4, 'cidr': '31.1.2.0/29', 'allocation_pools': [{'start': '31.1.2.2', 'end': '31.1.2.6'}], 'gateway_ip': '31.1.2.1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '31.1.2.2', 'tags': [], 'project_id': 'fcf045f74e614b419ba63302354d4797'}} 2025-03-27 13:45:38,326 - INFO - Created VN ctest-vn-31236606 2025-03-27 13:45:38,379 - DEBUG - VN ctest-vn-31236606 UUID is ddfd8547-9f33-4d22-93ec-6799fc3edc77 2025-03-27 13:45:38,784 - DEBUG - Response for create_port : {'port': {'name': 'fd997317-14b6-4727-b54a-774ec5dd6ca8', 'id': 'fd997317-14b6-4727-b54a-774ec5dd6ca8', 'tenant_id': 'fcf045f74e614b419ba63302354d4797', 'network_id': 'ddfd8547-9f33-4d22-93ec-6799fc3edc77', 'mac_address': '02:fd:99:73:17:14', 'binding:vnic_type': 'normal', 'binding:vif_details': {'port_filter': True}, 'binding:vif_type': 'unbound', 'binding:host_id': None, 'allowed_address_pairs': [], 'fixed_ips': [{'ip_address': '31.1.1.4', 'subnet_id': '01c65906-10c7-4af1-aa20-13a5e561e785'}], 'security_groups': ['0c485450-cff4-45e8-870a-4227348c82a3'], 'admin_state_up': True, 'device_id': '', 'device_owner': '', 'status': 'DOWN', 'port_security_enabled': True, 'description': '', 'project_id': 'fcf045f74e614b419ba63302354d4797'}} 2025-03-27 13:45:39,009 - DEBUG - Response for create_port : {'port': {'name': 'd767518b-48a3-449a-8e1d-8b6e663c0f0d', 'id': 'd767518b-48a3-449a-8e1d-8b6e663c0f0d', 'tenant_id': 'fcf045f74e614b419ba63302354d4797', 'network_id': 'ddfd8547-9f33-4d22-93ec-6799fc3edc77', 'mac_address': '02:d7:67:51:8b:48', 'binding:vnic_type': 'normal', 'binding:vif_details': {'port_filter': True}, 'binding:vif_type': 'unbound', 'binding:host_id': None, 'allowed_address_pairs': [], 'fixed_ips': [{'ip_address': '31.1.2.4', 'subnet_id': 'b92488fb-ca47-4c46-bfe4-47211fb223f7'}], 'security_groups': ['0c485450-cff4-45e8-870a-4227348c82a3'], 'admin_state_up': True, 'device_id': '', 'device_owner': '', 'status': 'DOWN', 'port_security_enabled': True, 'description': '', 'project_id': 'fcf045f74e614b419ba63302354d4797'}} 2025-03-27 13:45:39,473 - INFO - VM ([]) created on node: (cn-jenkins-deploy-platform-ansible-os-2972-1), Zone: (nova:cn-jenkins-deploy-platform-ansible-os-2972-1) 2025-03-27 13:45:39,888 - INFO - VM ([]) created on node: (cn-jenkins-deploy-platform-ansible-os-2972-1), Zone: (nova:cn-jenkins-deploy-platform-ansible-os-2972-1) 2025-03-27 13:45:41,143 - INFO - VM ([]) created on node: (cn-jenkins-deploy-platform-ansible-os-2972-1), Zone: (nova:cn-jenkins-deploy-platform-ansible-os-2972-1) 2025-03-27 13:45:41,143 - DEBUG - Requesting: http://10.0.0.28:8082/domains 2025-03-27 13:45:41,150 - DEBUG - Requesting: http://10.0.0.28:8082/domain/22fdbb07-328f-4bc4-b2d3-9d039227212e 2025-03-27 13:45:41,168 - DEBUG - Requesting: http://10.0.0.28:8082/project/fcf045f7-4e61-4b41-9ba6-3302354d4797 2025-03-27 13:45:41,254 - DEBUG - Requesting: http://10.0.0.28:8082/network-ipam/10bff406-f768-4b96-bc82-c32753c81146 2025-03-27 13:45:41,264 - INFO - Verifications in API Server for IPAM: ctest-ipam-63395955 passed 2025-03-27 13:45:41,271 - DEBUG - Control-node 10.0.0.28 : IPAM object is : {'node_name': 'network-ipam:default-domain:ctest-TestBasicVMVN-34967799:ctest-ipam-63395955', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestBasicVMVN-34967799', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '1206951535916370838', 'uuid-lslong': '13583633999779598662'}, 'enable': 'true', 'created': '2025-03-27T13:45:37', 'last-modified': '2025-03-27T13:45:37', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:03.563301'} 2025-03-27 13:45:41,271 - INFO - Verifications in Control node for IPAM: ctest-ipam-63395955 passed 2025-03-27 13:45:41,272 - DEBUG - Requesting: http://10.0.0.28:8082/domains 2025-03-27 13:45:41,279 - DEBUG - Requesting: http://10.0.0.28:8082/domain/22fdbb07-328f-4bc4-b2d3-9d039227212e 2025-03-27 13:45:41,291 - DEBUG - Requesting: http://10.0.0.28:8082/project/fcf045f7-4e61-4b41-9ba6-3302354d4797 2025-03-27 13:45:41,372 - DEBUG - Requesting: http://10.0.0.28:8082/virtual-network/ddfd8547-9f33-4d22-93ec-6799fc3edc77 2025-03-27 13:45:41,385 - DEBUG - Requesting: http://10.0.0.28:8082/virtual-network/ddfd8547-9f33-4d22-93ec-6799fc3edc77 2025-03-27 13:45:41,396 - DEBUG - Requesting: http://10.0.0.28:8082/routing-instance/95527ff9-809a-41fe-9be8-1c88ed21098b 2025-03-27 13:45:41,410 - DEBUG - Requesting: http://10.0.0.28:8082/routing-instance/95527ff9-809a-41fe-9be8-1c88ed21098b 2025-03-27 13:45:41,423 - DEBUG - Requesting: http://10.0.0.28:8082/route-target/5fac656d-3a1f-4e7c-a239-72b8a4818ef8 2025-03-27 13:45:41,430 - DEBUG - Route Targets: ['target:64512:8000004'] 2025-03-27 13:45:41,430 - DEBUG - Requesting: http://10.0.0.28:8082/virtual-network/ddfd8547-9f33-4d22-93ec-6799fc3edc77 2025-03-27 13:45:41,444 - DEBUG - Requesting: http://10.0.0.28:8082/routing-instance/95527ff9-809a-41fe-9be8-1c88ed21098b 2025-03-27 13:45:41,462 - INFO - Verified VN network id 8 for VN ddfd8547-9f33-4d22-93ec-6799fc3edc77 2025-03-27 13:45:41,462 - INFO - Verifications in API Server for VN ctest-vn-31236606 passed 2025-03-27 13:45:41,462 - DEBUG - Requesting: http://10.0.0.28:8082/virtual-network/ddfd8547-9f33-4d22-93ec-6799fc3edc77 2025-03-27 13:45:41,474 - DEBUG - Requesting: http://10.0.0.28:8082/routing-instance/95527ff9-809a-41fe-9be8-1c88ed21098b 2025-03-27 13:45:41,482 - DEBUG - Requesting: http://10.0.0.28:8082/routing-instance/95527ff9-809a-41fe-9be8-1c88ed21098b 2025-03-27 13:45:41,490 - DEBUG - Requesting: http://10.0.0.28:8082/route-target/5fac656d-3a1f-4e7c-a239-72b8a4818ef8 2025-03-27 13:45:41,505 - DEBUG - Control-node 10.0.0.28 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestBasicVMVN-34967799:ctest-vn-31236606', 'interests': '0', 'advertised': '0', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestBasicVMVN-34967799', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '15996087994149981474', 'uuid-lslong': '10659008329138756727'}, 'enable': 'true', 'created': '2025-03-27T13:45:37', 'last-modified': '2025-03-27T13:45:38', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:03.229387'} 2025-03-27 13:45:41,513 - DEBUG - Route Targets: ['target:64512:8000004'] 2025-03-27 13:45:41,513 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn-31236606 passed 2025-03-27 13:45:41,514 - DEBUG - ====Verifying policy data for ctest-vn-31236606 in API_Server ====== 2025-03-27 13:45:41,514 - DEBUG - Requesting: http://10.0.0.28:8082/domains 2025-03-27 13:45:41,521 - DEBUG - Requesting: http://10.0.0.28:8082/domain/22fdbb07-328f-4bc4-b2d3-9d039227212e 2025-03-27 13:45:41,532 - DEBUG - Requesting: http://10.0.0.28:8082/project/fcf045f7-4e61-4b41-9ba6-3302354d4797 2025-03-27 13:45:41,624 - DEBUG - Requesting: http://10.0.0.28:8082/virtual-network/ddfd8547-9f33-4d22-93ec-6799fc3edc77 2025-03-27 13:45:41,636 - DEBUG - =>VN ctest-vn-31236606 has no policy to be verified 2025-03-27 13:45:41,636 - DEBUG - Verifying the vn in opserver 2025-03-27 13:45:41,637 - DEBUG - Verifying the default-domain:ctest-TestBasicVMVN-34967799:ctest-vn-31236606 virtual network link through opserver 10.0.0.28 2025-03-27 13:45:41,637 - DEBUG - Requesting: http://10.0.0.28:8081/analytics/uves/virtual-networks 2025-03-27 13:45:41,960 - DEBUG - Requesting: http://10.0.0.28:8081/analytics/uves/virtual-networks 2025-03-27 13:45:42,066 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestBasicVMVN-34967799:ctest-vn-31236606', 'href': 'http://10.0.0.28:8081/analytics/uves/virtual-network/default-domain:ctest-TestBasicVMVN-34967799:ctest-vn-31236606?flat'} 2025-03-27 13:45:42,066 - INFO - Validated that VN default-domain:ctest-TestBasicVMVN-34967799:ctest-vn-31236606 is found in opserver 2025-03-27 13:45:42,073 - DEBUG - VRF ids for VN ctest-vn-31236606: {'10.0.0.28': '2'} 2025-03-27 13:45:42,074 - INFO - Waiting for VM ctest-TestBasicVMVN-34967799-54525142 to be up.. 2025-03-27 13:45:42,173 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-03-27 13:45:47,257 - DEBUG - VM is in ACTIVE state now 2025-03-27 13:45:47,257 - INFO - VM name : ctest-TestBasicVMVN-34967799-54525142 2025-03-27 13:45:47,345 - DEBUG - VM ctest-TestBasicVMVN-34967799-54525142 ID is e568defa-42e4-4c5e-8c7a-bd2ab78438d1 2025-03-27 13:45:47,345 - DEBUG - VM ctest-TestBasicVMVN-34967799-54525142 launched on Node cn-jenkins-deploy-platform-ansible-os-2972-1 2025-03-27 13:45:47,441 - DEBUG - Requesting: http://10.0.0.28:8082/virtual-machine/e568defa-42e4-4c5e-8c7a-bd2ab78438d1 2025-03-27 13:45:47,455 - DEBUG - Requesting: http://10.0.0.28:8082/virtual-machine-interface/fd997317-14b6-4727-b54a-774ec5dd6ca8 2025-03-27 13:45:50,706 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1003ms') 2025-03-27 13:45:50,706 - DEBUG - Ping to Metadata IP 169.254.0.4 of VM ctest-TestBasicVMVN-34967799-54525142 failed! 2025-03-27 13:45:50,763 - DEBUG - Gateway for vn default-domain:ctest-TestBasicVMVN-34967799:ctest-vn-31236606 is 31.1.1.1 and allocation pool is NOT set 2025-03-27 13:45:50,763 - DEBUG - Gateway for vn default-domain:ctest-TestBasicVMVN-34967799:ctest-vn-31236606 is 31.1.2.1 and allocation pool is NOT set 2025-03-27 13:45:52,831 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n64 bytes from 169.254.0.4: icmp_seq=1 ttl=63 time=5.39 ms\r\n64 bytes from 169.254.0.4: icmp_seq=2 ttl=63 time=0.833 ms\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 0.833/3.110/5.388/2.277 ms') 2025-03-27 13:45:52,831 - INFO - Ping to Metadata IP 169.254.0.4 of VM ctest-TestBasicVMVN-34967799-54525142 passed 2025-03-27 13:45:52,983 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-03-27 13:45:52,983 - DEBUG - Waiting to SSH to VM ctest-TestBasicVMVN-34967799-54525142, IP 31.1.1.4, Port 22 2025-03-27 13:45:53,147 - DEBUG - VM ctest-TestBasicVMVN-34967799-54525142 is ready for SSH connections 2025-03-27 13:45:53,147 - INFO - Waiting for VM ctest-TestBasicVMVN-34967799-05539787 to be up.. 2025-03-27 13:45:53,230 - DEBUG - VM is in ACTIVE state now 2025-03-27 13:45:53,230 - INFO - VM name : ctest-TestBasicVMVN-34967799-05539787 2025-03-27 13:45:53,326 - DEBUG - VM ctest-TestBasicVMVN-34967799-05539787 ID is 05c9a28c-dcf9-454d-967b-5b74f2d608b8 2025-03-27 13:45:53,326 - DEBUG - VM ctest-TestBasicVMVN-34967799-05539787 launched on Node cn-jenkins-deploy-platform-ansible-os-2972-1 2025-03-27 13:45:53,428 - DEBUG - Requesting: http://10.0.0.28:8082/virtual-machine/05c9a28c-dcf9-454d-967b-5b74f2d608b8 2025-03-27 13:45:53,437 - DEBUG - Requesting: http://10.0.0.28:8082/virtual-machine-interface/d767518b-48a3-449a-8e1d-8b6e663c0f0d 2025-03-27 13:45:54,690 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=6.16 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=1.14 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 1.137/3.646/6.155/2.509 ms') 2025-03-27 13:45:54,690 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-TestBasicVMVN-34967799-05539787 passed 2025-03-27 13:45:54,847 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-03-27 13:45:54,847 - DEBUG - Waiting to SSH to VM ctest-TestBasicVMVN-34967799-05539787, IP 31.1.2.4, Port 22 2025-03-27 13:45:55,002 - DEBUG - VM ctest-TestBasicVMVN-34967799-05539787 is ready for SSH connections 2025-03-27 13:45:55,002 - INFO - Waiting for VM ctest-TestBasicVMVN-34967799-53006973 to be up.. 2025-03-27 13:45:55,084 - DEBUG - VM is in ACTIVE state now 2025-03-27 13:45:55,085 - INFO - VM name : ctest-TestBasicVMVN-34967799-53006973 2025-03-27 13:45:55,164 - DEBUG - VM ctest-TestBasicVMVN-34967799-53006973 ID is 7f9f8e39-ea72-441b-95e2-d509fd36b9f8 2025-03-27 13:45:55,165 - DEBUG - VM ctest-TestBasicVMVN-34967799-53006973 launched on Node cn-jenkins-deploy-platform-ansible-os-2972-1 2025-03-27 13:45:55,249 - DEBUG - Requesting: http://10.0.0.28:8082/virtual-machine/7f9f8e39-ea72-441b-95e2-d509fd36b9f8 2025-03-27 13:45:55,259 - DEBUG - Requesting: http://10.0.0.28:8082/virtual-machine-interface/d39492f2-8b56-4b8d-ac73-f7678ef44596 2025-03-27 13:45:56,501 - DEBUG - (True, 'PING 169.254.0.5 (169.254.0.5) 56(84) bytes of data.\r\n64 bytes from 169.254.0.5: icmp_seq=1 ttl=63 time=1.99 ms\r\n64 bytes from 169.254.0.5: icmp_seq=2 ttl=63 time=0.343 ms\r\n\r\n--- 169.254.0.5 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1000ms\r\nrtt min/avg/max/mdev = 0.343/1.165/1.987/0.822 ms') 2025-03-27 13:45:56,501 - INFO - Ping to Metadata IP 169.254.0.5 of VM ctest-TestBasicVMVN-34967799-53006973 passed 2025-03-27 13:45:56,651 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-03-27 13:45:56,651 - DEBUG - Waiting to SSH to VM ctest-TestBasicVMVN-34967799-53006973, IP 31.1.1.3, Port 22 2025-03-27 13:45:56,811 - DEBUG - VM ctest-TestBasicVMVN-34967799-53006973 is ready for SSH connections 2025-03-27 13:45:56,811 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-03-27 13:45:56,811 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 31.1.2.4, host_string: cirros@169.254.0.4, password: cubswin:)gateway: ubuntu@10.0.0.28, gateway password: c0ntrail123 2025-03-27 13:45:56,811 - DEBUG - ping -s 56 -c 3 -W 1 31.1.2.4 2025-03-27 13:46:00,799 - DEBUG - PING 31.1.2.4 (31.1.2.4): 56 data bytes 64 bytes from 31.1.2.4: seq=0 ttl=63 time=4.222 ms 64 bytes from 31.1.2.4: seq=1 ttl=63 time=0.737 ms 64 bytes from 31.1.2.4: seq=2 ttl=63 time=0.826 ms --- 31.1.2.4 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.737/1.928/4.222 ms 2025-03-27 13:46:00,799 - INFO - Ping to IP 31.1.2.4 from VM ctest-TestBasicVMVN-34967799-54525142 passed 2025-03-27 13:46:00,799 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-03-27 13:46:00,799 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 31.1.1.4, host_string: cirros@169.254.0.3, password: cubswin:)gateway: ubuntu@10.0.0.28, gateway password: c0ntrail123 2025-03-27 13:46:00,799 - DEBUG - ping -s 56 -c 3 -W 1 31.1.1.4 2025-03-27 13:46:04,506 - DEBUG - PING 31.1.1.4 (31.1.1.4): 56 data bytes 64 bytes from 31.1.1.4: seq=0 ttl=63 time=2.968 ms 64 bytes from 31.1.1.4: seq=1 ttl=63 time=0.988 ms 64 bytes from 31.1.1.4: seq=2 ttl=63 time=1.111 ms --- 31.1.1.4 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.988/1.689/2.968 ms 2025-03-27 13:46:04,506 - INFO - Ping to IP 31.1.1.4 from VM ctest-TestBasicVMVN-34967799-05539787 passed 2025-03-27 13:46:04,506 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-03-27 13:46:04,506 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-03-27 13:46:04,506 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-03-27 13:46:04,506 - DEBUG - Running remote_cmd, Cmd : echo 0 > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts, host_string: cirros@169.254.0.4, password: cubswin:)gateway: ubuntu@10.0.0.28, gateway password: c0ntrail123 2025-03-27 13:46:04,506 - DEBUG - echo 0 > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts 2025-03-27 13:46:04,713 - DEBUG - None 2025-03-27 13:46:04,713 - DEBUG - Running remote_cmd, Cmd : echo 0 > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts, host_string: cirros@169.254.0.3, password: cubswin:)gateway: ubuntu@10.0.0.28, gateway password: c0ntrail123 2025-03-27 13:46:04,713 - DEBUG - echo 0 > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts 2025-03-27 13:46:04,909 - DEBUG - None 2025-03-27 13:46:04,909 - DEBUG - Running remote_cmd, Cmd : echo 0 > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts, host_string: cirros@169.254.0.5, password: cubswin:)gateway: ubuntu@10.0.0.28, gateway password: c0ntrail123 2025-03-27 13:46:04,909 - DEBUG - echo 0 > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts 2025-03-27 13:46:05,561 - DEBUG - None 2025-03-27 13:46:05,562 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 31.1.1.7, host_string: cirros@169.254.0.4, password: cubswin:)gateway: ubuntu@10.0.0.28, gateway password: c0ntrail123 2025-03-27 13:46:05,562 - DEBUG - ping -s 56 -c 3 -W 1 31.1.1.7 2025-03-27 13:46:07,790 - DEBUG - PING 31.1.1.7 (31.1.1.7): 56 data bytes 64 bytes from 31.1.1.4: seq=0 ttl=64 time=2.343 ms 64 bytes from 31.1.1.3: seq=0 ttl=64 time=5.599 ms (DUP!) 64 bytes from 31.1.1.4: seq=1 ttl=64 time=0.411 ms 64 bytes from 31.1.1.3: seq=1 ttl=64 time=2.327 ms (DUP!) 64 bytes from 31.1.1.4: seq=2 ttl=64 time=0.336 ms --- 31.1.1.7 ping statistics --- 3 packets transmitted, 3 packets received, 2 duplicates, 0% packet loss round-trip min/avg/max = 0.336/2.203/5.599 ms 2025-03-27 13:46:07,790 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 224.0.0.1, host_string: cirros@169.254.0.4, password: cubswin:)gateway: ubuntu@10.0.0.28, gateway password: c0ntrail123 2025-03-27 13:46:07,791 - DEBUG - ping -s 56 -c 3 -W 1 224.0.0.1 2025-03-27 13:46:10,054 - DEBUG - PING 224.0.0.1 (224.0.0.1): 56 data bytes 64 bytes from 31.1.1.4: seq=0 ttl=64 time=1.013 ms 64 bytes from 31.1.1.3: seq=0 ttl=64 time=2.965 ms (DUP!) 64 bytes from 31.1.2.4: seq=0 ttl=63 time=4.597 ms (DUP!) 64 bytes from 31.1.1.4: seq=1 ttl=64 time=0.428 ms 64 bytes from 31.1.1.3: seq=1 ttl=64 time=1.674 ms (DUP!) 64 bytes from 31.1.2.4: seq=1 ttl=63 time=2.626 ms (DUP!) 64 bytes from 31.1.1.4: seq=2 ttl=64 time=0.529 ms --- 224.0.0.1 ping statistics --- 3 packets transmitted, 3 packets received, 4 duplicates, 0% packet loss round-trip min/avg/max = 0.428/1.976/4.597 ms 2025-03-27 13:46:10,054 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 255.255.255.255, host_string: cirros@169.254.0.4, password: cubswin:)gateway: ubuntu@10.0.0.28, gateway password: c0ntrail123 2025-03-27 13:46:10,054 - DEBUG - ping -s 56 -c 3 -W 1 255.255.255.255 2025-03-27 13:46:12,311 - DEBUG - PING 255.255.255.255 (255.255.255.255): 56 data bytes 64 bytes from 31.1.1.4: seq=0 ttl=64 time=0.993 ms 64 bytes from 31.1.2.4: seq=0 ttl=63 time=4.940 ms (DUP!) 64 bytes from 31.1.1.3: seq=0 ttl=64 time=6.385 ms (DUP!) 64 bytes from 31.1.1.4: seq=1 ttl=64 time=0.430 ms 64 bytes from 31.1.1.3: seq=1 ttl=64 time=2.483 ms (DUP!) 64 bytes from 31.1.2.4: seq=1 ttl=63 time=3.575 ms (DUP!) 64 bytes from 31.1.1.4: seq=2 ttl=64 time=0.376 ms --- 255.255.255.255 ping statistics --- 3 packets transmitted, 3 packets received, 4 duplicates, 0% packet loss round-trip min/avg/max = 0.376/2.740/6.385 ms 2025-03-27 13:46:12,311 - INFO - Deleting VM ctest-TestBasicVMVN-34967799-53006973 2025-03-27 13:46:12,392 - INFO - Detaching port d767518b-48a3-449a-8e1d-8b6e663c0f0d from VM ctest-TestBasicVMVN-34967799-05539787 2025-03-27 13:46:12,447 - INFO - Deleting VM ctest-TestBasicVMVN-34967799-05539787 2025-03-27 13:46:12,530 - INFO - Detaching port fd997317-14b6-4727-b54a-774ec5dd6ca8 from VM ctest-TestBasicVMVN-34967799-54525142 2025-03-27 13:46:12,566 - INFO - Deleting VM ctest-TestBasicVMVN-34967799-54525142 2025-03-27 13:46:12,642 - INFO - Deleting VN ctest-vn-31236606 2025-03-27 13:46:12,835 - DEBUG - Response for delete_port : () 2025-03-27 13:46:13,086 - DEBUG - Response for delete_port : () 2025-03-27 13:46:13,134 - DEBUG - VN ddfd8547-9f33-4d22-93ec-6799fc3edc77 still in use: Unable to complete operation on network ddfd8547-9f33-4d22-93ec-6799fc3edc77. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-373b3c7d-9a87-44b9-aa4f-2a0ae45b5708'] 2025-03-27 13:46:13,134 - WARNING - Deleting VN ctest-vn-31236606 failed..Will retry 2025-03-27 13:46:15,305 - DEBUG - Response for deleting network () 2025-03-27 13:46:15,305 - DEBUG - Requesting: http://10.0.0.28:8082/routing-instance/95527ff9-809a-41fe-9be8-1c88ed21098b 2025-03-27 13:46:15,312 - DEBUG - Response Code: 404 2025-03-27 13:46:15,312 - DEBUG - Requesting: http://10.0.0.28:8082/domains 2025-03-27 13:46:15,320 - DEBUG - Requesting: http://10.0.0.28:8082/domain/22fdbb07-328f-4bc4-b2d3-9d039227212e 2025-03-27 13:46:15,377 - DEBUG - Requesting: http://10.0.0.28:8082/project/fcf045f7-4e61-4b41-9ba6-3302354d4797 2025-03-27 13:46:15,455 - INFO - Validated that VN ctest-vn-31236606 is not found in API Server 2025-03-27 13:46:15,470 - DEBUG - VN ctest-vn-31236606 is not present in Agent 10.0.0.28 2025-03-27 13:46:15,470 - INFO - Validated that VN ctest-vn-31236606 is not in any agent 2025-03-27 13:46:15,476 - DEBUG - VRF 2 is not seen in agent 10.0.0.28 2025-03-27 13:46:15,481 - DEBUG - Vrouter 10.0.0.28 does not have vrf 2 for VN ctest-vn-31236606 2025-03-27 13:46:15,481 - INFO - Validated that all vrouters do not have the route table for VN default-domain:ctest-TestBasicVMVN-34967799:ctest-vn-31236606 2025-03-27 13:46:15,492 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-vn-31236606 info 2025-03-27 13:46:15,583 - INFO - IPAM: ctest-ipam-63395955 is not found in API Server 2025-03-27 13:46:15,597 - INFO - IPAM:ctest-ipam-63395955 is not found in control node 2025-03-27 13:46:15,858 - DEBUG - No XMPP flaps were noticed during the test
2025-03-27 13:46:15,858 - INFO - END TEST : test_ping_within_vn_two_vms_two_different_subnets : PASSED[0:00:38]
2025-03-27 13:46:15,858 - INFO - -------------------------------------------------------------------------------- 2025-03-27 13:46:16,572 - INFO - Deleted project: ctest-TestBasicVMVN-34967799, ID : fcf045f7-4e61-4b41-9ba6-3302354d4797