2025-10-29 11:38:55,182 - INFO - Domain Default found not creating
2025-10-29 11:38:55,382 - INFO - Project ctest-TestMultiInlineSVC-58759782 not found, creating it
2025-10-29 11:38:56,053 - INFO - Created Project:ctest-TestMultiInlineSVC-58759782, ID : f42d390d-84f7-4ce6-a861-3f4e9ebb26d0 
2025-10-29 11:38:58,408 - DEBUG - Response for create_network : {'network': {'id': 'b7209b8e-27b7-4d71-bd08-804082639ea0', 'name': 'ctest-mgmt_ctest-TestMultiInlineSVC-58759782-33161390', 'tenant_id': 'f42d390d84f74ce6a8613f4e9ebb26d0', 'project_id': 'f42d390d84f74ce6a8613f4e9ebb26d0', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestMultiInlineSVC-58759782', 'ctest-mgmt_ctest-TestMultiInlineSVC-58759782-33161390'], 'port_security_enabled': True, 'description': ''}}
2025-10-29 11:38:58,745 - DEBUG - Response for create_subnet : {'subnet': {'id': 'a49271d3-d79c-4987-8638-cdc2e557aafc', 'name': '', 'tenant_id': 'f42d390d84f74ce6a8613f4e9ebb26d0', 'network_id': 'b7209b8e-27b7-4d71-bd08-804082639ea0', 'ip_version': 4, 'cidr': '196.177.234.192/26', 'allocation_pools': [{'start': '196.177.234.194', 'end': '196.177.234.254'}], 'gateway_ip': '196.177.234.193', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '196.177.234.194', 'tags': [], 'project_id': 'f42d390d84f74ce6a8613f4e9ebb26d0'}}
2025-10-29 11:38:58,769 - INFO - Created VN ctest-mgmt_ctest-TestMultiInlineSVC-58759782-33161390
2025-10-29 11:38:58,783 - DEBUG - VN ctest-mgmt_ctest-TestMultiInlineSVC-58759782-33161390 UUID is b7209b8e-27b7-4d71-bd08-804082639ea0
2025-10-29 11:38:59,153 - DEBUG - Response for create_network : {'network': {'id': '383ae0a8-8684-4f6e-ae65-f39e584f5a46', 'name': 'ctest-left_ctest-TestMultiInlineSVC-58759782-42878262', 'tenant_id': 'f42d390d84f74ce6a8613f4e9ebb26d0', 'project_id': 'f42d390d84f74ce6a8613f4e9ebb26d0', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestMultiInlineSVC-58759782', 'ctest-left_ctest-TestMultiInlineSVC-58759782-42878262'], 'port_security_enabled': True, 'description': ''}}
2025-10-29 11:38:59,366 - DEBUG - Response for create_subnet : {'subnet': {'id': '3a4d954a-bd23-4082-a66e-a1a7aa6ac662', 'name': '', 'tenant_id': 'f42d390d84f74ce6a8613f4e9ebb26d0', 'network_id': '383ae0a8-8684-4f6e-ae65-f39e584f5a46', 'ip_version': 4, 'cidr': '163.101.198.192/26', 'allocation_pools': [{'start': '163.101.198.194', 'end': '163.101.198.254'}], 'gateway_ip': '163.101.198.193', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '163.101.198.194', 'tags': [], 'project_id': 'f42d390d84f74ce6a8613f4e9ebb26d0'}}
2025-10-29 11:38:59,388 - INFO - Created VN ctest-left_ctest-TestMultiInlineSVC-58759782-42878262
2025-10-29 11:38:59,404 - DEBUG - VN ctest-left_ctest-TestMultiInlineSVC-58759782-42878262 UUID is 383ae0a8-8684-4f6e-ae65-f39e584f5a46
2025-10-29 11:38:59,799 - DEBUG - Response for create_network : {'network': {'id': '1849a738-6a06-4e8b-b5df-85cd12bb1fd5', 'name': 'ctest-right_ctest-TestMultiInlineSVC-58759782-84267650', 'tenant_id': 'f42d390d84f74ce6a8613f4e9ebb26d0', 'project_id': 'f42d390d84f74ce6a8613f4e9ebb26d0', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestMultiInlineSVC-58759782', 'ctest-right_ctest-TestMultiInlineSVC-58759782-84267650'], 'port_security_enabled': True, 'description': ''}}
2025-10-29 11:39:00,091 - DEBUG - Response for create_subnet : {'subnet': {'id': '155ff6ec-0195-4061-b0ec-a92c626ff05c', 'name': '', 'tenant_id': 'f42d390d84f74ce6a8613f4e9ebb26d0', 'network_id': '1849a738-6a06-4e8b-b5df-85cd12bb1fd5', 'ip_version': 4, 'cidr': '43.119.105.128/26', 'allocation_pools': [{'start': '43.119.105.130', 'end': '43.119.105.190'}], 'gateway_ip': '43.119.105.129', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '43.119.105.130', 'tags': [], 'project_id': 'f42d390d84f74ce6a8613f4e9ebb26d0'}}
2025-10-29 11:39:00,118 - INFO - Created VN ctest-right_ctest-TestMultiInlineSVC-58759782-84267650
2025-10-29 11:39:00,131 - DEBUG - VN ctest-right_ctest-TestMultiInlineSVC-58759782-84267650 UUID is 1849a738-6a06-4e8b-b5df-85cd12bb1fd5
2025-10-29 11:39:00,272 - DEBUG - Services list from nova: [, , , ]
2025-10-29 11:39:01,381 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4521-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4521-2)
2025-10-29 11:39:02,885 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4521-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4521-1)
2025-10-29 11:39:02,888 - INFO - ================================================================================
2025-10-29 11:39:02,888 - INFO - STARTING TEST    : test_svc_fate_sharing_basic
2025-10-29 11:39:02,888 - INFO - TEST DESCRIPTION : 
        Description: Basic test to validate fate sharing in a multi inline service chain with 2 SIs, 1 SVM each.
        Test steps:
                         1.Creating vm's - vm1 and vm2 in networks vn1 and vn2.
                         2.Creating 2 inline service instances SI1 and SI2.
                         3.Creating a service chain by applying the 2 service instances in a policy between t
                    he VNs.
                         4.Associate an HC instance with right intf of one of the SVMs
                         5.Bring down the intf in the step 4
                         6.Verify that all the re-origined routes(ServiceChain) are deleted from all the RIs of all the SIs
                         7.Verify the step 6 in both control node and agent
                         8.Bring up the intf in the step 5
                         8.Verify that the routes should be re-originated again, and the traffic also starts flowing
        Pass criteria: Ping/Route deletion/Route addition should be successful.
        Maintainer : ankitja@juniper.net
        
2025-10-29 11:39:03,611 - DEBUG - Nothing to compare xmpp stats {'10.0.0.49': {'10.20.0.17': '0', '10.20.0.254': '0'}, '10.0.0.50': {'10.20.0.254': '0', '10.20.0.14': '0'}} with
2025-10-29 11:39:03,612 - INFO - Initial checks done. Running the testcase now
2025-10-29 11:39:03,612 - INFO - 
2025-10-29 11:39:03,612 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-4521-2', 'an-jenkins-deploy-platform-ansible-os-4521-1']}
2025-10-29 11:39:03,613 - DEBUG - Creating service template: ['default-domain', 'ctest-service_template_1-53283853']
2025-10-29 11:39:03,979 - DEBUG - Service template: ['default-domain', 'ctest-service_template_1-53283853'] created succesfully
2025-10-29 11:39:04,300 - DEBUG - Image by name tiny_in_net not found
2025-10-29 11:39:04,300 - DEBUG - Installing image tiny_in_net
2025-10-29 11:39:04,300 - DEBUG - Image info {'name': 'tinycore-in-network.qcow2.gz', 'webserver': '', 'location': '/images/tinycore/', 'username': 'tc', 'password': 'secret', 'type': 'os', 'flavor': 'contrail_flavor_tiny', 'params': '--container-format bare --disk-format vmdk --property vmware_disktype="sparse" --property vmware_adaptertype="ide" --property hypervisor_type=qemu', 'name_docker': 'phusion-baseimage-enablesshd', 'vmdk': 'service_chain_images/nat_service/tiny_in_net/tinycore-in-network-flat.vmdk', 'shrinked_vmdk': "'no'"}
2025-10-29 11:39:04,300 - DEBUG - Image build path file:///contrail-test/images/tinycore-in-network.qcow2.gz
2025-10-29 11:39:04,300 - DEBUG - Download image from local file file:///contrail-test/images/tinycore-in-network.qcow2.gz
2025-10-29 11:39:04,300 - DEBUG - Unzip image
2025-10-29 11:39:04,300 - DEBUG - Image download (local=True) (proxy=None): Execute cmd: gunzip -f /contrail-test/images/tinycore-in-network.qcow2.gz
2025-10-29 11:39:04,314 - INFO - image /contrail-test/images/tinycore-in-network.qcow2 is not found
2025-10-29 11:39:04,315 - DEBUG - Deleting service template: ['default-domain', 'ctest-service_template_1-53283853']
2025-10-29 11:39:04,420 - INFO - Service template: ['default-domain', 'ctest-service_template_1-53283853'] deleted successfully.
2025-10-29 11:39:05,112 - DEBUG - No XMPP flaps were noticed during the test
2025-10-29 11:39:05,112 - INFO - 
2025-10-29 11:39:05,112 - INFO - END TEST : test_svc_fate_sharing_basic : SKIPPED[0:00:03]
2025-10-29 11:39:05,112 - INFO - --------------------------------------------------------------------------------
2025-10-29 11:39:05,119 - INFO - ================================================================================
2025-10-29 11:39:05,119 - INFO - STARTING TEST    : test_three_stage_v2_SC
2025-10-29 11:39:05,119 - INFO - TEST DESCRIPTION : 
        Description: Validate multi-Inline SVC version 2.
        Test steps:
                         1.Creating vm's - vm1 and vm2 in networks vn1 and vn2.
                         2.Creating 3 service instances.
                         3.Creating a service chain by applying the 3 service instances in a policy between t
                    he VNs.
                         4.There should be no traffic loss.
        Pass criteria: Ping between the VMs should be successful.
        Maintainer : ganeshahv@juniper.net
        
2025-10-29 11:39:05,777 - DEBUG - Nothing to compare xmpp stats {'10.0.0.49': {'10.20.0.17': '0', '10.20.0.254': '0'}, '10.0.0.50': {'10.20.0.254': '0', '10.20.0.14': '0'}} with
2025-10-29 11:39:05,777 - INFO - Initial checks done. Running the testcase now
2025-10-29 11:39:05,777 - INFO - 
2025-10-29 11:39:05,777 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-4521-2', 'an-jenkins-deploy-platform-ansible-os-4521-1']}
2025-10-29 11:39:05,778 - INFO - Skipped as test requires at least 3 nodes, but only 2 found
2025-10-29 11:39:06,449 - DEBUG - No XMPP flaps were noticed during the test
2025-10-29 11:39:06,449 - INFO - 
2025-10-29 11:39:06,449 - INFO - END TEST : test_three_stage_v2_SC : SKIPPED[0:00:01]
2025-10-29 11:39:06,449 - INFO - --------------------------------------------------------------------------------
2025-10-29 11:39:06,453 - INFO - Skip interface_detach for VM ctest-right_vm_ctest-TestMultiInlineSVC-58759782-93970418 in state BUILD
2025-10-29 11:39:06,453 - INFO - Deleting VM ctest-right_vm_ctest-TestMultiInlineSVC-58759782-93970418
2025-10-29 11:39:06,491 - ERROR - VM ctest-right_vm_ctest-TestMultiInlineSVC-58759782-93970418 has failed to come up
2025-10-29 11:39:06,491 - ERROR - Nova failed to get host of the VM
2025-10-29 11:39:06,491 - ERROR - VM ctest-right_vm_ctest-TestMultiInlineSVC-58759782-93970418 has failed to come up
2025-10-29 11:39:06,491 - ERROR - Nova failed to get host of the VM
2025-10-29 11:39:06,578 - INFO - Skip interface_detach for VM ctest-left_vm_ctest-TestMultiInlineSVC-58759782-29847870 in state BUILD
2025-10-29 11:39:06,578 - INFO - Deleting VM ctest-left_vm_ctest-TestMultiInlineSVC-58759782-29847870
2025-10-29 11:39:06,578 - ERROR - VM ctest-left_vm_ctest-TestMultiInlineSVC-58759782-29847870 has failed to come up
2025-10-29 11:39:06,578 - ERROR - Nova failed to get host of the VM
2025-10-29 11:39:06,578 - ERROR - VM ctest-left_vm_ctest-TestMultiInlineSVC-58759782-29847870 has failed to come up
2025-10-29 11:39:06,578 - ERROR - Nova failed to get host of the VM
2025-10-29 11:39:06,657 - INFO - Deleting VN ctest-left_ctest-TestMultiInlineSVC-58759782-42878262 
2025-10-29 11:39:06,706 - DEBUG - VN 383ae0a8-8684-4f6e-ae65-f39e584f5a46 still in use: Unable to complete operation on network 383ae0a8-8684-4f6e-ae65-f39e584f5a46. There are one or more ports still in use on the network.
Neutron server returns request_ids: ['req-04a4f907-cb4c-477d-a455-3df1c622066e']
2025-10-29 11:39:06,706 - WARNING - Deleting VN ctest-left_ctest-TestMultiInlineSVC-58759782-42878262 failed..Will retry
2025-10-29 11:39:08,901 - DEBUG - Response for deleting network ()
2025-10-29 11:39:08,902 - INFO - Deleting VN ctest-right_ctest-TestMultiInlineSVC-58759782-84267650 
2025-10-29 11:39:09,117 - DEBUG - Response for deleting network ()
2025-10-29 11:39:09,117 - INFO - Deleting VN ctest-mgmt_ctest-TestMultiInlineSVC-58759782-33161390 
2025-10-29 11:39:09,350 - DEBUG - Response for deleting network ()
2025-10-29 11:39:10,126 - INFO - Deleted project: ctest-TestMultiInlineSVC-58759782, ID : f42d390d-84f7-4ce6-a861-3f4e9ebb26d0