2026-08-03 02:46:03,771 - INFO - Domain Default found not creating 2026-08-03 02:46:03,967 - INFO - Project ctest-TestMultiInlineSVCIPv6-12010299 not found, creating it 2026-08-03 02:46:04,620 - INFO - Created Project:ctest-TestMultiInlineSVCIPv6-12010299, ID : 9ff32be5-1ca5-42e9-9bba-50a4c5b98d25 2026-08-03 02:46:07,745 - DEBUG - Response for create_network : {'network': {'id': '652daff2-3070-4219-9afe-2aa35bfdf2ad', 'name': 'ctest-mgmt_ctest-TestMultiInlineSVCIPv6-12010299-66200218', 'tenant_id': '9ff32be51ca542e99bba50a4c5b98d25', 'project_id': '9ff32be51ca542e99bba50a4c5b98d25', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestMultiInlineSVCIPv6-12010299', 'ctest-mgmt_ctest-TestMultiInlineSVCIPv6-12010299-66200218'], 'port_security_enabled': True, 'description': ''}} 2026-08-03 02:46:08,222 - DEBUG - Response for create_subnet : {'subnet': {'id': '0f55e85c-6212-4191-9d26-e91302bc3593', 'name': '', 'tenant_id': '9ff32be51ca542e99bba50a4c5b98d25', 'network_id': '652daff2-3070-4219-9afe-2aa35bfdf2ad', 'ip_version': 6, 'cidr': '2065:30bf:12fd:e5bb:826f:3ce::/96', 'allocation_pools': [{'start': '2065:30bf:12fd:e5bb:826f:3ce:0:2', 'end': '2065:30bf:12fd:e5bb:826f:3ce:ffff:fffe'}], 'gateway_ip': '2065:30bf:12fd:e5bb:826f:3ce:0:1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '2065:30bf:12fd:e5bb:826f:3ce:0:2', 'tags': [], 'project_id': '9ff32be51ca542e99bba50a4c5b98d25'}} 2026-08-03 02:46:08,512 - DEBUG - Response for create_subnet : {'subnet': {'id': '3b98b4c9-908f-40b3-aa34-e603e1cffc72', 'name': '', 'tenant_id': '9ff32be51ca542e99bba50a4c5b98d25', 'network_id': '652daff2-3070-4219-9afe-2aa35bfdf2ad', 'ip_version': 4, 'cidr': '18.81.198.192/26', 'allocation_pools': [{'start': '18.81.198.194', 'end': '18.81.198.254'}], 'gateway_ip': '18.81.198.193', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '18.81.198.194', 'tags': [], 'project_id': '9ff32be51ca542e99bba50a4c5b98d25'}} 2026-08-03 02:46:08,542 - INFO - Created VN ctest-mgmt_ctest-TestMultiInlineSVCIPv6-12010299-66200218 2026-08-03 02:46:08,557 - DEBUG - VN ctest-mgmt_ctest-TestMultiInlineSVCIPv6-12010299-66200218 UUID is 652daff2-3070-4219-9afe-2aa35bfdf2ad 2026-08-03 02:46:09,062 - DEBUG - Response for create_network : {'network': {'id': '63ce3d4b-dc03-450b-80be-cc0c8ec72978', 'name': 'ctest-left_ctest-TestMultiInlineSVCIPv6-12010299-72994488', 'tenant_id': '9ff32be51ca542e99bba50a4c5b98d25', 'project_id': '9ff32be51ca542e99bba50a4c5b98d25', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestMultiInlineSVCIPv6-12010299', 'ctest-left_ctest-TestMultiInlineSVCIPv6-12010299-72994488'], 'port_security_enabled': True, 'description': ''}} 2026-08-03 02:46:09,448 - DEBUG - Response for create_subnet : {'subnet': {'id': '4e030e8b-ad10-4ebb-b12a-44fc23723e40', 'name': '', 'tenant_id': '9ff32be51ca542e99bba50a4c5b98d25', 'network_id': '63ce3d4b-dc03-450b-80be-cc0c8ec72978', 'ip_version': 6, 'cidr': '2e92:3fc3:28c6:797d:43b0:dad1::/96', 'allocation_pools': [{'start': '2e92:3fc3:28c6:797d:43b0:dad1:0:2', 'end': '2e92:3fc3:28c6:797d:43b0:dad1:ffff:fffe'}], 'gateway_ip': '2e92:3fc3:28c6:797d:43b0:dad1:0:1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '2e92:3fc3:28c6:797d:43b0:dad1:0:2', 'tags': [], 'project_id': '9ff32be51ca542e99bba50a4c5b98d25'}} 2026-08-03 02:46:09,736 - DEBUG - Response for create_subnet : {'subnet': {'id': '9dbfe662-a524-4b0a-a653-960f616c773f', 'name': '', 'tenant_id': '9ff32be51ca542e99bba50a4c5b98d25', 'network_id': '63ce3d4b-dc03-450b-80be-cc0c8ec72978', 'ip_version': 4, 'cidr': '66.178.222.64/26', 'allocation_pools': [{'start': '66.178.222.66', 'end': '66.178.222.126'}], 'gateway_ip': '66.178.222.65', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '66.178.222.66', 'tags': [], 'project_id': '9ff32be51ca542e99bba50a4c5b98d25'}} 2026-08-03 02:46:09,770 - INFO - Created VN ctest-left_ctest-TestMultiInlineSVCIPv6-12010299-72994488 2026-08-03 02:46:09,785 - DEBUG - VN ctest-left_ctest-TestMultiInlineSVCIPv6-12010299-72994488 UUID is 63ce3d4b-dc03-450b-80be-cc0c8ec72978 2026-08-03 02:46:10,319 - DEBUG - Response for create_network : {'network': {'id': 'c336ad76-b7e6-40d2-950f-a492214228a7', 'name': 'ctest-right_ctest-TestMultiInlineSVCIPv6-12010299-28339592', 'tenant_id': '9ff32be51ca542e99bba50a4c5b98d25', 'project_id': '9ff32be51ca542e99bba50a4c5b98d25', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestMultiInlineSVCIPv6-12010299', 'ctest-right_ctest-TestMultiInlineSVCIPv6-12010299-28339592'], 'port_security_enabled': True, 'description': ''}} 2026-08-03 02:46:10,634 - DEBUG - Response for create_subnet : {'subnet': {'id': '6b1f4ffa-91f9-4ee2-85cd-ee3714444a63', 'name': '', 'tenant_id': '9ff32be51ca542e99bba50a4c5b98d25', 'network_id': 'c336ad76-b7e6-40d2-950f-a492214228a7', 'ip_version': 6, 'cidr': '2324:13f2:8919:edb8:6f19:8cb9::/96', 'allocation_pools': [{'start': '2324:13f2:8919:edb8:6f19:8cb9:0:2', 'end': '2324:13f2:8919:edb8:6f19:8cb9:ffff:fffe'}], 'gateway_ip': '2324:13f2:8919:edb8:6f19:8cb9:0:1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '2324:13f2:8919:edb8:6f19:8cb9:0:2', 'tags': [], 'project_id': '9ff32be51ca542e99bba50a4c5b98d25'}} 2026-08-03 02:46:10,962 - DEBUG - Response for create_subnet : {'subnet': {'id': '712fc46d-75f6-4b9d-9e98-571de4efc224', 'name': '', 'tenant_id': '9ff32be51ca542e99bba50a4c5b98d25', 'network_id': 'c336ad76-b7e6-40d2-950f-a492214228a7', 'ip_version': 4, 'cidr': '71.194.106.192/26', 'allocation_pools': [{'start': '71.194.106.194', 'end': '71.194.106.254'}], 'gateway_ip': '71.194.106.193', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '71.194.106.194', 'tags': [], 'project_id': '9ff32be51ca542e99bba50a4c5b98d25'}} 2026-08-03 02:46:11,007 - INFO - Created VN ctest-right_ctest-TestMultiInlineSVCIPv6-12010299-28339592 2026-08-03 02:46:11,035 - DEBUG - VN ctest-right_ctest-TestMultiInlineSVCIPv6-12010299-28339592 UUID is c336ad76-b7e6-40d2-950f-a492214228a7 2026-08-03 02:46:11,415 - DEBUG - Services list from nova: [, , , ] 2026-08-03 02:46:12,157 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-6129-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-6129-2) 2026-08-03 02:46:12,965 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-6129-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-6129-1) 2026-08-03 02:46:12,967 - INFO - ================================================================================ 2026-08-03 02:46:12,967 - INFO - STARTING TEST : test_svc_fate_sharing_basic 2026-08-03 02:46:13,697 - DEBUG - Nothing to compare xmpp stats {'10.0.0.38': {'10.20.0.25': '0', '10.20.0.29': '0'}, '10.0.0.19': {'10.20.0.25': '0', '10.20.0.29': '0'}} with 2026-08-03 02:46:13,697 - INFO - Initial checks done. Running the testcase now 2026-08-03 02:46:13,697 - INFO - 2026-08-03 02:46:13,697 - INFO - ================================================================================ 2026-08-03 02:46:13,697 - INFO - STARTING TEST : test_svc_fate_sharing_basic 2026-08-03 02:46:13,698 - INFO - TEST DESCRIPTION : Description: Basic test to validate fate sharing in a multi inline service chain with 2 SIs, 1 SVM each. Test steps: 1.Creating vm's - vm1 and vm2 in networks vn1 and vn2. 2.Creating 2 inline service instances SI1 and SI2. 3.Creating a service chain by applying the 2 service instances in a policy between t he VNs. 4.Associate an HC instance with right intf of one of the SVMs 5.Bring down the intf in the step 4 6.Verify that all the re-origined routes(ServiceChain) are deleted from all the RIs of all the SIs 7.Verify the step 6 in both control node and agent 8.Bring up the intf in the step 5 8.Verify that the routes should be re-originated again, and the traffic also starts flowing Pass criteria: Ping/Route deletion/Route addition should be successful. Maintainer : ankitja@juniper.net 2026-08-03 02:46:14,393 - DEBUG - Nothing to compare xmpp stats {'10.0.0.38': {'10.20.0.25': '0', '10.20.0.29': '0'}, '10.0.0.19': {'10.20.0.25': '0', '10.20.0.29': '0'}} with 2026-08-03 02:46:14,394 - INFO - Initial checks done. Running the testcase now 2026-08-03 02:46:14,394 - INFO - 2026-08-03 02:46:14,394 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-6129-2', 'an-jenkins-deploy-platform-ansible-os-6129-1']} 2026-08-03 02:46:14,394 - DEBUG - Creating service template: ['default-domain', 'ctest-service_template_1-32474883'] 2026-08-03 02:46:14,718 - DEBUG - Service template: ['default-domain', 'ctest-service_template_1-32474883'] created succesfully 2026-08-03 02:46:15,038 - DEBUG - Image by name tiny_in_net not found 2026-08-03 02:46:15,038 - DEBUG - Installing image tiny_in_net 2026-08-03 02:46:15,038 - DEBUG - Image info {'name': 'tinycore-in-network.qcow2.gz', 'webserver': '', 'location': '/images/tinycore/', 'username': 'tc', 'password': 'secret', 'type': 'os', 'flavor': 'contrail_flavor_tiny', 'params': '--container-format bare --disk-format vmdk --property vmware_disktype="sparse" --property vmware_adaptertype="ide" --property hypervisor_type=qemu', 'name_docker': 'phusion-baseimage-enablesshd', 'vmdk': 'service_chain_images/nat_service/tiny_in_net/tinycore-in-network-flat.vmdk', 'shrinked_vmdk': "'no'"} 2026-08-03 02:46:15,039 - DEBUG - Image build path file:///contrail-test/images/tinycore-in-network.qcow2.gz 2026-08-03 02:46:15,039 - DEBUG - Download image from local file file:///contrail-test/images/tinycore-in-network.qcow2.gz 2026-08-03 02:46:15,039 - DEBUG - Unzip image 2026-08-03 02:46:15,039 - DEBUG - Image download (local=True) (proxy=None): Execute cmd: gunzip -f /contrail-test/images/tinycore-in-network.qcow2.gz 2026-08-03 02:46:15,060 - INFO - image /contrail-test/images/tinycore-in-network.qcow2 is not found 2026-08-03 02:46:15,060 - DEBUG - Deleting service template: ['default-domain', 'ctest-service_template_1-32474883'] 2026-08-03 02:46:15,175 - INFO - Service template: ['default-domain', 'ctest-service_template_1-32474883'] deleted successfully. 2026-08-03 02:46:15,889 - DEBUG - No XMPP flaps were noticed during the test 2026-08-03 02:46:15,889 - INFO -
2026-08-03 02:46:15,889 - INFO - END TEST : test_svc_fate_sharing_basic : SKIPPED[0:00:02]
2026-08-03 02:46:15,890 - INFO - -------------------------------------------------------------------------------- 2026-08-03 02:46:15,890 - INFO - image /contrail-test/images/tinycore-in-network.qcow2 is not found 2026-08-03 02:46:16,601 - DEBUG - No XMPP flaps were noticed during the test 2026-08-03 02:46:16,601 - INFO -
2026-08-03 02:46:16,601 - INFO - END TEST : test_svc_fate_sharing_basic : SKIPPED[0:00:04]
2026-08-03 02:46:16,601 - INFO - -------------------------------------------------------------------------------- 2026-08-03 02:46:16,605 - INFO - Skip interface_detach for VM ctest-right_vm_ctest-TestMultiInlineSVCIPv6-12010299-37166212 in state BUILD 2026-08-03 02:46:16,605 - INFO - Deleting VM ctest-right_vm_ctest-TestMultiInlineSVCIPv6-12010299-37166212 2026-08-03 02:46:16,683 - ERROR - VM ctest-right_vm_ctest-TestMultiInlineSVCIPv6-12010299-37166212 has failed to come up 2026-08-03 02:46:16,683 - ERROR - Nova failed to get host of the VM 2026-08-03 02:46:16,683 - ERROR - VM ctest-right_vm_ctest-TestMultiInlineSVCIPv6-12010299-37166212 has failed to come up 2026-08-03 02:46:16,683 - ERROR - Nova failed to get host of the VM 2026-08-03 02:46:16,837 - INFO - Skip interface_detach for VM ctest-left_vm_ctest-TestMultiInlineSVCIPv6-12010299-95382003 in state BUILD 2026-08-03 02:46:16,837 - INFO - Deleting VM ctest-left_vm_ctest-TestMultiInlineSVCIPv6-12010299-95382003 2026-08-03 02:46:16,837 - ERROR - VM ctest-left_vm_ctest-TestMultiInlineSVCIPv6-12010299-95382003 has failed to come up 2026-08-03 02:46:16,837 - ERROR - Nova failed to get host of the VM 2026-08-03 02:46:16,837 - ERROR - VM ctest-left_vm_ctest-TestMultiInlineSVCIPv6-12010299-95382003 has failed to come up 2026-08-03 02:46:16,837 - ERROR - Nova failed to get host of the VM 2026-08-03 02:46:17,074 - INFO - Deleting VN ctest-left_ctest-TestMultiInlineSVCIPv6-12010299-72994488 2026-08-03 02:46:17,235 - DEBUG - VN 63ce3d4b-dc03-450b-80be-cc0c8ec72978 still in use: Unable to complete operation on network 63ce3d4b-dc03-450b-80be-cc0c8ec72978. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-46f917b8-3892-4f62-8990-05b2c59635ee'] 2026-08-03 02:46:17,235 - WARNING - Deleting VN ctest-left_ctest-TestMultiInlineSVCIPv6-12010299-72994488 failed..Will retry 2026-08-03 02:46:19,329 - DEBUG - VN 63ce3d4b-dc03-450b-80be-cc0c8ec72978 still in use: Unable to complete operation on network 63ce3d4b-dc03-450b-80be-cc0c8ec72978. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-6d8941cd-c5a6-459d-9038-97c1ce4b8496'] 2026-08-03 02:46:19,329 - WARNING - Deleting VN ctest-left_ctest-TestMultiInlineSVCIPv6-12010299-72994488 failed..Will retry 2026-08-03 02:46:21,382 - DEBUG - VN 63ce3d4b-dc03-450b-80be-cc0c8ec72978 still in use: Unable to complete operation on network 63ce3d4b-dc03-450b-80be-cc0c8ec72978. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-5c7dd39f-0d1d-4f3d-ba56-ccda7c44ecb8'] 2026-08-03 02:46:21,382 - WARNING - Deleting VN ctest-left_ctest-TestMultiInlineSVCIPv6-12010299-72994488 failed..Will retry 2026-08-03 02:46:23,436 - DEBUG - VN 63ce3d4b-dc03-450b-80be-cc0c8ec72978 still in use: Unable to complete operation on network 63ce3d4b-dc03-450b-80be-cc0c8ec72978. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-03003b11-7b7f-43aa-ac30-b69856eb1fd1'] 2026-08-03 02:46:23,436 - WARNING - Deleting VN ctest-left_ctest-TestMultiInlineSVCIPv6-12010299-72994488 failed..Will retry 2026-08-03 02:46:25,490 - DEBUG - VN 63ce3d4b-dc03-450b-80be-cc0c8ec72978 still in use: Unable to complete operation on network 63ce3d4b-dc03-450b-80be-cc0c8ec72978. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-57e5f2d3-f180-45f3-b330-bc46fe4433fe'] 2026-08-03 02:46:25,491 - WARNING - Deleting VN ctest-left_ctest-TestMultiInlineSVCIPv6-12010299-72994488 failed..Will retry 2026-08-03 02:46:27,539 - DEBUG - VN 63ce3d4b-dc03-450b-80be-cc0c8ec72978 still in use: Unable to complete operation on network 63ce3d4b-dc03-450b-80be-cc0c8ec72978. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-94f27853-d4cf-4bf3-914f-68d8e31aa916'] 2026-08-03 02:46:27,540 - WARNING - Deleting VN ctest-left_ctest-TestMultiInlineSVCIPv6-12010299-72994488 failed..Will retry 2026-08-03 02:46:29,588 - DEBUG - VN 63ce3d4b-dc03-450b-80be-cc0c8ec72978 still in use: Unable to complete operation on network 63ce3d4b-dc03-450b-80be-cc0c8ec72978. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-00608771-ce34-4ac1-adb4-5075227e750b'] 2026-08-03 02:46:29,588 - WARNING - Deleting VN ctest-left_ctest-TestMultiInlineSVCIPv6-12010299-72994488 failed..Will retry 2026-08-03 02:46:31,640 - DEBUG - VN 63ce3d4b-dc03-450b-80be-cc0c8ec72978 still in use: Unable to complete operation on network 63ce3d4b-dc03-450b-80be-cc0c8ec72978. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-306e94f2-f32f-41b7-8d9f-5d03c6f425d8'] 2026-08-03 02:46:31,640 - WARNING - Deleting VN ctest-left_ctest-TestMultiInlineSVCIPv6-12010299-72994488 failed..Will retry 2026-08-03 02:46:33,688 - DEBUG - VN 63ce3d4b-dc03-450b-80be-cc0c8ec72978 still in use: Unable to complete operation on network 63ce3d4b-dc03-450b-80be-cc0c8ec72978. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-6b9b4afe-ddfc-4f33-8b63-700d01dd071b'] 2026-08-03 02:46:33,688 - WARNING - Deleting VN ctest-left_ctest-TestMultiInlineSVCIPv6-12010299-72994488 failed..Will retry 2026-08-03 02:46:35,747 - DEBUG - VN 63ce3d4b-dc03-450b-80be-cc0c8ec72978 still in use: Unable to complete operation on network 63ce3d4b-dc03-450b-80be-cc0c8ec72978. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-2adcb94f-dd82-444a-91e2-dae5c39205b2'] 2026-08-03 02:46:35,747 - WARNING - Deleting VN ctest-left_ctest-TestMultiInlineSVCIPv6-12010299-72994488 failed..Will retry 2026-08-03 02:46:37,809 - DEBUG - VN 63ce3d4b-dc03-450b-80be-cc0c8ec72978 still in use: Unable to complete operation on network 63ce3d4b-dc03-450b-80be-cc0c8ec72978. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-2e740bba-d5fa-42ee-bf99-1a0c1d647f9d'] 2026-08-03 02:46:37,810 - WARNING - Deleting VN ctest-left_ctest-TestMultiInlineSVCIPv6-12010299-72994488 failed..Will retry 2026-08-03 02:46:39,857 - DEBUG - VN 63ce3d4b-dc03-450b-80be-cc0c8ec72978 still in use: Unable to complete operation on network 63ce3d4b-dc03-450b-80be-cc0c8ec72978. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-74fcfbc4-dbaa-4f86-887f-3080ad5201f1'] 2026-08-03 02:46:39,857 - WARNING - Deleting VN ctest-left_ctest-TestMultiInlineSVCIPv6-12010299-72994488 failed..Will retry 2026-08-03 02:46:41,906 - DEBUG - VN 63ce3d4b-dc03-450b-80be-cc0c8ec72978 still in use: Unable to complete operation on network 63ce3d4b-dc03-450b-80be-cc0c8ec72978. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-b6a522fa-d345-420d-8548-7c16301a12cb'] 2026-08-03 02:46:41,906 - WARNING - Deleting VN ctest-left_ctest-TestMultiInlineSVCIPv6-12010299-72994488 failed..Will retry 2026-08-03 02:46:44,218 - DEBUG - Response for deleting network () 2026-08-03 02:46:44,218 - INFO - Deleting VN ctest-right_ctest-TestMultiInlineSVCIPv6-12010299-28339592 2026-08-03 02:46:44,516 - DEBUG - Response for deleting network () 2026-08-03 02:46:44,516 - INFO - Deleting VN ctest-mgmt_ctest-TestMultiInlineSVCIPv6-12010299-66200218 2026-08-03 02:46:44,777 - DEBUG - Response for deleting network () 2026-08-03 02:46:45,878 - INFO - Deleted project: ctest-TestMultiInlineSVCIPv6-12010299, ID : 9ff32be5-1ca5-42e9-9bba-50a4c5b98d25