2024-10-25 15:21:57,740 - INFO - Domain Default found not creating 2024-10-25 15:21:57,943 - INFO - Project ctest-TestSvcRegr-42593455 not found, creating it 2024-10-25 15:21:58,472 - INFO - Created Project:ctest-TestSvcRegr-42593455, ID : 944418ee-ac66-4359-804f-c1002088adc7 2024-10-25 15:22:00,265 - INFO - ================================================================================ 2024-10-25 15:22:00,265 - INFO - STARTING TEST : test_svc_in_network_datapath 2024-10-25 15:22:00,528 - DEBUG - Nothing to compare xmpp stats {'10.0.0.31': {'10.20.0.14': '0'}} with 2024-10-25 15:22:00,528 - INFO - Initial checks done. Running the testcase now 2024-10-25 15:22:00,528 - INFO - 2024-10-25 15:22:01,196 - DEBUG - Response for create_network : {'network': {'id': 'ca6b6625-31d8-4176-8b80-0c0305ff6583', 'name': 'ctest-mgmt_vn-02141399', 'tenant_id': '944418eeac664359804fc1002088adc7', 'project_id': '944418eeac664359804fc1002088adc7', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcRegr-42593455', 'ctest-mgmt_vn-02141399'], 'port_security_enabled': True, 'description': ''}} 2024-10-25 15:22:01,395 - DEBUG - Response for create_subnet : {'subnet': {'id': '2a2f056d-4e2d-4f52-bee5-84001b9d36d7', 'name': '', 'tenant_id': '944418eeac664359804fc1002088adc7', 'network_id': 'ca6b6625-31d8-4176-8b80-0c0305ff6583', 'ip_version': 4, 'cidr': '115.152.213.0/26', 'allocation_pools': [{'start': '115.152.213.2', 'end': '115.152.213.62'}], 'gateway_ip': '115.152.213.1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '115.152.213.2', 'tags': [], 'project_id': '944418eeac664359804fc1002088adc7'}} 2024-10-25 15:22:01,420 - INFO - Created VN ctest-mgmt_vn-02141399 2024-10-25 15:22:01,484 - DEBUG - VN ctest-mgmt_vn-02141399 UUID is ca6b6625-31d8-4176-8b80-0c0305ff6583 2024-10-25 15:22:01,621 - DEBUG - Requesting: http://10.0.0.31:8082/domains 2024-10-25 15:22:01,964 - DEBUG - Requesting: http://10.0.0.31:8082/domains 2024-10-25 15:22:02,004 - DEBUG - Requesting: http://10.0.0.31:8082/domain/5c9222a5-1004-4c8d-a5ba-6a8cd4423477 2024-10-25 15:22:02,027 - DEBUG - Requesting: http://10.0.0.31:8082/project/944418ee-ac66-4359-804f-c1002088adc7 2024-10-25 15:22:02,119 - DEBUG - Requesting: http://10.0.0.31:8082/virtual-network/ca6b6625-31d8-4176-8b80-0c0305ff6583 2024-10-25 15:22:02,138 - DEBUG - Requesting: http://10.0.0.31:8082/virtual-network/ca6b6625-31d8-4176-8b80-0c0305ff6583 2024-10-25 15:22:02,154 - DEBUG - Requesting: http://10.0.0.31:8082/routing-instance/fdd5bc74-2ff0-4447-8d6c-e0a926ec246a 2024-10-25 15:22:02,165 - DEBUG - Requesting: http://10.0.0.31:8082/routing-instance/fdd5bc74-2ff0-4447-8d6c-e0a926ec246a 2024-10-25 15:22:02,176 - DEBUG - Requesting: http://10.0.0.31:8082/route-target/3a40366d-2546-4743-96a6-14e0ea77b97b 2024-10-25 15:22:02,185 - DEBUG - Route Targets: ['target:64512:8000005'] 2024-10-25 15:22:02,185 - DEBUG - Requesting: http://10.0.0.31:8082/virtual-network/ca6b6625-31d8-4176-8b80-0c0305ff6583 2024-10-25 15:22:02,204 - DEBUG - Requesting: http://10.0.0.31:8082/routing-instance/fdd5bc74-2ff0-4447-8d6c-e0a926ec246a 2024-10-25 15:22:02,232 - INFO - Verified VN network id 8 for VN ca6b6625-31d8-4176-8b80-0c0305ff6583 2024-10-25 15:22:02,232 - INFO - Verifications in API Server for VN ctest-mgmt_vn-02141399 passed 2024-10-25 15:22:02,232 - DEBUG - Requesting: http://10.0.0.31:8082/virtual-network/ca6b6625-31d8-4176-8b80-0c0305ff6583 2024-10-25 15:22:02,252 - DEBUG - Requesting: http://10.0.0.31:8082/routing-instance/fdd5bc74-2ff0-4447-8d6c-e0a926ec246a 2024-10-25 15:22:02,265 - DEBUG - Requesting: http://10.0.0.31:8082/routing-instance/fdd5bc74-2ff0-4447-8d6c-e0a926ec246a 2024-10-25 15:22:02,276 - DEBUG - Requesting: http://10.0.0.31:8082/route-target/3a40366d-2546-4743-96a6-14e0ea77b97b 2024-10-25 15:22:02,293 - DEBUG - Control-node 10.0.0.31 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-42593455:ctest-mgmt_vn-02141399', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-42593455', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '14585864128105562486', 'uuid-lslong': '10052047575416006019'}, 'enable': 'true', 'created': '2024-10-25T15:22:01', 'last-modified': '2024-10-25T15:22:01', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.938503'} 2024-10-25 15:22:02,302 - DEBUG - Route Targets: ['target:64512:8000005'] 2024-10-25 15:22:02,302 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-mgmt_vn-02141399 passed 2024-10-25 15:22:02,303 - DEBUG - ====Verifying policy data for ctest-mgmt_vn-02141399 in API_Server ====== 2024-10-25 15:22:02,303 - DEBUG - Requesting: http://10.0.0.31:8082/domains 2024-10-25 15:22:02,311 - DEBUG - Requesting: http://10.0.0.31:8082/domain/5c9222a5-1004-4c8d-a5ba-6a8cd4423477 2024-10-25 15:22:02,330 - DEBUG - Requesting: http://10.0.0.31:8082/project/944418ee-ac66-4359-804f-c1002088adc7 2024-10-25 15:22:02,428 - DEBUG - Requesting: http://10.0.0.31:8082/virtual-network/ca6b6625-31d8-4176-8b80-0c0305ff6583 2024-10-25 15:22:02,448 - DEBUG - =>VN ctest-mgmt_vn-02141399 has no policy to be verified 2024-10-25 15:22:02,448 - DEBUG - Verifying the vn in opserver 2024-10-25 15:22:02,448 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-42593455:ctest-mgmt_vn-02141399 virtual network link through opserver 10.0.0.31 2024-10-25 15:22:02,449 - DEBUG - Requesting: http://10.0.0.31:8081/analytics/uves/virtual-networks 2024-10-25 15:22:02,789 - DEBUG - Requesting: http://10.0.0.31:8081/analytics/uves/virtual-networks 2024-10-25 15:22:02,910 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-42593455:ctest-mgmt_vn-02141399', 'href': 'http://10.0.0.31:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-42593455:ctest-mgmt_vn-02141399?flat'} 2024-10-25 15:22:02,910 - INFO - Validated that VN default-domain:ctest-TestSvcRegr-42593455:ctest-mgmt_vn-02141399 is found in opserver 2024-10-25 15:22:02,924 - DEBUG - Do not have enough data to verify VN in agent 2024-10-25 15:22:02,932 - DEBUG - VRF ids for VN ctest-mgmt_vn-02141399: {} 2024-10-25 15:22:03,559 - DEBUG - Response for create_network : {'network': {'id': '38781dff-7c93-4305-a700-448b8101d306', 'name': 'ctest-left_vn-76866856', 'tenant_id': '944418eeac664359804fc1002088adc7', 'project_id': '944418eeac664359804fc1002088adc7', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcRegr-42593455', 'ctest-left_vn-76866856'], 'port_security_enabled': True, 'description': ''}} 2024-10-25 15:22:03,892 - DEBUG - Response for create_subnet : {'subnet': {'id': '37fb7b3e-7a88-43ff-a8c7-8be545aa2dac', 'name': '', 'tenant_id': '944418eeac664359804fc1002088adc7', 'network_id': '38781dff-7c93-4305-a700-448b8101d306', 'ip_version': 4, 'cidr': '171.145.10.192/26', 'allocation_pools': [{'start': '171.145.10.194', 'end': '171.145.10.254'}], 'gateway_ip': '171.145.10.193', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '171.145.10.194', 'tags': [], 'project_id': '944418eeac664359804fc1002088adc7'}} 2024-10-25 15:22:03,915 - INFO - Created VN ctest-left_vn-76866856 2024-10-25 15:22:03,980 - DEBUG - VN ctest-left_vn-76866856 UUID is 38781dff-7c93-4305-a700-448b8101d306 2024-10-25 15:22:04,110 - DEBUG - Requesting: http://10.0.0.31:8082/domains 2024-10-25 15:22:04,119 - DEBUG - Requesting: http://10.0.0.31:8082/domain/5c9222a5-1004-4c8d-a5ba-6a8cd4423477 2024-10-25 15:22:04,137 - DEBUG - Requesting: http://10.0.0.31:8082/project/944418ee-ac66-4359-804f-c1002088adc7 2024-10-25 15:22:04,227 - DEBUG - Requesting: http://10.0.0.31:8082/virtual-network/38781dff-7c93-4305-a700-448b8101d306 2024-10-25 15:22:04,248 - DEBUG - Requesting: http://10.0.0.31:8082/virtual-network/38781dff-7c93-4305-a700-448b8101d306 2024-10-25 15:22:04,267 - DEBUG - Requesting: http://10.0.0.31:8082/routing-instance/df6c5be0-1bb2-4f69-8e9e-0ea46fe3d2f1 2024-10-25 15:22:04,279 - DEBUG - Requesting: http://10.0.0.31:8082/routing-instance/df6c5be0-1bb2-4f69-8e9e-0ea46fe3d2f1 2024-10-25 15:22:04,292 - DEBUG - Route Targets: [] 2024-10-25 15:22:04,292 - DEBUG - RT names not yet present for VN ctest-left_vn-76866856 2024-10-25 15:22:09,293 - DEBUG - Requesting: http://10.0.0.31:8082/domains 2024-10-25 15:22:09,301 - DEBUG - Requesting: http://10.0.0.31:8082/domain/5c9222a5-1004-4c8d-a5ba-6a8cd4423477 2024-10-25 15:22:09,318 - DEBUG - Requesting: http://10.0.0.31:8082/project/944418ee-ac66-4359-804f-c1002088adc7 2024-10-25 15:22:09,412 - DEBUG - Requesting: http://10.0.0.31:8082/virtual-network/38781dff-7c93-4305-a700-448b8101d306 2024-10-25 15:22:09,430 - DEBUG - Requesting: http://10.0.0.31:8082/virtual-network/38781dff-7c93-4305-a700-448b8101d306 2024-10-25 15:22:09,448 - DEBUG - Requesting: http://10.0.0.31:8082/routing-instance/df6c5be0-1bb2-4f69-8e9e-0ea46fe3d2f1 2024-10-25 15:22:09,460 - DEBUG - Requesting: http://10.0.0.31:8082/routing-instance/df6c5be0-1bb2-4f69-8e9e-0ea46fe3d2f1 2024-10-25 15:22:09,470 - DEBUG - Requesting: http://10.0.0.31:8082/route-target/6d353139-2c14-4c95-95f9-1784d4ec08cf 2024-10-25 15:22:09,479 - DEBUG - Route Targets: ['target:64512:8000006'] 2024-10-25 15:22:09,479 - DEBUG - Requesting: http://10.0.0.31:8082/virtual-network/38781dff-7c93-4305-a700-448b8101d306 2024-10-25 15:22:09,496 - DEBUG - Requesting: http://10.0.0.31:8082/routing-instance/df6c5be0-1bb2-4f69-8e9e-0ea46fe3d2f1 2024-10-25 15:22:09,521 - INFO - Verified VN network id 9 for VN 38781dff-7c93-4305-a700-448b8101d306 2024-10-25 15:22:09,521 - INFO - Verifications in API Server for VN ctest-left_vn-76866856 passed 2024-10-25 15:22:09,521 - DEBUG - Requesting: http://10.0.0.31:8082/virtual-network/38781dff-7c93-4305-a700-448b8101d306 2024-10-25 15:22:09,538 - DEBUG - Requesting: http://10.0.0.31:8082/routing-instance/df6c5be0-1bb2-4f69-8e9e-0ea46fe3d2f1 2024-10-25 15:22:09,550 - DEBUG - Requesting: http://10.0.0.31:8082/routing-instance/df6c5be0-1bb2-4f69-8e9e-0ea46fe3d2f1 2024-10-25 15:22:09,560 - DEBUG - Requesting: http://10.0.0.31:8082/route-target/6d353139-2c14-4c95-95f9-1784d4ec08cf 2024-10-25 15:22:09,576 - DEBUG - Control-node 10.0.0.31 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-42593455:ctest-left_vn-76866856', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-42593455', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '4069035246473134853', 'uuid-lslong': '12033693570289488646'}, 'enable': 'true', 'created': '2024-10-25T15:22:03', 'last-modified': '2024-10-25T15:22:03', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:05.722142'} 2024-10-25 15:22:09,584 - DEBUG - Route Targets: ['target:64512:8000006'] 2024-10-25 15:22:09,584 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-left_vn-76866856 passed 2024-10-25 15:22:09,584 - DEBUG - ====Verifying policy data for ctest-left_vn-76866856 in API_Server ====== 2024-10-25 15:22:09,584 - DEBUG - Requesting: http://10.0.0.31:8082/domains 2024-10-25 15:22:09,592 - DEBUG - Requesting: http://10.0.0.31:8082/domain/5c9222a5-1004-4c8d-a5ba-6a8cd4423477 2024-10-25 15:22:09,607 - DEBUG - Requesting: http://10.0.0.31:8082/project/944418ee-ac66-4359-804f-c1002088adc7 2024-10-25 15:22:09,702 - DEBUG - Requesting: http://10.0.0.31:8082/virtual-network/38781dff-7c93-4305-a700-448b8101d306 2024-10-25 15:22:09,721 - DEBUG - =>VN ctest-left_vn-76866856 has no policy to be verified 2024-10-25 15:22:09,721 - DEBUG - Verifying the vn in opserver 2024-10-25 15:22:09,722 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-42593455:ctest-left_vn-76866856 virtual network link through opserver 10.0.0.31 2024-10-25 15:22:09,722 - DEBUG - Requesting: http://10.0.0.31:8081/analytics/uves/virtual-networks 2024-10-25 15:22:09,740 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-42593455:ctest-left_vn-76866856', 'href': 'http://10.0.0.31:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-42593455:ctest-left_vn-76866856?flat'} 2024-10-25 15:22:09,740 - INFO - Validated that VN default-domain:ctest-TestSvcRegr-42593455:ctest-left_vn-76866856 is found in opserver 2024-10-25 15:22:09,752 - DEBUG - Do not have enough data to verify VN in agent 2024-10-25 15:22:09,758 - DEBUG - VRF ids for VN ctest-left_vn-76866856: {} 2024-10-25 15:22:09,992 - DEBUG - Response for create_network : {'network': {'id': 'f07e9ccd-9060-4e7b-8f76-09171c3e2aaf', 'name': 'ctest-right_vn-31067898', 'tenant_id': '944418eeac664359804fc1002088adc7', 'project_id': '944418eeac664359804fc1002088adc7', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcRegr-42593455', 'ctest-right_vn-31067898'], 'port_security_enabled': True, 'description': ''}} 2024-10-25 15:22:10,208 - DEBUG - Response for create_subnet : {'subnet': {'id': '6d41a67f-8d40-431b-a2e0-91812ad8662d', 'name': '', 'tenant_id': '944418eeac664359804fc1002088adc7', 'network_id': 'f07e9ccd-9060-4e7b-8f76-09171c3e2aaf', 'ip_version': 4, 'cidr': '176.147.68.64/26', 'allocation_pools': [{'start': '176.147.68.66', 'end': '176.147.68.126'}], 'gateway_ip': '176.147.68.65', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '176.147.68.66', 'tags': [], 'project_id': '944418eeac664359804fc1002088adc7'}} 2024-10-25 15:22:10,232 - INFO - Created VN ctest-right_vn-31067898 2024-10-25 15:22:10,292 - DEBUG - VN ctest-right_vn-31067898 UUID is f07e9ccd-9060-4e7b-8f76-09171c3e2aaf 2024-10-25 15:22:10,426 - DEBUG - Requesting: http://10.0.0.31:8082/domains 2024-10-25 15:22:10,435 - DEBUG - Requesting: http://10.0.0.31:8082/domain/5c9222a5-1004-4c8d-a5ba-6a8cd4423477 2024-10-25 15:22:10,453 - DEBUG - Requesting: http://10.0.0.31:8082/project/944418ee-ac66-4359-804f-c1002088adc7 2024-10-25 15:22:10,540 - DEBUG - Requesting: http://10.0.0.31:8082/virtual-network/f07e9ccd-9060-4e7b-8f76-09171c3e2aaf 2024-10-25 15:22:10,560 - DEBUG - Requesting: http://10.0.0.31:8082/virtual-network/f07e9ccd-9060-4e7b-8f76-09171c3e2aaf 2024-10-25 15:22:10,578 - DEBUG - Requesting: http://10.0.0.31:8082/routing-instance/e755447d-c88a-4b6c-a3af-04c07f13037a 2024-10-25 15:22:10,590 - DEBUG - Requesting: http://10.0.0.31:8082/routing-instance/e755447d-c88a-4b6c-a3af-04c07f13037a 2024-10-25 15:22:10,604 - DEBUG - Requesting: http://10.0.0.31:8082/route-target/c757a6c8-3634-4488-89de-ace213ed9abc 2024-10-25 15:22:10,615 - DEBUG - Route Targets: ['target:64512:8000007'] 2024-10-25 15:22:10,615 - DEBUG - Requesting: http://10.0.0.31:8082/virtual-network/f07e9ccd-9060-4e7b-8f76-09171c3e2aaf 2024-10-25 15:22:10,633 - DEBUG - Requesting: http://10.0.0.31:8082/routing-instance/e755447d-c88a-4b6c-a3af-04c07f13037a 2024-10-25 15:22:10,663 - INFO - Verified VN network id 10 for VN f07e9ccd-9060-4e7b-8f76-09171c3e2aaf 2024-10-25 15:22:10,663 - INFO - Verifications in API Server for VN ctest-right_vn-31067898 passed 2024-10-25 15:22:10,663 - DEBUG - Requesting: http://10.0.0.31:8082/virtual-network/f07e9ccd-9060-4e7b-8f76-09171c3e2aaf 2024-10-25 15:22:10,681 - DEBUG - Requesting: http://10.0.0.31:8082/routing-instance/e755447d-c88a-4b6c-a3af-04c07f13037a 2024-10-25 15:22:10,693 - DEBUG - Requesting: http://10.0.0.31:8082/routing-instance/e755447d-c88a-4b6c-a3af-04c07f13037a 2024-10-25 15:22:10,706 - DEBUG - Requesting: http://10.0.0.31:8082/route-target/c757a6c8-3634-4488-89de-ace213ed9abc 2024-10-25 15:22:10,725 - DEBUG - Control-node 10.0.0.31 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-42593455:ctest-right_vn-31067898', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-42593455', 'owner-access': '7', 'group': 'member', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '17329460822872706683', 'uuid-lslong': '10337459989538286255'}, 'enable': 'true', 'created': '2024-10-25T15:22:09', 'last-modified': '2024-10-25T15:22:10', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.558066'} 2024-10-25 15:22:10,736 - DEBUG - Route Targets: ['target:64512:8000007'] 2024-10-25 15:22:10,737 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-right_vn-31067898 passed 2024-10-25 15:22:10,737 - DEBUG - ====Verifying policy data for ctest-right_vn-31067898 in API_Server ====== 2024-10-25 15:22:10,737 - DEBUG - Requesting: http://10.0.0.31:8082/domains 2024-10-25 15:22:10,748 - DEBUG - Requesting: http://10.0.0.31:8082/domain/5c9222a5-1004-4c8d-a5ba-6a8cd4423477 2024-10-25 15:22:10,771 - DEBUG - Requesting: http://10.0.0.31:8082/project/944418ee-ac66-4359-804f-c1002088adc7 2024-10-25 15:22:10,875 - DEBUG - Requesting: http://10.0.0.31:8082/virtual-network/f07e9ccd-9060-4e7b-8f76-09171c3e2aaf 2024-10-25 15:22:10,894 - DEBUG - =>VN ctest-right_vn-31067898 has no policy to be verified 2024-10-25 15:22:10,894 - DEBUG - Verifying the vn in opserver 2024-10-25 15:22:10,895 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-42593455:ctest-right_vn-31067898 virtual network link through opserver 10.0.0.31 2024-10-25 15:22:10,895 - DEBUG - Requesting: http://10.0.0.31:8081/analytics/uves/virtual-networks 2024-10-25 15:22:10,912 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-42593455:ctest-right_vn-31067898', 'href': 'http://10.0.0.31:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-42593455:ctest-right_vn-31067898?flat'} 2024-10-25 15:22:10,912 - INFO - Validated that VN default-domain:ctest-TestSvcRegr-42593455:ctest-right_vn-31067898 is found in opserver 2024-10-25 15:22:10,927 - DEBUG - Do not have enough data to verify VN in agent 2024-10-25 15:22:10,933 - DEBUG - VRF ids for VN ctest-right_vn-31067898: {} 2024-10-25 15:22:10,997 - DEBUG - Services list from nova: [, , ] 2024-10-25 15:22:12,210 - INFO - VM ([]) created on node: (cn-jenkins-deploy-platform-ansible-os-2145-1), Zone: (nova:cn-jenkins-deploy-platform-ansible-os-2145-1) 2024-10-25 15:22:13,612 - INFO - VM ([]) created on node: (cn-jenkins-deploy-platform-ansible-os-2145-1), Zone: (nova:cn-jenkins-deploy-platform-ansible-os-2145-1) 2024-10-25 15:22:13,613 - DEBUG - Creating service template: ['default-domain', 'ctest-service_template_1-71491873'] 2024-10-25 15:22:14,032 - DEBUG - Service template: ['default-domain', 'ctest-service_template_1-71491873'] created succesfully 2024-10-25 15:22:14,414 - DEBUG - Image by name cirros_in_net not found 2024-10-25 15:22:14,414 - DEBUG - Installing image cirros_in_net 2024-10-25 15:22:14,414 - DEBUG - Image info {'name': 'cirros-in-net.vmdk.gz', 'webserver': '', 'location': '/images/converts/', 'username': 'cirros', 'password': 'gocubsgo', 'type': 'os', 'flavor': 'contrail_flavor_tiny', 'vcpath': '/images/vcenter/', 'params': '--container-format bare --disk-format vmdk --property vmware_disktype="sparse" --property vmware_adaptertype="ide" --property hw_scsi_model=virtio-scsi --property hw_disk_bus_model=virtio-scsi --property hw_disk_bus=virtio', 'name_docker': 'phusion-baseimage-enablesshd'} 2024-10-25 15:22:14,414 - DEBUG - Image build path http://nexus.opensdn.io/repository///images/converts//cirros-in-net.vmdk.gz 2024-10-25 15:22:14,414 - DEBUG - Download image from web http://nexus.opensdn.io/repository///images/converts//cirros-in-net.vmdk.gz 2024-10-25 15:22:14,428 - DEBUG - Image download (local=True) (proxy=None): Execute cmd: wget -nv http://nexus.opensdn.io/repository///images/converts//cirros-in-net.vmdk.gz -O /contrail-test/images/cirros-in-net.vmdk.gz 2024-10-25 15:22:14,774 - DEBUG - Image has been downloaded to /contrail-test/images/cirros-in-net.vmdk.gz 2024-10-25 15:22:14,774 - DEBUG - Unzip image 2024-10-25 15:22:14,774 - DEBUG - Image download (local=True) (proxy=None): Execute cmd: gunzip -f /contrail-test/images/cirros-in-net.vmdk.gz 2024-10-25 15:22:14,985 - DEBUG - Try to create image in user context 2024-10-25 15:22:15,267 - DEBUG - Image has been created in user context 2024-10-25 15:22:15,267 - DEBUG - Image download (local=True) (proxy=None): Execute cmd: rm -f /contrail-test/images/cirros-in-net.vmdk 2024-10-25 15:22:15,283 - DEBUG - Downloaded image has been removed 2024-10-25 15:22:15,305 - DEBUG - Uploaded image status = active 2024-10-25 15:22:16,716 - INFO - VM ([]) created on node: (cn-jenkins-deploy-platform-ansible-os-2145-1), Zone: (nova:cn-jenkins-deploy-platform-ansible-os-2145-1) 2024-10-25 15:22:16,717 - INFO - Waiting for VM ctest-pt_svm0-38514793 to be up.. 2024-10-25 15:22:16,780 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2024-10-25 15:22:21,899 - DEBUG - VM is in ACTIVE state now 2024-10-25 15:22:21,899 - INFO - VM name : ctest-pt_svm0-38514793 2024-10-25 15:22:22,011 - DEBUG - VM ctest-pt_svm0-38514793 ID is 8b5f0a7a-ba0a-40e8-b2ba-e5da970fb155 2024-10-25 15:22:22,035 - DEBUG - VM ctest-pt_svm0-38514793 launched on Node cn-jenkins-deploy-platform-ansible-os-2145-1 2024-10-25 15:22:22,363 - DEBUG - Requesting: http://10.0.0.31:8082/virtual-machine/8b5f0a7a-ba0a-40e8-b2ba-e5da970fb155 2024-10-25 15:22:22,377 - DEBUG - Requesting: http://10.0.0.31:8082/virtual-machine-interface/2179f041-f4b1-45d2-8b9c-1d68716aedaf 2024-10-25 15:22:22,400 - DEBUG - Requesting: http://10.0.0.31:8082/virtual-machine-interface/73ddac22-a8b7-4f30-bf23-44a970f9548a 2024-10-25 15:22:22,420 - DEBUG - Requesting: http://10.0.0.31:8082/virtual-machine-interface/cd995a4b-7526-4aca-8095-d768b925eae3 2024-10-25 15:22:25,769 - DEBUG - (True, 'PING 169.254.0.5 (169.254.0.5) 56(84) bytes of data.\r\n\r\n--- 169.254.0.5 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1005ms') 2024-10-25 15:22:25,769 - DEBUG - Ping to Metadata IP 169.254.0.5 of VM ctest-pt_svm0-38514793 failed! 2024-10-25 15:22:25,836 - DEBUG - Gateway for vn default-domain:ctest-TestSvcRegr-42593455:ctest-mgmt_vn-02141399 is 115.152.213.1 and allocation pool is NOT set 2024-10-25 15:22:29,926 - DEBUG - (True, 'PING 169.254.0.5 (169.254.0.5) 56(84) bytes of data.\r\n\r\n--- 169.254.0.5 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1025ms') 2024-10-25 15:22:29,926 - DEBUG - Ping to Metadata IP 169.254.0.5 of VM ctest-pt_svm0-38514793 failed! 2024-10-25 15:22:29,992 - DEBUG - Gateway for vn default-domain:ctest-TestSvcRegr-42593455:ctest-mgmt_vn-02141399 is 115.152.213.1 and allocation pool is NOT set 2024-10-25 15:22:34,054 - DEBUG - (True, 'PING 169.254.0.5 (169.254.0.5) 56(84) bytes of data.\r\n\r\n--- 169.254.0.5 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1002ms') 2024-10-25 15:22:34,055 - DEBUG - Ping to Metadata IP 169.254.0.5 of VM ctest-pt_svm0-38514793 failed! 2024-10-25 15:22:34,124 - DEBUG - Gateway for vn default-domain:ctest-TestSvcRegr-42593455:ctest-mgmt_vn-02141399 is 115.152.213.1 and allocation pool is NOT set 2024-10-25 15:22:38,210 - DEBUG - (True, 'PING 169.254.0.5 (169.254.0.5) 56(84) bytes of data.\r\n\r\n--- 169.254.0.5 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1026ms') 2024-10-25 15:22:38,210 - DEBUG - Ping to Metadata IP 169.254.0.5 of VM ctest-pt_svm0-38514793 failed! 2024-10-25 15:22:38,276 - DEBUG - Gateway for vn default-domain:ctest-TestSvcRegr-42593455:ctest-mgmt_vn-02141399 is 115.152.213.1 and allocation pool is NOT set 2024-10-25 15:22:40,343 - DEBUG - (True, 'PING 169.254.0.5 (169.254.0.5) 56(84) bytes of data.\r\n64 bytes from 169.254.0.5: icmp_seq=1 ttl=63 time=30.2 ms\r\n64 bytes from 169.254.0.5: icmp_seq=2 ttl=63 time=0.995 ms\r\n\r\n--- 169.254.0.5 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 0.995/15.607/30.219/14.612 ms') 2024-10-25 15:22:40,343 - INFO - Ping to Metadata IP 169.254.0.5 of VM ctest-pt_svm0-38514793 passed 2024-10-25 15:22:40,504 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2024-10-25 15:22:40,504 - DEBUG - Waiting to SSH to VM ctest-pt_svm0-38514793, IP 115.152.213.3, Port 22 2024-10-25 15:22:40,570 - DEBUG - Error on ssh to cirros@169.254.0.5:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.5/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.5/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.5/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2024-10-25 15:22:40,680 - DEBUG - VM ctest-pt_svm0-38514793 is NOT ready for SSH connections, VM status: ACTIVE 2024-10-25 15:22:45,681 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2024-10-25 15:22:45,682 - DEBUG - Waiting to SSH to VM ctest-pt_svm0-38514793, IP 115.152.213.3, Port 22 2024-10-25 15:22:45,740 - DEBUG - Error on ssh to cirros@169.254.0.5:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.5/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.5/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.5/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2024-10-25 15:22:45,839 - DEBUG - VM ctest-pt_svm0-38514793 is NOT ready for SSH connections, VM status: ACTIVE 2024-10-25 15:22:50,840 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2024-10-25 15:22:50,840 - DEBUG - Waiting to SSH to VM ctest-pt_svm0-38514793, IP 115.152.213.3, Port 22 2024-10-25 15:22:50,906 - DEBUG - Error on ssh to cirros@169.254.0.5:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.5/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.5/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.5/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2024-10-25 15:22:51,044 - DEBUG - VM ctest-pt_svm0-38514793 is NOT ready for SSH connections, VM status: ACTIVE 2024-10-25 15:22:56,044 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2024-10-25 15:22:56,045 - DEBUG - Waiting to SSH to VM ctest-pt_svm0-38514793, IP 115.152.213.3, Port 22 2024-10-25 15:22:56,101 - DEBUG - Error on ssh to cirros@169.254.0.5:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.5/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.5/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.5/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2024-10-25 15:22:56,208 - DEBUG - VM ctest-pt_svm0-38514793 is NOT ready for SSH connections, VM status: ACTIVE 2024-10-25 15:23:01,209 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2024-10-25 15:23:01,209 - DEBUG - Waiting to SSH to VM ctest-pt_svm0-38514793, IP 115.152.213.3, Port 22 2024-10-25 15:23:01,379 - DEBUG - VM ctest-pt_svm0-38514793 is ready for SSH connections 2024-10-25 15:23:01,548 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2024-10-25 15:23:01,548 - DEBUG - Waiting to SSH to VM ctest-pt_svm0-38514793, IP 115.152.213.3, Port 22 2024-10-25 15:23:01,714 - DEBUG - VM ctest-pt_svm0-38514793 is ready for SSH connections 2024-10-25 15:23:01,876 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2024-10-25 15:23:01,876 - DEBUG - Waiting to SSH to VM ctest-pt_svm0-38514793, IP 115.152.213.3, Port 22 2024-10-25 15:23:02,042 - DEBUG - VM ctest-pt_svm0-38514793 is ready for SSH connections 2024-10-25 15:23:02,049 - DEBUG - Creating service instance: ['default-domain', 'ctest-TestSvcRegr-42593455', 'ctest-si-98986743'] 2024-10-25 15:23:03,020 - INFO - Waiting for VM ctest-left_vm-25705975 to be up.. 2024-10-25 15:23:03,136 - DEBUG - VM is in ACTIVE state now 2024-10-25 15:23:03,136 - INFO - VM name : ctest-left_vm-25705975 2024-10-25 15:23:03,242 - DEBUG - VM ctest-left_vm-25705975 ID is 394db34a-6013-42dd-a147-26bb64093675 2024-10-25 15:23:03,242 - DEBUG - VM ctest-left_vm-25705975 launched on Node cn-jenkins-deploy-platform-ansible-os-2145-1 2024-10-25 15:23:03,343 - DEBUG - Requesting: http://10.0.0.31:8082/virtual-machine/394db34a-6013-42dd-a147-26bb64093675 2024-10-25 15:23:03,355 - DEBUG - Requesting: http://10.0.0.31:8082/virtual-machine-interface/98b3cf04-a1d4-439e-a990-892240b7cccb 2024-10-25 15:23:04,619 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=3.49 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=0.506 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 0.506/2.000/3.494/1.494 ms') 2024-10-25 15:23:04,619 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-left_vm-25705975 passed 2024-10-25 15:23:04,780 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2024-10-25 15:23:04,780 - DEBUG - Waiting to SSH to VM ctest-left_vm-25705975, IP 171.145.10.195, Port 22 2024-10-25 15:23:04,935 - DEBUG - VM ctest-left_vm-25705975 is ready for SSH connections 2024-10-25 15:23:04,936 - INFO - Waiting for VM ctest-right_vm-05280760 to be up.. 2024-10-25 15:23:05,041 - DEBUG - VM is in ACTIVE state now 2024-10-25 15:23:05,041 - INFO - VM name : ctest-right_vm-05280760 2024-10-25 15:23:05,152 - DEBUG - VM ctest-right_vm-05280760 ID is 7e59f171-c58d-4373-a6ab-c8e5a5692419 2024-10-25 15:23:05,153 - DEBUG - VM ctest-right_vm-05280760 launched on Node cn-jenkins-deploy-platform-ansible-os-2145-1 2024-10-25 15:23:05,254 - DEBUG - Requesting: http://10.0.0.31:8082/virtual-machine/7e59f171-c58d-4373-a6ab-c8e5a5692419 2024-10-25 15:23:05,266 - DEBUG - Requesting: http://10.0.0.31:8082/virtual-machine-interface/31eec347-4a11-43ed-b0de-414bfc0d8423 2024-10-25 15:23:06,542 - DEBUG - (True, 'PING 169.254.0.4 (169.254.0.4) 56(84) bytes of data.\r\n64 bytes from 169.254.0.4: icmp_seq=1 ttl=63 time=3.25 ms\r\n64 bytes from 169.254.0.4: icmp_seq=2 ttl=63 time=0.570 ms\r\n\r\n--- 169.254.0.4 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 0.570/1.910/3.251/1.340 ms') 2024-10-25 15:23:06,542 - INFO - Ping to Metadata IP 169.254.0.4 of VM ctest-right_vm-05280760 passed 2024-10-25 15:23:06,716 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2024-10-25 15:23:06,716 - DEBUG - Waiting to SSH to VM ctest-right_vm-05280760, IP 176.147.68.67, Port 22 2024-10-25 15:23:06,872 - DEBUG - VM ctest-right_vm-05280760 is ready for SSH connections 2024-10-25 15:23:06,878 - DEBUG - Policy np_rules : [rule_sequence = None, rule_uuid = None, direction = <>, protocol = any, src_addresses = [subnet = None, virtual_network = default-domain:ctest-TestSvcRegr-42593455:ctest-left_vn-76866856, security_group = None, network_policy = None, subnet_list = []], src_ports = [start_port = 0, end_port = 65535], application = [], dst_addresses = [subnet = None, virtual_network = default-domain:ctest-TestSvcRegr-42593455:ctest-right_vn-31067898, security_group = None, network_policy = None, subnet_list = []], dst_ports = [start_port = 0, end_port = 65535], action_list = simple_action = pass, gateway_name = None, apply_service = ['default-domain:ctest-TestSvcRegr-42593455:ctest-si-98986743'], service_properties = None, mirror_to = None, assign_routing_instance = None, log = False, alert = False, qos_action = None, host_based_service = False, ethertype = None, created = None, last_modified = None, description = None] 2024-10-25 15:23:07,392 - INFO - Attaching policy ctest-policy-93545486 to vn ctest-left_vn-76866856 2024-10-25 15:23:07,948 - INFO - Attaching policy ctest-policy-93545486 to vn ctest-right_vn-31067898 2024-10-25 15:23:08,444 - DEBUG - Service template: ['default-domain', 'ctest-service_template_1-71491873'] created succesfully 2024-10-25 15:23:08,684 - DEBUG - Service instance: ['default-domain', 'ctest-TestSvcRegr-42593455', 'ctest-si-98986743'] created succesfully 2024-10-25 15:23:08,684 - DEBUG - Requesting: http://10.0.0.31:8082/domains 2024-10-25 15:23:08,692 - DEBUG - Requesting: http://10.0.0.31:8082/domain/5c9222a5-1004-4c8d-a5ba-6a8cd4423477 2024-10-25 15:23:08,710 - DEBUG - Requesting: http://10.0.0.31:8082/project/944418ee-ac66-4359-804f-c1002088adc7 2024-10-25 15:23:08,797 - DEBUG - Requesting: http://10.0.0.31:8082/service-instance/dacf46d7-d471-479b-999d-365e7cf7a0f1 2024-10-25 15:23:08,812 - DEBUG - SI 'ctest-si-98986743' has service template ref to ctest-service_template_1-71491873 2024-10-25 15:23:08,812 - DEBUG - Requesting: http://10.0.0.31:8082/domains 2024-10-25 15:23:08,819 - DEBUG - Requesting: http://10.0.0.31:8082/domain/5c9222a5-1004-4c8d-a5ba-6a8cd4423477 2024-10-25 15:23:08,834 - DEBUG - Requesting: http://10.0.0.31:8082/project/944418ee-ac66-4359-804f-c1002088adc7 2024-10-25 15:23:08,922 - DEBUG - Requesting: http://10.0.0.31:8082/service-instance/dacf46d7-d471-479b-999d-365e7cf7a0f1 2024-10-25 15:23:08,933 - DEBUG - Requesting: http://10.0.0.31:8082/port-tuple/4b14056d-c984-42d0-a587-5b19d6177286 2024-10-25 15:23:08,943 - DEBUG - Requesting: http://10.0.0.31:8082/virtual-machine-interface/2179f041-f4b1-45d2-8b9c-1d68716aedaf 2024-10-25 15:23:08,960 - DEBUG - Requesting: http://10.0.0.31:8082/virtual-machine-interface/73ddac22-a8b7-4f30-bf23-44a970f9548a 2024-10-25 15:23:08,977 - DEBUG - Requesting: http://10.0.0.31:8082/virtual-machine-interface/cd995a4b-7526-4aca-8095-d768b925eae3 2024-10-25 15:23:09,196 - DEBUG - VM is in ACTIVE state now 2024-10-25 15:23:09,672 - DEBUG - Requesting: http://10.0.0.31:8082/virtual-machine/8b5f0a7a-ba0a-40e8-b2ba-e5da970fb155 2024-10-25 15:23:09,687 - DEBUG - Requesting: http://10.0.0.31:8082/virtual-machine-interface/2179f041-f4b1-45d2-8b9c-1d68716aedaf 2024-10-25 15:23:09,707 - DEBUG - Requesting: http://10.0.0.31:8082/virtual-machine-interface/73ddac22-a8b7-4f30-bf23-44a970f9548a 2024-10-25 15:23:09,725 - DEBUG - Requesting: http://10.0.0.31:8082/virtual-machine-interface/cd995a4b-7526-4aca-8095-d768b925eae3 2024-10-25 15:23:09,744 - DEBUG - Requesting: http://10.0.0.31:8082/instance-ip/9b36fc02-8bc6-4abf-a8f0-08956a6cbb1e 2024-10-25 15:23:09,756 - DEBUG - Requesting: http://10.0.0.31:8082/instance-ip/1da79ade-3e4d-4d8d-b761-d524d026a356 2024-10-25 15:23:09,768 - DEBUG - Requesting: http://10.0.0.31:8082/instance-ip/96878f7c-7a49-4b5d-a5cb-f0985512a4ab 2024-10-25 15:23:10,225 - DEBUG - VM is in ACTIVE state now 2024-10-25 15:23:10,225 - DEBUG - Requesting: http://10.0.0.31:8082/virtual-machine/8b5f0a7a-ba0a-40e8-b2ba-e5da970fb155 2024-10-25 15:23:10,235 - DEBUG - Requesting: http://10.0.0.31:8082/virtual-machine-interface/2179f041-f4b1-45d2-8b9c-1d68716aedaf 2024-10-25 15:23:10,253 - DEBUG - Requesting: http://10.0.0.31:8082/virtual-machine-interface/73ddac22-a8b7-4f30-bf23-44a970f9548a 2024-10-25 15:23:10,271 - DEBUG - Requesting: http://10.0.0.31:8082/virtual-machine-interface/cd995a4b-7526-4aca-8095-d768b925eae3 2024-10-25 15:23:11,536 - DEBUG - (True, 'PING 169.254.0.6 (169.254.0.6) 56(84) bytes of data.\r\n64 bytes from 169.254.0.6: icmp_seq=1 ttl=63 time=1.23 ms\r\n64 bytes from 169.254.0.6: icmp_seq=2 ttl=63 time=0.612 ms\r\n\r\n--- 169.254.0.6 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 0.612/0.923/1.234/0.311 ms') 2024-10-25 15:23:11,536 - INFO - Ping to Metadata IP 169.254.0.6 of VM ctest-pt_svm0-38514793 passed 2024-10-25 15:23:11,583 - DEBUG - VM ctest-pt_svm0-38514793 Tap interface: {'index': '6', 'name': 'tap2179f041-f4', 'uuid': '2179f041-f4b1-45d2-8b9c-1d68716aedaf', 'vrf_name': 'default-domain:ctest-TestSvcRegr-42593455:ctest-left_vn-76866856:ctest-left_vn-76866856', 'active': 'Active', 'ipv4_active': 'Active', 'l2_active': 'L2 Active', 'ip6_active': 'Ipv6 Inactive < no-ipv6-addr >', 'health_check_active': 'Active', 'dhcp_service': 'Enable', 'dhcp_service_v6': 'Disable', 'dns_service': 'Enable', 'type': 'vport', 'label': '44', 'l2_label': '48', 'vxlan_id': '9', 'vn_name': 'default-domain:ctest-TestSvcRegr-42593455:ctest-left_vn-76866856', 'vm_uuid': '8b5f0a7a-ba0a-40e8-b2ba-e5da970fb155', 'vm_name': 'ctest-pt_svm0-38514793', 'ip_addr': '171.145.10.196', 'mac_addr': '02:21:79:f0:41:f4', 'policy': 'Enable', 'fip_list': [], 'mdata_ip_addr': '169.254.0.6', 'service_vlan_list': None, 'os_ifindex': '27', 'fabric_port': 'NotFabricPort', 'alloc_linklocal_ip': 'LL-Enable', 'analyzer_name': None, 'config_name': 'default-domain:ctest-TestSvcRegr-42593455:2179f041-f4b1-45d2-8b9c-1d68716aedaf', 'sg_uuid_list': ['badea825-0520-4ced-a7f5-6704510394d6'], 'static_route_list': None, 'vm_project_uuid': '944418ee-ac66-4359-804f-c1002088adc7', 'admin_state': 'Enabled', 'flow_key_idx': '42', 'allowed_address_pair_list': None, 'ip6_addr': '::', 'local_preference': '0', 'tx_vlan_id': '-1', 'rx_vlan_id': '-1', 'parent_interface': None, 'subnet': '--NA--', 'sub_type': 'Tap', 'vrf_assign_acl_uuid': '--NA--', 'vmi_type': 'Virtual Machine', 'transport': 'Ethernet', 'logical_interface_uuid': '00000000-0000-0000-0000-000000000000', 'flood_unknown_unicast': 'false', 'physical_device': None, 'physical_interface': None, 'fixed_ip4_list': None, 'fixed_ip6_list': None, 'fat_flow_list': None, 'metadata_ip_active': 'Active', 'service_health_check_ip': '0.0.0.0', 'alias_ip_list': None, 'drop_new_flows': 'false', 'bridge_domain_list': [], 'vmi_tag_list': None, 'policy_set_acl_list': None, 'slo_list': None, 'vhostuser_mode': '0', 'si_other_end_vmi': 'cd995a4b-7526-4aca-8095-d768b925eae3', 'cfg_igmp_enable': 'false', 'igmp_enabled': 'false', 'max_flows': '0', 'policy_set_fwaas_list': None, 'bond_interface_list': None, 'mac_ip_learning_enable': 'false', 'mac_ip_list': []} 2024-10-25 15:23:11,583 - DEBUG - Agent 10.0.0.31 vrf name: default-domain:ctest-TestSvcRegr-42593455:ctest-left_vn-76866856:ctest-left_vn-76866856 2024-10-25 15:23:11,590 - DEBUG - Agent VRF Object : {'vrf_list': [{'name': 'default-domain:ctest-TestSvcRegr-42593455:ctest-left_vn-76866856:ctest-left_vn-76866856', 'ucindex': '2', 'mcindex': '2', 'l2index': '2', 'source': 'Config; ', 'uc6index': '2', 'vn': 'default-domain:ctest-TestSvcRegr-42593455:ctest-left_vn-76866856', 'table_label': '-1', 'vxlan_id': '9', 'evpnindex': '2', 'brindex': '2', 'mplsindex': '0', 'RD': '10.20.0.14:2', 'mac_aging_time': '0', 'layer2_control_word': 'false', 'forwarding_vrf': None, 'hbf_rintf': '-1', 'hbf_lintf': '-1'}]} 2024-10-25 15:23:11,762 - DEBUG - VM ctest-pt_svm0-38514793 : Tap interface tap2179f041-f4 is set to Active 2024-10-25 15:23:11,762 - DEBUG - Tap interface tap2179f041-f4 detail : {'index': '6', 'name': 'tap2179f041-f4', 'uuid': '2179f041-f4b1-45d2-8b9c-1d68716aedaf', 'vrf_name': 'default-domain:ctest-TestSvcRegr-42593455:ctest-left_vn-76866856:ctest-left_vn-76866856', 'active': 'Active', 'ipv4_active': 'Active', 'l2_active': 'L2 Active', 'ip6_active': 'Ipv6 Inactive < no-ipv6-addr >', 'health_check_active': 'Active', 'dhcp_service': 'Enable', 'dhcp_service_v6': 'Disable', 'dns_service': 'Enable', 'type': 'vport', 'label': '44', 'l2_label': '48', 'vxlan_id': '9', 'vn_name': 'default-domain:ctest-TestSvcRegr-42593455:ctest-left_vn-76866856', 'vm_uuid': '8b5f0a7a-ba0a-40e8-b2ba-e5da970fb155', 'vm_name': 'ctest-pt_svm0-38514793', 'ip_addr': '171.145.10.196', 'mac_addr': '02:21:79:f0:41:f4', 'policy': 'Enable', 'fip_list': [], 'mdata_ip_addr': '169.254.0.6', 'service_vlan_list': None, 'os_ifindex': '27', 'fabric_port': 'NotFabricPort', 'alloc_linklocal_ip': 'LL-Enable', 'analyzer_name': None, 'config_name': 'default-domain:ctest-TestSvcRegr-42593455:2179f041-f4b1-45d2-8b9c-1d68716aedaf', 'sg_uuid_list': ['badea825-0520-4ced-a7f5-6704510394d6'], 'static_route_list': None, 'vm_project_uuid': '944418ee-ac66-4359-804f-c1002088adc7', 'admin_state': 'Enabled', 'flow_key_idx': '42', 'allowed_address_pair_list': None, 'ip6_addr': '::', 'local_preference': '0', 'tx_vlan_id': '-1', 'rx_vlan_id': '-1', 'parent_interface': None, 'subnet': '--NA--', 'sub_type': 'Tap', 'vrf_assign_acl_uuid': '--NA--', 'vmi_type': 'Virtual Machine', 'transport': 'Ethernet', 'logical_interface_uuid': '00000000-0000-0000-0000-000000000000', 'flood_unknown_unicast': 'false', 'physical_device': None, 'physical_interface': None, 'fixed_ip4_list': None, 'fixed_ip6_list': None, 'fat_flow_list': None, 'metadata_ip_active': 'Active', 'service_health_check_ip': '0.0.0.0', 'alias_ip_list': None, 'drop_new_flows': 'false', 'bridge_domain_list': [], 'vmi_tag_list': None, 'policy_set_acl_list': None, 'slo_list': None, 'vhostuser_mode': '0', 'si_other_end_vmi': 'cd995a4b-7526-4aca-8095-d768b925eae3', 'cfg_igmp_enable': 'false', 'igmp_enabled': 'false', 'max_flows': '0', 'policy_set_fwaas_list': None, 'bond_interface_list': None, 'mac_ip_learning_enable': 'false', 'mac_ip_list': []} 2024-10-25 15:23:11,968 - DEBUG - Starting Layer 2 verification in Agent 2024-10-25 15:23:11,983 - DEBUG - Layer 2 path is seen for VM MAC 02:21:79:f0:41:f4 in agent 10.0.0.31 2024-10-25 15:23:11,983 - DEBUG - Active layer 2 route in agent is present for VMI tap2179f041-f4 2024-10-25 15:23:11,983 - DEBUG - L2 label(48) matches bw route table and interface table 2024-10-25 15:23:11,990 - DEBUG - VN default-domain:ctest-TestSvcRegr-42593455:ctest-left_vn-76866856 is found in Agent of node 10.0.0.31 2024-10-25 15:23:12,212 - DEBUG - VRF IDs of VN default-domain:ctest-TestSvcRegr-42593455:ctest-left_vn-76866856 is consistent in agent 10.0.0.31 2024-10-25 15:23:12,212 - DEBUG - Route for VM IP ['171.145.10.196'] is consistent in agent 10.0.0.31 2024-10-25 15:23:12,212 - DEBUG - VN default-domain:ctest-TestSvcRegr-42593455:ctest-left_vn-76866856 verification for VM ctest-pt_svm0-38514793 in Agent 10.0.0.31 passed 2024-10-25 15:23:12,416 - DEBUG - Starting all layer 2 verification in agent 10.0.0.31 2024-10-25 15:23:12,431 - DEBUG - Route for VM MAC 02:21:79:f0:41:f4 is consistent in agent 10.0.0.31 2024-10-25 15:23:12,457 - DEBUG - VM ctest-pt_svm0-38514793 Tap interface: {'index': '5', 'name': 'tap73ddac22-a8', 'uuid': '73ddac22-a8b7-4f30-bf23-44a970f9548a', 'vrf_name': 'default-domain:ctest-TestSvcRegr-42593455:ctest-mgmt_vn-02141399:ctest-mgmt_vn-02141399', 'active': 'Active', 'ipv4_active': 'Active', 'l2_active': 'L2 Active', 'ip6_active': 'Ipv6 Inactive < no-ipv6-addr >', 'health_check_active': 'Active', 'dhcp_service': 'Enable', 'dhcp_service_v6': 'Disable', 'dns_service': 'Enable', 'type': 'vport', 'label': '39', 'l2_label': '43', 'vxlan_id': '8', 'vn_name': 'default-domain:ctest-TestSvcRegr-42593455:ctest-mgmt_vn-02141399', 'vm_uuid': '8b5f0a7a-ba0a-40e8-b2ba-e5da970fb155', 'vm_name': 'ctest-pt_svm0-38514793', 'ip_addr': '115.152.213.3', 'mac_addr': '02:73:dd:ac:22:a8', 'policy': 'Enable', 'fip_list': [], 'mdata_ip_addr': '169.254.0.5', 'service_vlan_list': None, 'os_ifindex': '26', 'fabric_port': 'NotFabricPort', 'alloc_linklocal_ip': 'LL-Enable', 'analyzer_name': None, 'config_name': 'default-domain:ctest-TestSvcRegr-42593455:73ddac22-a8b7-4f30-bf23-44a970f9548a', 'sg_uuid_list': ['badea825-0520-4ced-a7f5-6704510394d6'], 'static_route_list': None, 'vm_project_uuid': '944418ee-ac66-4359-804f-c1002088adc7', 'admin_state': 'Enabled', 'flow_key_idx': '43', 'allowed_address_pair_list': None, 'ip6_addr': '::', 'local_preference': '0', 'tx_vlan_id': '-1', 'rx_vlan_id': '-1', 'parent_interface': None, 'subnet': '--NA--', 'sub_type': 'Tap', 'vrf_assign_acl_uuid': '--NA--', 'vmi_type': 'Virtual Machine', 'transport': 'Ethernet', 'logical_interface_uuid': '00000000-0000-0000-0000-000000000000', 'flood_unknown_unicast': 'false', 'physical_device': None, 'physical_interface': None, 'fixed_ip4_list': None, 'fixed_ip6_list': None, 'fat_flow_list': None, 'metadata_ip_active': 'Active', 'service_health_check_ip': '0.0.0.0', 'alias_ip_list': None, 'drop_new_flows': 'false', 'bridge_domain_list': [], 'vmi_tag_list': None, 'policy_set_acl_list': None, 'slo_list': None, 'vhostuser_mode': '0', 'si_other_end_vmi': '00000000-0000-0000-0000-000000000000', 'cfg_igmp_enable': 'false', 'igmp_enabled': 'false', 'max_flows': '0', 'policy_set_fwaas_list': None, 'bond_interface_list': None, 'mac_ip_learning_enable': 'false', 'mac_ip_list': []} 2024-10-25 15:23:12,457 - DEBUG - Agent 10.0.0.31 vrf name: default-domain:ctest-TestSvcRegr-42593455:ctest-mgmt_vn-02141399:ctest-mgmt_vn-02141399 2024-10-25 15:23:12,464 - DEBUG - Agent VRF Object : {'vrf_list': [{'name': 'default-domain:ctest-TestSvcRegr-42593455:ctest-mgmt_vn-02141399:ctest-mgmt_vn-02141399', 'ucindex': '4', 'mcindex': '4', 'l2index': '4', 'source': 'Config; ', 'uc6index': '4', 'vn': 'default-domain:ctest-TestSvcRegr-42593455:ctest-mgmt_vn-02141399', 'table_label': '-1', 'vxlan_id': '8', 'evpnindex': '4', 'brindex': '4', 'mplsindex': '0', 'RD': '10.20.0.14:4', 'mac_aging_time': '0', 'layer2_control_word': 'false', 'forwarding_vrf': None, 'hbf_rintf': '-1', 'hbf_lintf': '-1'}]} 2024-10-25 15:23:12,636 - DEBUG - VM ctest-pt_svm0-38514793 : Tap interface tap73ddac22-a8 is set to Active 2024-10-25 15:23:12,636 - DEBUG - Tap interface tap73ddac22-a8 detail : {'index': '5', 'name': 'tap73ddac22-a8', 'uuid': '73ddac22-a8b7-4f30-bf23-44a970f9548a', 'vrf_name': 'default-domain:ctest-TestSvcRegr-42593455:ctest-mgmt_vn-02141399:ctest-mgmt_vn-02141399', 'active': 'Active', 'ipv4_active': 'Active', 'l2_active': 'L2 Active', 'ip6_active': 'Ipv6 Inactive < no-ipv6-addr >', 'health_check_active': 'Active', 'dhcp_service': 'Enable', 'dhcp_service_v6': 'Disable', 'dns_service': 'Enable', 'type': 'vport', 'label': '39', 'l2_label': '43', 'vxlan_id': '8', 'vn_name': 'default-domain:ctest-TestSvcRegr-42593455:ctest-mgmt_vn-02141399', 'vm_uuid': '8b5f0a7a-ba0a-40e8-b2ba-e5da970fb155', 'vm_name': 'ctest-pt_svm0-38514793', 'ip_addr': '115.152.213.3', 'mac_addr': '02:73:dd:ac:22:a8', 'policy': 'Enable', 'fip_list': [], 'mdata_ip_addr': '169.254.0.5', 'service_vlan_list': None, 'os_ifindex': '26', 'fabric_port': 'NotFabricPort', 'alloc_linklocal_ip': 'LL-Enable', 'analyzer_name': None, 'config_name': 'default-domain:ctest-TestSvcRegr-42593455:73ddac22-a8b7-4f30-bf23-44a970f9548a', 'sg_uuid_list': ['badea825-0520-4ced-a7f5-6704510394d6'], 'static_route_list': None, 'vm_project_uuid': '944418ee-ac66-4359-804f-c1002088adc7', 'admin_state': 'Enabled', 'flow_key_idx': '43', 'allowed_address_pair_list': None, 'ip6_addr': '::', 'local_preference': '0', 'tx_vlan_id': '-1', 'rx_vlan_id': '-1', 'parent_interface': None, 'subnet': '--NA--', 'sub_type': 'Tap', 'vrf_assign_acl_uuid': '--NA--', 'vmi_type': 'Virtual Machine', 'transport': 'Ethernet', 'logical_interface_uuid': '00000000-0000-0000-0000-000000000000', 'flood_unknown_unicast': 'false', 'physical_device': None, 'physical_interface': None, 'fixed_ip4_list': None, 'fixed_ip6_list': None, 'fat_flow_list': None, 'metadata_ip_active': 'Active', 'service_health_check_ip': '0.0.0.0', 'alias_ip_list': None, 'drop_new_flows': 'false', 'bridge_domain_list': [], 'vmi_tag_list': None, 'policy_set_acl_list': None, 'slo_list': None, 'vhostuser_mode': '0', 'si_other_end_vmi': '00000000-0000-0000-0000-000000000000', 'cfg_igmp_enable': 'false', 'igmp_enabled': 'false', 'max_flows': '0', 'policy_set_fwaas_list': None, 'bond_interface_list': None, 'mac_ip_learning_enable': 'false', 'mac_ip_list': []} 2024-10-25 15:23:12,844 - DEBUG - Starting Layer 2 verification in Agent 2024-10-25 15:23:12,856 - DEBUG - Layer 2 path is seen for VM MAC 02:73:dd:ac:22:a8 in agent 10.0.0.31 2024-10-25 15:23:12,856 - DEBUG - Active layer 2 route in agent is present for VMI tap73ddac22-a8 2024-10-25 15:23:12,857 - DEBUG - L2 label(43) matches bw route table and interface table 2024-10-25 15:23:12,863 - DEBUG - VN default-domain:ctest-TestSvcRegr-42593455:ctest-mgmt_vn-02141399 is found in Agent of node 10.0.0.31 2024-10-25 15:23:13,091 - DEBUG - VRF IDs of VN default-domain:ctest-TestSvcRegr-42593455:ctest-mgmt_vn-02141399 is consistent in agent 10.0.0.31 2024-10-25 15:23:13,092 - DEBUG - Route for VM IP ['115.152.213.3'] is consistent in agent 10.0.0.31 2024-10-25 15:23:13,092 - DEBUG - VN default-domain:ctest-TestSvcRegr-42593455:ctest-mgmt_vn-02141399 verification for VM ctest-pt_svm0-38514793 in Agent 10.0.0.31 passed 2024-10-25 15:23:13,296 - DEBUG - Starting all layer 2 verification in agent 10.0.0.31 2024-10-25 15:23:13,310 - DEBUG - Route for VM MAC 02:73:dd:ac:22:a8 is consistent in agent 10.0.0.31 2024-10-25 15:23:13,338 - DEBUG - VM ctest-pt_svm0-38514793 Tap interface: {'index': '7', 'name': 'tapcd995a4b-75', 'uuid': 'cd995a4b-7526-4aca-8095-d768b925eae3', 'vrf_name': 'default-domain:ctest-TestSvcRegr-42593455:ctest-right_vn-31067898:ctest-right_vn-31067898', 'active': 'Active', 'ipv4_active': 'Active', 'l2_active': 'L2 Active', 'ip6_active': 'Ipv6 Inactive < no-ipv6-addr >', 'health_check_active': 'Active', 'dhcp_service': 'Enable', 'dhcp_service_v6': 'Disable', 'dns_service': 'Enable', 'type': 'vport', 'label': '49', 'l2_label': '53', 'vxlan_id': '10', 'vn_name': 'default-domain:ctest-TestSvcRegr-42593455:ctest-right_vn-31067898', 'vm_uuid': '8b5f0a7a-ba0a-40e8-b2ba-e5da970fb155', 'vm_name': 'ctest-pt_svm0-38514793', 'ip_addr': '176.147.68.68', 'mac_addr': '02:cd:99:5a:4b:75', 'policy': 'Enable', 'fip_list': [], 'mdata_ip_addr': '169.254.0.7', 'service_vlan_list': None, 'os_ifindex': '28', 'fabric_port': 'NotFabricPort', 'alloc_linklocal_ip': 'LL-Enable', 'analyzer_name': None, 'config_name': 'default-domain:ctest-TestSvcRegr-42593455:cd995a4b-7526-4aca-8095-d768b925eae3', 'sg_uuid_list': ['badea825-0520-4ced-a7f5-6704510394d6'], 'static_route_list': None, 'vm_project_uuid': '944418ee-ac66-4359-804f-c1002088adc7', 'admin_state': 'Enabled', 'flow_key_idx': '28', 'allowed_address_pair_list': None, 'ip6_addr': '::', 'local_preference': '0', 'tx_vlan_id': '-1', 'rx_vlan_id': '-1', 'parent_interface': None, 'subnet': '--NA--', 'sub_type': 'Tap', 'vrf_assign_acl_uuid': '--NA--', 'vmi_type': 'Virtual Machine', 'transport': 'Ethernet', 'logical_interface_uuid': '00000000-0000-0000-0000-000000000000', 'flood_unknown_unicast': 'false', 'physical_device': None, 'physical_interface': None, 'fixed_ip4_list': None, 'fixed_ip6_list': None, 'fat_flow_list': None, 'metadata_ip_active': 'Active', 'service_health_check_ip': '0.0.0.0', 'alias_ip_list': None, 'drop_new_flows': 'false', 'bridge_domain_list': [], 'vmi_tag_list': None, 'policy_set_acl_list': None, 'slo_list': None, 'vhostuser_mode': '0', 'si_other_end_vmi': '2179f041-f4b1-45d2-8b9c-1d68716aedaf', 'cfg_igmp_enable': 'false', 'igmp_enabled': 'false', 'max_flows': '0', 'policy_set_fwaas_list': None, 'bond_interface_list': None, 'mac_ip_learning_enable': 'false', 'mac_ip_list': []} 2024-10-25 15:23:13,338 - DEBUG - Agent 10.0.0.31 vrf name: default-domain:ctest-TestSvcRegr-42593455:ctest-right_vn-31067898:ctest-right_vn-31067898 2024-10-25 15:23:13,346 - DEBUG - Agent VRF Object : {'vrf_list': [{'name': 'default-domain:ctest-TestSvcRegr-42593455:ctest-right_vn-31067898:ctest-right_vn-31067898', 'ucindex': '3', 'mcindex': '3', 'l2index': '3', 'source': 'Config; ', 'uc6index': '3', 'vn': 'default-domain:ctest-TestSvcRegr-42593455:ctest-right_vn-31067898', 'table_label': '-1', 'vxlan_id': '10', 'evpnindex': '3', 'brindex': '3', 'mplsindex': '0', 'RD': '10.20.0.14:3', 'mac_aging_time': '0', 'layer2_control_word': 'false', 'forwarding_vrf': None, 'hbf_rintf': '-1', 'hbf_lintf': '-1'}]} 2024-10-25 15:23:13,521 - DEBUG - VM ctest-pt_svm0-38514793 : Tap interface tapcd995a4b-75 is set to Active 2024-10-25 15:23:13,521 - DEBUG - Tap interface tapcd995a4b-75 detail : {'index': '7', 'name': 'tapcd995a4b-75', 'uuid': 'cd995a4b-7526-4aca-8095-d768b925eae3', 'vrf_name': 'default-domain:ctest-TestSvcRegr-42593455:ctest-right_vn-31067898:ctest-right_vn-31067898', 'active': 'Active', 'ipv4_active': 'Active', 'l2_active': 'L2 Active', 'ip6_active': 'Ipv6 Inactive < no-ipv6-addr >', 'health_check_active': 'Active', 'dhcp_service': 'Enable', 'dhcp_service_v6': 'Disable', 'dns_service': 'Enable', 'type': 'vport', 'label': '49', 'l2_label': '53', 'vxlan_id': '10', 'vn_name': 'default-domain:ctest-TestSvcRegr-42593455:ctest-right_vn-31067898', 'vm_uuid': '8b5f0a7a-ba0a-40e8-b2ba-e5da970fb155', 'vm_name': 'ctest-pt_svm0-38514793', 'ip_addr': '176.147.68.68', 'mac_addr': '02:cd:99:5a:4b:75', 'policy': 'Enable', 'fip_list': [], 'mdata_ip_addr': '169.254.0.7', 'service_vlan_list': None, 'os_ifindex': '28', 'fabric_port': 'NotFabricPort', 'alloc_linklocal_ip': 'LL-Enable', 'analyzer_name': None, 'config_name': 'default-domain:ctest-TestSvcRegr-42593455:cd995a4b-7526-4aca-8095-d768b925eae3', 'sg_uuid_list': ['badea825-0520-4ced-a7f5-6704510394d6'], 'static_route_list': None, 'vm_project_uuid': '944418ee-ac66-4359-804f-c1002088adc7', 'admin_state': 'Enabled', 'flow_key_idx': '28', 'allowed_address_pair_list': None, 'ip6_addr': '::', 'local_preference': '0', 'tx_vlan_id': '-1', 'rx_vlan_id': '-1', 'parent_interface': None, 'subnet': '--NA--', 'sub_type': 'Tap', 'vrf_assign_acl_uuid': '--NA--', 'vmi_type': 'Virtual Machine', 'transport': 'Ethernet', 'logical_interface_uuid': '00000000-0000-0000-0000-000000000000', 'flood_unknown_unicast': 'false', 'physical_device': None, 'physical_interface': None, 'fixed_ip4_list': None, 'fixed_ip6_list': None, 'fat_flow_list': None, 'metadata_ip_active': 'Active', 'service_health_check_ip': '0.0.0.0', 'alias_ip_list': None, 'drop_new_flows': 'false', 'bridge_domain_list': [], 'vmi_tag_list': None, 'policy_set_acl_list': None, 'slo_list': None, 'vhostuser_mode': '0', 'si_other_end_vmi': '2179f041-f4b1-45d2-8b9c-1d68716aedaf', 'cfg_igmp_enable': 'false', 'igmp_enabled': 'false', 'max_flows': '0', 'policy_set_fwaas_list': None, 'bond_interface_list': None, 'mac_ip_learning_enable': 'false', 'mac_ip_list': []} 2024-10-25 15:23:13,728 - DEBUG - Starting Layer 2 verification in Agent 2024-10-25 15:23:13,745 - DEBUG - Layer 2 path is seen for VM MAC 02:cd:99:5a:4b:75 in agent 10.0.0.31 2024-10-25 15:23:13,745 - DEBUG - Active layer 2 route in agent is present for VMI tapcd995a4b-75 2024-10-25 15:23:13,746 - DEBUG - L2 label(53) matches bw route table and interface table 2024-10-25 15:23:13,754 - DEBUG - VN default-domain:ctest-TestSvcRegr-42593455:ctest-right_vn-31067898 is found in Agent of node 10.0.0.31 2024-10-25 15:23:13,986 - DEBUG - VRF IDs of VN default-domain:ctest-TestSvcRegr-42593455:ctest-right_vn-31067898 is consistent in agent 10.0.0.31 2024-10-25 15:23:13,987 - DEBUG - Route for VM IP ['176.147.68.68'] is consistent in agent 10.0.0.31 2024-10-25 15:23:13,987 - DEBUG - VN default-domain:ctest-TestSvcRegr-42593455:ctest-right_vn-31067898 verification for VM ctest-pt_svm0-38514793 in Agent 10.0.0.31 passed 2024-10-25 15:23:14,196 - DEBUG - Starting all layer 2 verification in agent 10.0.0.31 2024-10-25 15:23:14,210 - DEBUG - Route for VM MAC 02:cd:99:5a:4b:75 is consistent in agent 10.0.0.31 2024-10-25 15:23:14,210 - INFO - VM ctest-pt_svm0-38514793 verifications in Compute nodes passed 2024-10-25 15:23:14,210 - DEBUG - The SVMs in the SI are : [] 2024-10-25 15:23:14,210 - DEBUG - Requesting: http://10.0.0.31:8082/virtual-machine/8b5f0a7a-ba0a-40e8-b2ba-e5da970fb155 2024-10-25 15:23:14,221 - DEBUG - Service VM for SI 'ctest-si-98986743' is launched 2024-10-25 15:23:14,221 - DEBUG - Requesting: http://10.0.0.31:8082/domains 2024-10-25 15:23:14,228 - DEBUG - Requesting: http://10.0.0.31:8082/domain/5c9222a5-1004-4c8d-a5ba-6a8cd4423477 2024-10-25 15:23:14,244 - DEBUG - Requesting: http://10.0.0.31:8082/project/944418ee-ac66-4359-804f-c1002088adc7 2024-10-25 15:23:14,337 - DEBUG - Requesting: http://10.0.0.31:8082/service-instance/dacf46d7-d471-479b-999d-365e7cf7a0f1 2024-10-25 15:23:14,350 - DEBUG - SI ctest-si-98986743 has Port Tuple: ['ctest-port_tuple-17101521'] 2024-10-25 15:23:14,460 - WARNING - RI refs is none for RI ['default-domain', 'ctest-TestSvcRegr-42593455', 'ctest-left_vn-76866856', 'ctest-left_vn-76866856'] 2024-10-25 15:23:19,576 - WARNING - RI refs is none for RI ['default-domain', 'ctest-TestSvcRegr-42593455', 'ctest-left_vn-76866856', 'ctest-left_vn-76866856'] 2024-10-25 15:23:24,684 - WARNING - RI refs is none for RI ['default-domain', 'ctest-TestSvcRegr-42593455', 'ctest-left_vn-76866856', 'ctest-left_vn-76866856'] 2024-10-25 15:23:29,796 - WARNING - RI refs is none for RI ['default-domain', 'ctest-TestSvcRegr-42593455', 'ctest-left_vn-76866856', 'ctest-left_vn-76866856'] 2024-10-25 15:23:34,904 - WARNING - RI refs is none for RI ['default-domain', 'ctest-TestSvcRegr-42593455', 'ctest-left_vn-76866856', 'ctest-left_vn-76866856'] 2024-10-25 15:23:40,016 - WARNING - RI refs is none for RI ['default-domain', 'ctest-TestSvcRegr-42593455', 'ctest-left_vn-76866856', 'ctest-left_vn-76866856'] 2024-10-25 15:23:45,132 - WARNING - RI refs is none for RI ['default-domain', 'ctest-TestSvcRegr-42593455', 'ctest-left_vn-76866856', 'ctest-left_vn-76866856'] 2024-10-25 15:23:50,244 - WARNING - RI refs is none for RI ['default-domain', 'ctest-TestSvcRegr-42593455', 'ctest-left_vn-76866856', 'ctest-left_vn-76866856'] 2024-10-25 15:23:55,360 - WARNING - RI refs is none for RI ['default-domain', 'ctest-TestSvcRegr-42593455', 'ctest-left_vn-76866856', 'ctest-left_vn-76866856'] 2024-10-25 15:24:00,476 - WARNING - RI refs is none for RI ['default-domain', 'ctest-TestSvcRegr-42593455', 'ctest-left_vn-76866856', 'ctest-left_vn-76866856'] 2024-10-25 15:24:05,588 - WARNING - RI refs is none for RI ['default-domain', 'ctest-TestSvcRegr-42593455', 'ctest-left_vn-76866856', 'ctest-left_vn-76866856'] 2024-10-25 15:24:10,700 - WARNING - RI refs is none for RI ['default-domain', 'ctest-TestSvcRegr-42593455', 'ctest-left_vn-76866856', 'ctest-left_vn-76866856'] 2024-10-25 15:24:15,812 - WARNING - RI refs is none for RI ['default-domain', 'ctest-TestSvcRegr-42593455', 'ctest-left_vn-76866856', 'ctest-left_vn-76866856'] 2024-10-25 15:24:20,924 - WARNING - RI refs is none for RI ['default-domain', 'ctest-TestSvcRegr-42593455', 'ctest-left_vn-76866856', 'ctest-left_vn-76866856'] 2024-10-25 15:24:26,032 - WARNING - RI refs is none for RI ['default-domain', 'ctest-TestSvcRegr-42593455', 'ctest-left_vn-76866856', 'ctest-left_vn-76866856'] 2024-10-25 15:24:31,149 - WARNING - RI refs is none for RI ['default-domain', 'ctest-TestSvcRegr-42593455', 'ctest-left_vn-76866856', 'ctest-left_vn-76866856'] 2024-10-25 15:24:31,153 - INFO - Dettaching policy ctest-policy-93545486 from vn ctest-right_vn-31067898 2024-10-25 15:24:31,428 - INFO - Dettaching policy ctest-policy-93545486 from vn ctest-left_vn-76866856 2024-10-25 15:24:31,757 - INFO - Deleted policy ctest-policy-93545486 2024-10-25 15:24:31,758 - DEBUG - Deleting service instance: ['default-domain', 'ctest-TestSvcRegr-42593455', 'ctest-si-98986743'] 2024-10-25 15:24:31,854 - INFO - Deleted SI ['default-domain', 'ctest-TestSvcRegr-42593455', 'ctest-si-98986743'] 2024-10-25 15:24:31,854 - DEBUG - Requesting: http://10.0.0.31:8082/domains 2024-10-25 15:24:31,869 - DEBUG - Requesting: http://10.0.0.31:8082/domain/5c9222a5-1004-4c8d-a5ba-6a8cd4423477 2024-10-25 15:24:31,891 - DEBUG - Requesting: http://10.0.0.31:8082/project/944418ee-ac66-4359-804f-c1002088adc7 2024-10-25 15:24:31,981 - DEBUG - Service instance ctest-si-98986743 removed from api server 2024-10-25 15:24:31,981 - DEBUG - Requesting: http://10.0.0.31:8082/virtual-machine/8b5f0a7a-ba0a-40e8-b2ba-e5da970fb155 2024-10-25 15:24:31,992 - DEBUG - All Service VMs unlinked from SI ctest-si-98986743 2024-10-25 15:24:31,999 - DEBUG - 0 svc intances found in all projects. They are [] 2024-10-25 15:24:31,999 - DEBUG - 0 svc intances found in current project. They are [] 2024-10-25 15:24:32,048 - DEBUG - 0 svc intances found in all projects. They are [] 2024-10-25 15:24:32,048 - DEBUG - 0 svc intances found in current project. They are [] 2024-10-25 15:24:32,048 - INFO - Deleting VM ctest-pt_svm0-38514793 2024-10-25 15:24:32,137 - DEBUG - Deleting service template: ['default-domain', 'ctest-service_template_1-71491873'] 2024-10-25 15:24:32,240 - INFO - Service template: ['default-domain', 'ctest-service_template_1-71491873'] deleted successfully. 2024-10-25 15:24:32,241 - INFO - Deleting VM ctest-right_vm-05280760 2024-10-25 15:24:32,338 - INFO - Deleting VM ctest-left_vm-25705975 2024-10-25 15:24:32,444 - INFO - Deleting VN ctest-right_vn-31067898 2024-10-25 15:24:32,495 - DEBUG - VN f07e9ccd-9060-4e7b-8f76-09171c3e2aaf still in use: Unable to complete operation on network f07e9ccd-9060-4e7b-8f76-09171c3e2aaf. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-4fbaf1f3-28ce-4023-8653-f498ec1f6325'] 2024-10-25 15:24:32,496 - WARNING - Deleting VN ctest-right_vn-31067898 failed..Will retry 2024-10-25 15:24:34,632 - DEBUG - Response for deleting network () 2024-10-25 15:24:34,632 - DEBUG - Requesting: http://10.0.0.31:8082/routing-instance/e755447d-c88a-4b6c-a3af-04c07f13037a 2024-10-25 15:24:34,643 - DEBUG - Response Code: 404 2024-10-25 15:24:34,643 - DEBUG - Requesting: http://10.0.0.31:8082/domains 2024-10-25 15:24:34,650 - DEBUG - Requesting: http://10.0.0.31:8082/domain/5c9222a5-1004-4c8d-a5ba-6a8cd4423477 2024-10-25 15:24:34,709 - DEBUG - Requesting: http://10.0.0.31:8082/project/944418ee-ac66-4359-804f-c1002088adc7 2024-10-25 15:24:34,798 - INFO - Validated that VN ctest-right_vn-31067898 is not found in API Server 2024-10-25 15:24:34,813 - DEBUG - VN ctest-right_vn-31067898 is not present in Agent 10.0.0.31 2024-10-25 15:24:34,813 - INFO - Validated that VN ctest-right_vn-31067898 is not in any agent 2024-10-25 15:24:34,832 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-right_vn-31067898 info 2024-10-25 15:24:34,832 - INFO - Deleting VN ctest-left_vn-76866856 2024-10-25 15:24:34,963 - DEBUG - Response for deleting network () 2024-10-25 15:24:34,963 - DEBUG - Requesting: http://10.0.0.31:8082/routing-instance/df6c5be0-1bb2-4f69-8e9e-0ea46fe3d2f1 2024-10-25 15:24:34,969 - DEBUG - Response Code: 404 2024-10-25 15:24:34,969 - DEBUG - Requesting: http://10.0.0.31:8082/domains 2024-10-25 15:24:34,986 - DEBUG - Requesting: http://10.0.0.31:8082/domain/5c9222a5-1004-4c8d-a5ba-6a8cd4423477 2024-10-25 15:24:35,047 - DEBUG - Requesting: http://10.0.0.31:8082/project/944418ee-ac66-4359-804f-c1002088adc7 2024-10-25 15:24:35,127 - INFO - Validated that VN ctest-left_vn-76866856 is not found in API Server 2024-10-25 15:24:35,140 - DEBUG - VN ctest-left_vn-76866856 is not present in Agent 10.0.0.31 2024-10-25 15:24:35,140 - INFO - Validated that VN ctest-left_vn-76866856 is not in any agent 2024-10-25 15:24:35,159 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-left_vn-76866856 info 2024-10-25 15:24:35,159 - INFO - Deleting VN ctest-mgmt_vn-02141399 2024-10-25 15:24:35,276 - DEBUG - Response for deleting network () 2024-10-25 15:24:35,276 - DEBUG - Requesting: http://10.0.0.31:8082/routing-instance/fdd5bc74-2ff0-4447-8d6c-e0a926ec246a 2024-10-25 15:24:35,282 - DEBUG - Response Code: 404 2024-10-25 15:24:35,282 - DEBUG - Requesting: http://10.0.0.31:8082/domains 2024-10-25 15:24:35,294 - DEBUG - Requesting: http://10.0.0.31:8082/domain/5c9222a5-1004-4c8d-a5ba-6a8cd4423477 2024-10-25 15:24:35,352 - DEBUG - Requesting: http://10.0.0.31:8082/project/944418ee-ac66-4359-804f-c1002088adc7 2024-10-25 15:24:35,438 - INFO - Validated that VN ctest-mgmt_vn-02141399 is not found in API Server 2024-10-25 15:24:35,452 - DEBUG - VN ctest-mgmt_vn-02141399 is not present in Agent 10.0.0.31 2024-10-25 15:24:35,452 - INFO - Validated that VN ctest-mgmt_vn-02141399 is not in any agent 2024-10-25 15:24:35,470 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-mgmt_vn-02141399 info 2024-10-25 15:24:35,720 - ERROR - AssertionError Python 3.6.8: /usr/bin/python3 Fri Oct 25 15:24:31 2024 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={}) 79 log.info('Initial checks done. Running the testcase now') 80 log.info('') 81 result = function(self, *args, **kwargs) 82 if self.inputs.upgrade: 83 pid = os.getpid() result = None function = self = args = () kwargs = {} /contrail-test/scripts/svc_firewall/test_svc_fw.py in test_svc_in_network_datapath(self=) 31 def test_svc_in_network_datapath(self): 32 return self.verify_svc_chain(svc_img_name='cirros_in_net', service_mode='in-network', 33 create_svms=True) 34 35 create_svms undefined /contrail-test/common/servicechain/verify.py in verify_svc_chain(self=, *args=(), **kwargs={'create_svms': True, 'service_mode': 'in-network', 'svc_img_name': 'cirros_in_net'}) 344 if not evpn: 345 result, msg = self.validate_vn(left_vn_fq_name) 346 assert result, msg 347 right_vn = True if st_fixture.service_mode == 'in-network-nat' else False 348 result, msg = self.validate_vn(right_vn_fq_name, right_vn=right_vn) result = False msg = "RI refs is none for RI ['default-domain', 'ctest...test-left_vn-76866856', 'ctest-left_vn-76866856']" AssertionError: RI refs is none for RI ['default-domain', 'ctest-TestSvcRegr-42593455', 'ctest-left_vn-76866856', 'ctest-left_vn-76866856'] __cause__ = None __class__ = __context__ = None __delattr__ = __dict__ = {} __dir__ = __doc__ = 'Assertion failed.' __eq__ = __format__ = __ge__ = __getattribute__ = __gt__ = __hash__ = __init__ = __init_subclass__ = __le__ = __lt__ = __ne__ = __new__ = __reduce__ = __reduce_ex__ = __repr__ = __setattr__ = __setstate__ = __sizeof__ = __str__ = __subclasshook__ = __suppress_context__ = False __traceback__ = args = ("RI refs is none for RI ['default-domain', 'ctest...test-left_vn-76866856', 'ctest-left_vn-76866856']",) with_traceback = The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/contrail-test/tcutils/wrappers.py", line 81, in wrapper result = function(self, *args, **kwargs) File "/contrail-test/scripts/svc_firewall/test_svc_fw.py", line 33, in test_svc_in_network_datapath create_svms=True) File "/contrail-test/common/servicechain/verify.py", line 346, in verify_svc_chain assert result, msg AssertionError: RI refs is none for RI ['default-domain', 'ctest-TestSvcRegr-42593455', 'ctest-left_vn-76866856', 'ctest-left_vn-76866856'] 2024-10-25 15:24:35,727 - DEBUG - No XMPP flaps were noticed during the test 2024-10-25 15:24:35,727 - INFO - 2024-10-25 15:24:35,727 - INFO - END TEST : test_svc_in_network_datapath : FAILED[0:02:35] 2024-10-25 15:24:35,728 - INFO - -------------------------------------------------------------------------------- 2024-10-25 15:24:36,572 - INFO - Deleted project: ctest-TestSvcRegr-42593455, ID : 944418ee-ac66-4359-804f-c1002088adc7