2025-07-31 04:17:57,383 - INFO - Domain Default found not creating 2025-07-31 04:17:57,632 - INFO - Project ctest-TestSvcRegr-57366020 not found, creating it 2025-07-31 04:17:58,296 - INFO - Created Project:ctest-TestSvcRegr-57366020, ID : 5a616162-254d-44cd-9d78-bdd1b735b3df 2025-07-31 04:17:59,908 - INFO - ================================================================================ 2025-07-31 04:17:59,908 - INFO - STARTING TEST : test_svc_in_net_nat_with_static_routes 2025-07-31 04:18:01,231 - DEBUG - Nothing to compare xmpp stats {'10.0.0.27': {'10.20.0.193': '0', '10.20.0.18': '0'}, '10.0.0.57': {'10.20.0.18': '0', '10.20.0.193': '0'}} with 2025-07-31 04:18:01,231 - INFO - Initial checks done. Running the testcase now 2025-07-31 04:18:01,231 - INFO - 2025-07-31 04:18:01,929 - DEBUG - Response for create_network : {'network': {'id': '8ca8a68e-b34a-406e-b68a-af0373baf716', 'name': 'ctest-third-vn-88616602', 'tenant_id': '5a616162254d44cd9d78bdd1b735b3df', 'project_id': '5a616162254d44cd9d78bdd1b735b3df', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcRegr-57366020', 'ctest-third-vn-88616602'], 'port_security_enabled': True, 'description': ''}} 2025-07-31 04:18:02,228 - DEBUG - Response for create_subnet : {'subnet': {'id': '5398c505-3130-4227-91a4-66395ed0cb73', 'name': '', 'tenant_id': '5a616162254d44cd9d78bdd1b735b3df', 'network_id': '8ca8a68e-b34a-406e-b68a-af0373baf716', 'ip_version': 4, 'cidr': '49.241.67.0/26', 'allocation_pools': [{'start': '49.241.67.2', 'end': '49.241.67.62'}], 'gateway_ip': '49.241.67.1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '49.241.67.2', 'tags': [], 'project_id': '5a616162254d44cd9d78bdd1b735b3df'}} 2025-07-31 04:18:02,255 - INFO - Created VN ctest-third-vn-88616602 2025-07-31 04:18:02,269 - DEBUG - VN ctest-third-vn-88616602 UUID is 8ca8a68e-b34a-406e-b68a-af0373baf716 2025-07-31 04:18:02,414 - DEBUG - Services list from nova: [, , , ] 2025-07-31 04:18:03,846 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3732-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3732-1) 2025-07-31 04:18:03,897 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-07-31 04:18:09,048 - DEBUG - VM is in ACTIVE state now 2025-07-31 04:18:09,564 - DEBUG - Response for create_network : {'network': {'id': '5a238d4d-35ea-4e1c-8149-18ef0b4f9c60', 'name': 'ctest-mgmt_vn-54779878', 'tenant_id': '5a616162254d44cd9d78bdd1b735b3df', 'project_id': '5a616162254d44cd9d78bdd1b735b3df', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcRegr-57366020', 'ctest-mgmt_vn-54779878'], 'port_security_enabled': True, 'description': ''}} 2025-07-31 04:18:09,836 - DEBUG - Response for create_subnet : {'subnet': {'id': '59a9c5ab-9a9d-4707-8413-3f3f31396302', 'name': '', 'tenant_id': '5a616162254d44cd9d78bdd1b735b3df', 'network_id': '5a238d4d-35ea-4e1c-8149-18ef0b4f9c60', 'ip_version': 4, 'cidr': '14.104.196.64/26', 'allocation_pools': [{'start': '14.104.196.66', 'end': '14.104.196.126'}], 'gateway_ip': '14.104.196.65', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '14.104.196.66', 'tags': [], 'project_id': '5a616162254d44cd9d78bdd1b735b3df'}} 2025-07-31 04:18:09,860 - INFO - Created VN ctest-mgmt_vn-54779878 2025-07-31 04:18:09,878 - DEBUG - VN ctest-mgmt_vn-54779878 UUID is 5a238d4d-35ea-4e1c-8149-18ef0b4f9c60 2025-07-31 04:18:09,973 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:18:10,307 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:18:10,340 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:18:10,370 - DEBUG - Requesting: http://10.0.0.41:8082/project/5a616162-254d-44cd-9d78-bdd1b735b3df 2025-07-31 04:18:10,485 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/5a238d4d-35ea-4e1c-8149-18ef0b4f9c60 2025-07-31 04:18:10,499 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/5a238d4d-35ea-4e1c-8149-18ef0b4f9c60 2025-07-31 04:18:10,519 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/42b3b332-ff95-4813-a672-108174fecf82 2025-07-31 04:18:10,543 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/42b3b332-ff95-4813-a672-108174fecf82 2025-07-31 04:18:10,558 - DEBUG - Requesting: http://10.0.0.41:8082/route-target/5f5c4655-1385-4eb4-aa8e-b9b1451a065f 2025-07-31 04:18:10,572 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-31 04:18:10,573 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/5a238d4d-35ea-4e1c-8149-18ef0b4f9c60 2025-07-31 04:18:10,591 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/42b3b332-ff95-4813-a672-108174fecf82 2025-07-31 04:18:10,614 - INFO - Verified VN network id 12 for VN 5a238d4d-35ea-4e1c-8149-18ef0b4f9c60 2025-07-31 04:18:10,614 - INFO - Verifications in API Server for VN ctest-mgmt_vn-54779878 passed 2025-07-31 04:18:10,614 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/5a238d4d-35ea-4e1c-8149-18ef0b4f9c60 2025-07-31 04:18:10,632 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/42b3b332-ff95-4813-a672-108174fecf82 2025-07-31 04:18:10,643 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/42b3b332-ff95-4813-a672-108174fecf82 2025-07-31 04:18:10,653 - DEBUG - Requesting: http://10.0.0.41:8082/route-target/5f5c4655-1385-4eb4-aa8e-b9b1451a065f 2025-07-31 04:18:10,678 - DEBUG - Control-node 10.0.0.41 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-57366020:ctest-mgmt_vn-54779878', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-57366020', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '6495190450354933276', 'uuid-lslong': '9316004719158598752'}, 'enable': 'true', 'created': '2025-07-31T04:18:09', 'last-modified': '2025-07-31T04:18:09', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.913537'} 2025-07-31 04:18:10,689 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-31 04:18:10,698 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-57366020:ctest-mgmt_vn-54779878', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-57366020', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '6495190450354933276', 'uuid-lslong': '9316004719158598752'}, 'enable': 'true', 'created': '2025-07-31T04:18:09', 'last-modified': '2025-07-31T04:18:09', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.937721'} 2025-07-31 04:18:10,705 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-31 04:18:10,711 - DEBUG - Control-node 10.0.0.61 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-57366020:ctest-mgmt_vn-54779878', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-57366020', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '6495190450354933276', 'uuid-lslong': '9316004719158598752'}, 'enable': 'true', 'created': '2025-07-31T04:18:09', 'last-modified': '2025-07-31T04:18:09', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.952029'} 2025-07-31 04:18:10,718 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-31 04:18:10,718 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-mgmt_vn-54779878 passed 2025-07-31 04:18:10,718 - DEBUG - ====Verifying policy data for ctest-mgmt_vn-54779878 in API_Server ====== 2025-07-31 04:18:10,718 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:18:10,726 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:18:10,751 - DEBUG - Requesting: http://10.0.0.41:8082/project/5a616162-254d-44cd-9d78-bdd1b735b3df 2025-07-31 04:18:10,897 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/5a238d4d-35ea-4e1c-8149-18ef0b4f9c60 2025-07-31 04:18:10,912 - DEBUG - =>VN ctest-mgmt_vn-54779878 has no policy to be verified 2025-07-31 04:18:10,912 - DEBUG - Verifying the vn in opserver 2025-07-31 04:18:10,912 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-57366020:ctest-mgmt_vn-54779878 virtual network link through opserver 10.0.0.41 2025-07-31 04:18:10,913 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/virtual-networks 2025-07-31 04:18:11,214 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/virtual-networks 2025-07-31 04:18:11,313 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-57366020:ctest-mgmt_vn-54779878', 'href': 'http://10.0.0.41:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-57366020:ctest-mgmt_vn-54779878?flat'} 2025-07-31 04:18:11,313 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-57366020:ctest-mgmt_vn-54779878 virtual network link through opserver 10.0.0.38 2025-07-31 04:18:11,313 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-07-31 04:18:11,675 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-07-31 04:18:11,773 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-57366020:ctest-mgmt_vn-54779878', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-57366020:ctest-mgmt_vn-54779878?flat'} 2025-07-31 04:18:11,773 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-57366020:ctest-mgmt_vn-54779878 virtual network link through opserver 10.0.0.61 2025-07-31 04:18:11,773 - DEBUG - Requesting: http://10.0.0.61:8081/analytics/uves/virtual-networks 2025-07-31 04:18:12,100 - DEBUG - Requesting: http://10.0.0.61:8081/analytics/uves/virtual-networks 2025-07-31 04:18:12,167 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-57366020:ctest-mgmt_vn-54779878', 'href': 'http://10.0.0.61:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-57366020:ctest-mgmt_vn-54779878?flat'} 2025-07-31 04:18:12,167 - INFO - Validated that VN default-domain:ctest-TestSvcRegr-57366020:ctest-mgmt_vn-54779878 is found in opserver 2025-07-31 04:18:12,195 - DEBUG - Do not have enough data to verify VN in agent 2025-07-31 04:18:12,210 - DEBUG - VRF ids for VN ctest-mgmt_vn-54779878: {} 2025-07-31 04:18:12,559 - DEBUG - Response for create_network : {'network': {'id': 'ae1ad208-64d0-4e50-ac0e-c1025a2a1660', 'name': 'ctest-left_vn-72041942', 'tenant_id': '5a616162254d44cd9d78bdd1b735b3df', 'project_id': '5a616162254d44cd9d78bdd1b735b3df', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcRegr-57366020', 'ctest-left_vn-72041942'], 'port_security_enabled': True, 'description': ''}} 2025-07-31 04:18:12,825 - DEBUG - Response for create_subnet : {'subnet': {'id': '7c9ec1d4-9f76-4863-8f4e-9fafeefc5a51', 'name': '', 'tenant_id': '5a616162254d44cd9d78bdd1b735b3df', 'network_id': 'ae1ad208-64d0-4e50-ac0e-c1025a2a1660', 'ip_version': 4, 'cidr': '56.101.225.0/26', 'allocation_pools': [{'start': '56.101.225.2', 'end': '56.101.225.62'}], 'gateway_ip': '56.101.225.1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '56.101.225.2', 'tags': [], 'project_id': '5a616162254d44cd9d78bdd1b735b3df'}} 2025-07-31 04:18:12,852 - INFO - Created VN ctest-left_vn-72041942 2025-07-31 04:18:12,866 - DEBUG - VN ctest-left_vn-72041942 UUID is ae1ad208-64d0-4e50-ac0e-c1025a2a1660 2025-07-31 04:18:12,970 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:18:12,977 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:18:12,991 - DEBUG - Requesting: http://10.0.0.41:8082/project/5a616162-254d-44cd-9d78-bdd1b735b3df 2025-07-31 04:18:13,090 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/ae1ad208-64d0-4e50-ac0e-c1025a2a1660 2025-07-31 04:18:13,104 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/ae1ad208-64d0-4e50-ac0e-c1025a2a1660 2025-07-31 04:18:13,114 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/5e8f7b22-b968-4f8c-9c96-d8ac007b77b0 2025-07-31 04:18:13,125 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/5e8f7b22-b968-4f8c-9c96-d8ac007b77b0 2025-07-31 04:18:13,132 - DEBUG - Requesting: http://10.0.0.41:8082/route-target/00d75ab9-1d23-4512-a793-d1f9dd13b792 2025-07-31 04:18:13,146 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-31 04:18:13,146 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/ae1ad208-64d0-4e50-ac0e-c1025a2a1660 2025-07-31 04:18:13,158 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/5e8f7b22-b968-4f8c-9c96-d8ac007b77b0 2025-07-31 04:18:13,179 - INFO - Verified VN network id 13 for VN ae1ad208-64d0-4e50-ac0e-c1025a2a1660 2025-07-31 04:18:13,179 - INFO - Verifications in API Server for VN ctest-left_vn-72041942 passed 2025-07-31 04:18:13,180 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/ae1ad208-64d0-4e50-ac0e-c1025a2a1660 2025-07-31 04:18:13,196 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/5e8f7b22-b968-4f8c-9c96-d8ac007b77b0 2025-07-31 04:18:13,208 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/5e8f7b22-b968-4f8c-9c96-d8ac007b77b0 2025-07-31 04:18:13,216 - DEBUG - Requesting: http://10.0.0.41:8082/route-target/00d75ab9-1d23-4512-a793-d1f9dd13b792 2025-07-31 04:18:13,233 - DEBUG - Control-node 10.0.0.41 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-57366020:ctest-left_vn-72041942', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-57366020', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12545570645486882384', 'uuid-lslong': '12398059040044357216'}, 'enable': 'true', 'created': '2025-07-31T04:18:12', 'last-modified': '2025-07-31T04:18:12', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.485212'} 2025-07-31 04:18:13,241 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-31 04:18:13,250 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-57366020:ctest-left_vn-72041942', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-57366020', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12545570645486882384', 'uuid-lslong': '12398059040044357216'}, 'enable': 'true', 'created': '2025-07-31T04:18:12', 'last-modified': '2025-07-31T04:18:12', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.503804'} 2025-07-31 04:18:13,259 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-31 04:18:13,267 - DEBUG - Control-node 10.0.0.61 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-57366020:ctest-left_vn-72041942', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-57366020', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12545570645486882384', 'uuid-lslong': '12398059040044357216'}, 'enable': 'true', 'created': '2025-07-31T04:18:12', 'last-modified': '2025-07-31T04:18:12', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.502628'} 2025-07-31 04:18:13,275 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-31 04:18:13,275 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-left_vn-72041942 passed 2025-07-31 04:18:13,275 - DEBUG - ====Verifying policy data for ctest-left_vn-72041942 in API_Server ====== 2025-07-31 04:18:13,275 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:18:13,286 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:18:13,312 - DEBUG - Requesting: http://10.0.0.41:8082/project/5a616162-254d-44cd-9d78-bdd1b735b3df 2025-07-31 04:18:13,434 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/ae1ad208-64d0-4e50-ac0e-c1025a2a1660 2025-07-31 04:18:13,446 - DEBUG - =>VN ctest-left_vn-72041942 has no policy to be verified 2025-07-31 04:18:13,447 - DEBUG - Verifying the vn in opserver 2025-07-31 04:18:13,447 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-57366020:ctest-left_vn-72041942 virtual network link through opserver 10.0.0.41 2025-07-31 04:18:13,447 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/virtual-networks 2025-07-31 04:18:13,527 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-57366020:ctest-left_vn-72041942', 'href': 'http://10.0.0.41:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-57366020:ctest-left_vn-72041942?flat'} 2025-07-31 04:18:13,528 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-57366020:ctest-left_vn-72041942 virtual network link through opserver 10.0.0.38 2025-07-31 04:18:13,528 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-07-31 04:18:13,622 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-57366020:ctest-left_vn-72041942', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-57366020:ctest-left_vn-72041942?flat'} 2025-07-31 04:18:13,622 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-57366020:ctest-left_vn-72041942 virtual network link through opserver 10.0.0.61 2025-07-31 04:18:13,622 - DEBUG - Requesting: http://10.0.0.61:8081/analytics/uves/virtual-networks 2025-07-31 04:18:13,728 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-57366020:ctest-left_vn-72041942', 'href': 'http://10.0.0.61:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-57366020:ctest-left_vn-72041942?flat'} 2025-07-31 04:18:13,728 - INFO - Validated that VN default-domain:ctest-TestSvcRegr-57366020:ctest-left_vn-72041942 is found in opserver 2025-07-31 04:18:13,781 - DEBUG - Do not have enough data to verify VN in agent 2025-07-31 04:18:13,808 - DEBUG - VRF ids for VN ctest-left_vn-72041942: {} 2025-07-31 04:18:14,174 - DEBUG - Response for create_network : {'network': {'id': '1217d3f0-aa59-41e2-b988-6dc68a00a21a', 'name': 'ctest-right_vn-56468087', 'tenant_id': '5a616162254d44cd9d78bdd1b735b3df', 'project_id': '5a616162254d44cd9d78bdd1b735b3df', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcRegr-57366020', 'ctest-right_vn-56468087'], 'port_security_enabled': True, 'description': ''}} 2025-07-31 04:18:14,419 - DEBUG - Response for create_subnet : {'subnet': {'id': '0e3fb6ef-9f4e-4ad2-8481-90d22a925e1e', 'name': '', 'tenant_id': '5a616162254d44cd9d78bdd1b735b3df', 'network_id': '1217d3f0-aa59-41e2-b988-6dc68a00a21a', 'ip_version': 4, 'cidr': '45.143.40.128/26', 'allocation_pools': [{'start': '45.143.40.130', 'end': '45.143.40.190'}], 'gateway_ip': '45.143.40.129', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '45.143.40.130', 'tags': [], 'project_id': '5a616162254d44cd9d78bdd1b735b3df'}} 2025-07-31 04:18:14,447 - INFO - Created VN ctest-right_vn-56468087 2025-07-31 04:18:14,463 - DEBUG - VN ctest-right_vn-56468087 UUID is 1217d3f0-aa59-41e2-b988-6dc68a00a21a 2025-07-31 04:18:14,543 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:18:14,550 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:18:14,565 - DEBUG - Requesting: http://10.0.0.41:8082/project/5a616162-254d-44cd-9d78-bdd1b735b3df 2025-07-31 04:18:14,692 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/1217d3f0-aa59-41e2-b988-6dc68a00a21a 2025-07-31 04:18:14,714 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/1217d3f0-aa59-41e2-b988-6dc68a00a21a 2025-07-31 04:18:14,730 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/6b9b362c-332c-43dc-8c92-9f2fcefad9f0 2025-07-31 04:18:14,741 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/6b9b362c-332c-43dc-8c92-9f2fcefad9f0 2025-07-31 04:18:14,750 - DEBUG - Requesting: http://10.0.0.41:8082/route-target/99a47fd9-387a-4040-a943-da969792ff30 2025-07-31 04:18:14,761 - DEBUG - Route Targets: ['target:64512:8000008'] 2025-07-31 04:18:14,761 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/1217d3f0-aa59-41e2-b988-6dc68a00a21a 2025-07-31 04:18:14,780 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/6b9b362c-332c-43dc-8c92-9f2fcefad9f0 2025-07-31 04:18:14,799 - INFO - Verified VN network id 14 for VN 1217d3f0-aa59-41e2-b988-6dc68a00a21a 2025-07-31 04:18:14,799 - INFO - Verifications in API Server for VN ctest-right_vn-56468087 passed 2025-07-31 04:18:14,799 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/1217d3f0-aa59-41e2-b988-6dc68a00a21a 2025-07-31 04:18:14,819 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/6b9b362c-332c-43dc-8c92-9f2fcefad9f0 2025-07-31 04:18:14,829 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/6b9b362c-332c-43dc-8c92-9f2fcefad9f0 2025-07-31 04:18:14,839 - DEBUG - Requesting: http://10.0.0.41:8082/route-target/99a47fd9-387a-4040-a943-da969792ff30 2025-07-31 04:18:14,856 - DEBUG - Control-node 10.0.0.41 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-57366020:ctest-right_vn-56468087', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-57366020', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '1303743647750636002', 'uuid-lslong': '13369056193335566874'}, 'enable': 'true', 'created': '2025-07-31T04:18:14', 'last-modified': '2025-07-31T04:18:14', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.507137'} 2025-07-31 04:18:14,867 - DEBUG - Route Targets: ['target:64512:8000008'] 2025-07-31 04:18:14,876 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-57366020:ctest-right_vn-56468087', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-57366020', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '1303743647750636002', 'uuid-lslong': '13369056193335566874'}, 'enable': 'true', 'created': '2025-07-31T04:18:14', 'last-modified': '2025-07-31T04:18:14', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.530547'} 2025-07-31 04:18:14,885 - DEBUG - Route Targets: ['target:64512:8000008'] 2025-07-31 04:18:14,894 - DEBUG - Control-node 10.0.0.61 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-57366020:ctest-right_vn-56468087', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-57366020', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '1303743647750636002', 'uuid-lslong': '13369056193335566874'}, 'enable': 'true', 'created': '2025-07-31T04:18:14', 'last-modified': '2025-07-31T04:18:14', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.549704'} 2025-07-31 04:18:14,903 - DEBUG - Route Targets: ['target:64512:8000008'] 2025-07-31 04:18:14,904 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-right_vn-56468087 passed 2025-07-31 04:18:14,904 - DEBUG - ====Verifying policy data for ctest-right_vn-56468087 in API_Server ====== 2025-07-31 04:18:14,904 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:18:14,913 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:18:14,935 - DEBUG - Requesting: http://10.0.0.41:8082/project/5a616162-254d-44cd-9d78-bdd1b735b3df 2025-07-31 04:18:15,067 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/1217d3f0-aa59-41e2-b988-6dc68a00a21a 2025-07-31 04:18:15,086 - DEBUG - =>VN ctest-right_vn-56468087 has no policy to be verified 2025-07-31 04:18:15,086 - DEBUG - Verifying the vn in opserver 2025-07-31 04:18:15,086 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-57366020:ctest-right_vn-56468087 virtual network link through opserver 10.0.0.41 2025-07-31 04:18:15,086 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/virtual-networks 2025-07-31 04:18:15,180 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-57366020:ctest-right_vn-56468087', 'href': 'http://10.0.0.41:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-57366020:ctest-right_vn-56468087?flat'} 2025-07-31 04:18:15,180 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-57366020:ctest-right_vn-56468087 virtual network link through opserver 10.0.0.38 2025-07-31 04:18:15,181 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-07-31 04:18:15,285 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-57366020:ctest-right_vn-56468087', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-57366020:ctest-right_vn-56468087?flat'} 2025-07-31 04:18:15,285 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-57366020:ctest-right_vn-56468087 virtual network link through opserver 10.0.0.61 2025-07-31 04:18:15,285 - DEBUG - Requesting: http://10.0.0.61:8081/analytics/uves/virtual-networks 2025-07-31 04:18:15,381 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-57366020:ctest-right_vn-56468087', 'href': 'http://10.0.0.61:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-57366020:ctest-right_vn-56468087?flat'} 2025-07-31 04:18:15,381 - INFO - Validated that VN default-domain:ctest-TestSvcRegr-57366020:ctest-right_vn-56468087 is found in opserver 2025-07-31 04:18:15,423 - DEBUG - Do not have enough data to verify VN in agent 2025-07-31 04:18:15,441 - DEBUG - VRF ids for VN ctest-right_vn-56468087: {} 2025-07-31 04:18:17,097 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3732-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3732-2) 2025-07-31 04:18:18,708 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3732-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3732-1) 2025-07-31 04:18:18,709 - DEBUG - Creating service template: ['default-domain', 'ctest-service_template_1-64178301'] 2025-07-31 04:18:19,019 - DEBUG - Service template: ['default-domain', 'ctest-service_template_1-64178301'] created succesfully 2025-07-31 04:18:19,507 - DEBUG - Image by name tiny_nat_fw not found 2025-07-31 04:18:19,507 - DEBUG - Installing image tiny_nat_fw 2025-07-31 04:18:19,507 - DEBUG - Image info {'name': 'tinycore-in-network-nat.qcow2.gz', 'webserver': '', 'location': '/images/tinycore/', 'username': 'tc', 'password': 'secret', 'type': 'os', 'flavor': 'contrail_flavor_tiny', 'params': '--container-format bare --disk-format vmdk --property vmware_disktype="sparse" --property vmware_adaptertype="ide" --property hypervisor_type=qemu', 'name_docker': 'phusion-baseimage-enablesshd', 'vctmpl': 'service_chain_images/nat_service/tiny_nat_fw/tinycore-in-network-nat.vmtx', 'vcname': 'service_chain_images/nat_service/tiny_nat_fw/tinycore-in-network-nat.vmdk', 'vmdk': 'service_chain_images/nat_service/tiny_nat_fw/tinycore-in-network-nat-flat.vmdk', 'shrinked_vmdk': "'no'"} 2025-07-31 04:18:19,507 - DEBUG - Image build path http://nexus.opensdn.io/repository///images/tinycore//tinycore-in-network-nat.qcow2.gz 2025-07-31 04:18:19,508 - DEBUG - Download image from web http://nexus.opensdn.io/repository///images/tinycore//tinycore-in-network-nat.qcow2.gz 2025-07-31 04:18:19,523 - DEBUG - Image download (local=True) (proxy=None): Execute cmd: wget -nv http://nexus.opensdn.io/repository///images/tinycore//tinycore-in-network-nat.qcow2.gz -O /contrail-test/images/tinycore-in-network-nat.qcow2.gz 2025-07-31 04:18:19,582 - DEBUG - Image has been downloaded to /contrail-test/images/tinycore-in-network-nat.qcow2.gz 2025-07-31 04:18:19,582 - DEBUG - Unzip image 2025-07-31 04:18:19,582 - DEBUG - Image download (local=True) (proxy=None): Execute cmd: gunzip -f /contrail-test/images/tinycore-in-network-nat.qcow2.gz 2025-07-31 04:18:19,600 - DEBUG - Try to create image in user context 2025-07-31 04:18:19,705 - DEBUG - Deleting service template: ['default-domain', 'ctest-service_template_1-64178301'] 2025-07-31 04:18:19,809 - INFO - Service template: ['default-domain', 'ctest-service_template_1-64178301'] deleted successfully. 2025-07-31 04:18:19,809 - INFO - Skip interface_detach for VM ctest-right_vm-75769786 in state BUILD 2025-07-31 04:18:19,809 - INFO - Deleting VM ctest-right_vm-75769786 2025-07-31 04:18:19,852 - ERROR - VM ctest-right_vm-75769786 has failed to come up 2025-07-31 04:18:19,853 - ERROR - Nova failed to get host of the VM 2025-07-31 04:18:19,853 - ERROR - VM ctest-right_vm-75769786 has failed to come up 2025-07-31 04:18:19,853 - ERROR - Nova failed to get host of the VM 2025-07-31 04:18:19,924 - INFO - Skip interface_detach for VM ctest-left_vm-25055917 in state BUILD 2025-07-31 04:18:19,924 - INFO - Deleting VM ctest-left_vm-25055917 2025-07-31 04:18:19,924 - ERROR - VM ctest-left_vm-25055917 has failed to come up 2025-07-31 04:18:19,925 - ERROR - Nova failed to get host of the VM 2025-07-31 04:18:19,925 - ERROR - VM ctest-left_vm-25055917 has failed to come up 2025-07-31 04:18:19,925 - ERROR - Nova failed to get host of the VM 2025-07-31 04:18:20,001 - INFO - Deleting VN ctest-right_vn-56468087 2025-07-31 04:18:20,063 - DEBUG - VN 1217d3f0-aa59-41e2-b988-6dc68a00a21a still in use: Unable to complete operation on network 1217d3f0-aa59-41e2-b988-6dc68a00a21a. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-766b41e6-9795-49d9-8cb7-7a0b010775fd'] 2025-07-31 04:18:20,063 - WARNING - Deleting VN ctest-right_vn-56468087 failed..Will retry 2025-07-31 04:18:22,228 - DEBUG - Response for deleting network () 2025-07-31 04:18:22,228 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/6b9b362c-332c-43dc-8c92-9f2fcefad9f0 2025-07-31 04:18:22,236 - DEBUG - Response Code: 404 2025-07-31 04:18:22,236 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:18:22,243 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:18:22,302 - DEBUG - Requesting: http://10.0.0.41:8082/project/5a616162-254d-44cd-9d78-bdd1b735b3df 2025-07-31 04:18:22,375 - INFO - Validated that VN ctest-right_vn-56468087 is not found in API Server 2025-07-31 04:18:22,388 - DEBUG - VN ctest-right_vn-56468087 is not present in Agent 10.0.0.27 2025-07-31 04:18:22,403 - DEBUG - VN ctest-right_vn-56468087 is not present in Agent 10.0.0.57 2025-07-31 04:18:22,403 - INFO - Validated that VN ctest-right_vn-56468087 is not in any agent 2025-07-31 04:18:22,445 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-right_vn-56468087 info 2025-07-31 04:18:22,445 - INFO - Deleting VN ctest-left_vn-72041942 2025-07-31 04:18:22,618 - DEBUG - Response for deleting network () 2025-07-31 04:18:22,618 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/5e8f7b22-b968-4f8c-9c96-d8ac007b77b0 2025-07-31 04:18:22,630 - DEBUG - Response Code: 404 2025-07-31 04:18:22,631 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:18:22,638 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:18:22,655 - DEBUG - Requesting: http://10.0.0.41:8082/project/5a616162-254d-44cd-9d78-bdd1b735b3df 2025-07-31 04:18:22,776 - INFO - Validated that VN ctest-left_vn-72041942 is not found in API Server 2025-07-31 04:18:22,796 - DEBUG - VN ctest-left_vn-72041942 is not present in Agent 10.0.0.27 2025-07-31 04:18:22,812 - DEBUG - VN ctest-left_vn-72041942 is not present in Agent 10.0.0.57 2025-07-31 04:18:22,812 - INFO - Validated that VN ctest-left_vn-72041942 is not in any agent 2025-07-31 04:18:22,857 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-left_vn-72041942 info 2025-07-31 04:18:22,857 - INFO - Deleting VN ctest-mgmt_vn-54779878 2025-07-31 04:18:23,044 - DEBUG - Response for deleting network () 2025-07-31 04:18:23,044 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/42b3b332-ff95-4813-a672-108174fecf82 2025-07-31 04:18:23,056 - DEBUG - Response Code: 404 2025-07-31 04:18:23,056 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:18:23,063 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:18:23,096 - DEBUG - Requesting: http://10.0.0.41:8082/project/5a616162-254d-44cd-9d78-bdd1b735b3df 2025-07-31 04:18:23,220 - INFO - Validated that VN ctest-mgmt_vn-54779878 is not found in API Server 2025-07-31 04:18:23,239 - DEBUG - VN ctest-mgmt_vn-54779878 is not present in Agent 10.0.0.27 2025-07-31 04:18:23,255 - DEBUG - VN ctest-mgmt_vn-54779878 is not present in Agent 10.0.0.57 2025-07-31 04:18:23,255 - INFO - Validated that VN ctest-mgmt_vn-54779878 is not in any agent 2025-07-31 04:18:23,301 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-mgmt_vn-54779878 info 2025-07-31 04:18:23,301 - INFO - Deleting VM ctest-vm-in-third-vn-30988062 2025-07-31 04:18:23,402 - INFO - Deleting VN ctest-third-vn-88616602 2025-07-31 04:18:23,453 - DEBUG - VN 8ca8a68e-b34a-406e-b68a-af0373baf716 still in use: Unable to complete operation on network 8ca8a68e-b34a-406e-b68a-af0373baf716. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-ac21eb47-47cc-42b5-923c-efa0a65c684c'] 2025-07-31 04:18:23,453 - WARNING - Deleting VN ctest-third-vn-88616602 failed..Will retry 2025-07-31 04:18:25,634 - DEBUG - Response for deleting network () 2025-07-31 04:18:26,964 - ERROR - 
Traceback (most recent call last):
  File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper
    result = function(self, *args, **kwargs)
  File "/contrail-test/scripts/svc_firewall/test_svc_fw.py", line 49, in test_svc_in_net_nat_with_static_routes
    static_route={'left': routes})
  File "/contrail-test/common/servicechain/verify.py", line 320, in verify_svc_chain
    ret_dict = svc_chain_info or self.config_svc_chain(*args, **kwargs)
  File "/contrail-test/common/servicechain/config.py", line 533, in config_svc_chain
    max_inst=max_inst)
  File "/contrail-test/common/servicechain/config.py", line 335, in create_service_vms
    zone=random.choice(valid_zones))
  File "/contrail-test/common/servicechain/config.py", line 161, in config_vm_only
    flavor=flavor, zone=zone, node_name=node_name,**kwargs)
  File "/contrail-test/common/servicechain/config.py", line 203, in config_vm
    flavor=flavor, vn_objs=vns, count=count, zone=zone,**kwargs))
  File "/usr/local/lib/python3.6/site-packages/testtools/testcase.py", line 756, in useFixture
    reraise(*exc_info)
  File "/usr/local/lib/python3.6/site-packages/testtools/_compat3x.py", line 16, in reraise
    raise exc_obj.with_traceback(exc_tb)
  File "/usr/local/lib/python3.6/site-packages/testtools/testcase.py", line 731, in useFixture
    fixture.setUp()
  File "/contrail-test/fixtures/vm_test.py", line 188, in setUp
    self.create()
  File "/contrail-test/fixtures/vm_test.py", line 240, in create
    fixed_ips=self.fixed_ips)
  File "/contrail-test/fixtures/openstack.py", line 118, in create_vm
    return self.nova_h.create_vm(vm_name=vm_name, image_name=image_name, vn_ids=vn_ids, **kwargs)
  File "/contrail-test/fixtures/nova_test.py", line 577, in create_vm
    image = self.get_image(image_name=image_name)
  File "/contrail-test/fixtures/nova_test.py", line 198, in get_image
    self._install_image(image_name=image_name)
  File "/contrail-test/fixtures/nova_test.py", line 338, in _install_image
    return self.copy_and_glance(build_path, image_name, params)
  File "/contrail-test/fixtures/nova_test.py", line 407, in copy_and_glance
    image_id = self.glance_h.create_image(generic_image_name, image_path_real, **params)
  File "/contrail-test/fixtures/glance_test.py", line 50, in create_image
    self.upload_image(obj['id'], filename)
  File "/contrail-test/fixtures/glance_test.py", line 54, in upload_image
    self.obj.images.upload(uuid, open(filename, 'rb'))
FileNotFoundError: [Errno 2] No such file or directory: '/contrail-test/images/tinycore-in-network-nat.qcow2'
2025-07-31 04:18:26,990 - DEBUG - No XMPP flaps were noticed during the test 2025-07-31 04:18:26,990 - INFO - 2025-07-31 04:18:26,991 - INFO - END TEST : test_svc_in_net_nat_with_static_routes : FAILED[0:00:27] 2025-07-31 04:18:26,991 - INFO - -------------------------------------------------------------------------------- 2025-07-31 04:18:27,007 - INFO - ================================================================================ 2025-07-31 04:18:27,008 - INFO - STARTING TEST : test_svc_v2_transparent_datapath 2025-07-31 04:18:28,370 - DEBUG - Nothing to compare xmpp stats {'10.0.0.27': {'10.20.0.193': '0', '10.20.0.18': '0'}, '10.0.0.57': {'10.20.0.18': '0', '10.20.0.193': '0'}} with 2025-07-31 04:18:28,371 - INFO - Initial checks done. Running the testcase now 2025-07-31 04:18:28,371 - INFO - 2025-07-31 04:18:28,730 - DEBUG - Response for create_network : {'network': {'id': 'cb96e2bc-9574-47cf-9828-a8e85d0ca5c1', 'name': 'ctest-mgmt_vn-01374105', 'tenant_id': '5a616162254d44cd9d78bdd1b735b3df', 'project_id': '5a616162254d44cd9d78bdd1b735b3df', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcRegr-57366020', 'ctest-mgmt_vn-01374105'], 'port_security_enabled': True, 'description': ''}} 2025-07-31 04:18:28,978 - DEBUG - Response for create_subnet : {'subnet': {'id': '7cc134b0-8c4f-437c-824c-125264a4fcca', 'name': '', 'tenant_id': '5a616162254d44cd9d78bdd1b735b3df', 'network_id': 'cb96e2bc-9574-47cf-9828-a8e85d0ca5c1', 'ip_version': 4, 'cidr': '131.167.245.64/26', 'allocation_pools': [{'start': '131.167.245.66', 'end': '131.167.245.126'}], 'gateway_ip': '131.167.245.65', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '131.167.245.66', 'tags': [], 'project_id': '5a616162254d44cd9d78bdd1b735b3df'}} 2025-07-31 04:18:29,005 - INFO - Created VN ctest-mgmt_vn-01374105 2025-07-31 04:18:29,021 - DEBUG - VN ctest-mgmt_vn-01374105 UUID is cb96e2bc-9574-47cf-9828-a8e85d0ca5c1 2025-07-31 04:18:29,107 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:18:29,116 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:18:29,137 - DEBUG - Requesting: http://10.0.0.41:8082/project/5a616162-254d-44cd-9d78-bdd1b735b3df 2025-07-31 04:18:29,258 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/cb96e2bc-9574-47cf-9828-a8e85d0ca5c1 2025-07-31 04:18:29,279 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/cb96e2bc-9574-47cf-9828-a8e85d0ca5c1 2025-07-31 04:18:29,294 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/ad15d8ab-07e3-4f60-b272-4df0348892de 2025-07-31 04:18:29,313 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/ad15d8ab-07e3-4f60-b272-4df0348892de 2025-07-31 04:18:29,336 - DEBUG - Requesting: http://10.0.0.41:8082/route-target/cfdd919e-1bb7-4c82-b692-9024696a3b08 2025-07-31 04:18:29,357 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-31 04:18:29,357 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/cb96e2bc-9574-47cf-9828-a8e85d0ca5c1 2025-07-31 04:18:29,384 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/ad15d8ab-07e3-4f60-b272-4df0348892de 2025-07-31 04:18:29,417 - INFO - Verified VN network id 11 for VN cb96e2bc-9574-47cf-9828-a8e85d0ca5c1 2025-07-31 04:18:29,417 - INFO - Verifications in API Server for VN ctest-mgmt_vn-01374105 passed 2025-07-31 04:18:29,418 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/cb96e2bc-9574-47cf-9828-a8e85d0ca5c1 2025-07-31 04:18:29,447 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/ad15d8ab-07e3-4f60-b272-4df0348892de 2025-07-31 04:18:29,470 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/ad15d8ab-07e3-4f60-b272-4df0348892de 2025-07-31 04:18:29,493 - DEBUG - Requesting: http://10.0.0.41:8082/route-target/cfdd919e-1bb7-4c82-b692-9024696a3b08 2025-07-31 04:18:29,539 - DEBUG - Control-node 10.0.0.41 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-57366020:ctest-mgmt_vn-01374105', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-57366020', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '14670162135795124175', 'uuid-lslong': '10964199008780461505'}, 'enable': 'true', 'created': '2025-07-31T04:18:28', 'last-modified': '2025-07-31T04:18:28', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.612856'} 2025-07-31 04:18:29,563 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-31 04:18:29,578 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-57366020:ctest-mgmt_vn-01374105', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-57366020', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '14670162135795124175', 'uuid-lslong': '10964199008780461505'}, 'enable': 'true', 'created': '2025-07-31T04:18:28', 'last-modified': '2025-07-31T04:18:28', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.653017'} 2025-07-31 04:18:29,592 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-31 04:18:29,608 - DEBUG - Control-node 10.0.0.61 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-57366020:ctest-mgmt_vn-01374105', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-57366020', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '14670162135795124175', 'uuid-lslong': '10964199008780461505'}, 'enable': 'true', 'created': '2025-07-31T04:18:28', 'last-modified': '2025-07-31T04:18:28', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.678205'} 2025-07-31 04:18:29,632 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-31 04:18:29,633 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-mgmt_vn-01374105 passed 2025-07-31 04:18:29,634 - DEBUG - ====Verifying policy data for ctest-mgmt_vn-01374105 in API_Server ====== 2025-07-31 04:18:29,634 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:18:29,657 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:18:29,685 - DEBUG - Requesting: http://10.0.0.41:8082/project/5a616162-254d-44cd-9d78-bdd1b735b3df 2025-07-31 04:18:29,821 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/cb96e2bc-9574-47cf-9828-a8e85d0ca5c1 2025-07-31 04:18:29,844 - DEBUG - =>VN ctest-mgmt_vn-01374105 has no policy to be verified 2025-07-31 04:18:29,845 - DEBUG - Verifying the vn in opserver 2025-07-31 04:18:29,845 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-57366020:ctest-mgmt_vn-01374105 virtual network link through opserver 10.0.0.41 2025-07-31 04:18:29,845 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/virtual-networks 2025-07-31 04:18:29,881 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-57366020:ctest-mgmt_vn-01374105', 'href': 'http://10.0.0.41:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-57366020:ctest-mgmt_vn-01374105?flat'} 2025-07-31 04:18:29,881 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-57366020:ctest-mgmt_vn-01374105 virtual network link through opserver 10.0.0.38 2025-07-31 04:18:29,881 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-07-31 04:18:29,899 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-57366020:ctest-mgmt_vn-01374105', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-57366020:ctest-mgmt_vn-01374105?flat'} 2025-07-31 04:18:29,899 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-57366020:ctest-mgmt_vn-01374105 virtual network link through opserver 10.0.0.61 2025-07-31 04:18:29,899 - DEBUG - Requesting: http://10.0.0.61:8081/analytics/uves/virtual-networks 2025-07-31 04:18:29,918 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-57366020:ctest-mgmt_vn-01374105', 'href': 'http://10.0.0.61:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-57366020:ctest-mgmt_vn-01374105?flat'} 2025-07-31 04:18:29,918 - INFO - Validated that VN default-domain:ctest-TestSvcRegr-57366020:ctest-mgmt_vn-01374105 is found in opserver 2025-07-31 04:18:29,960 - DEBUG - Do not have enough data to verify VN in agent 2025-07-31 04:18:29,982 - DEBUG - VRF ids for VN ctest-mgmt_vn-01374105: {} 2025-07-31 04:18:30,385 - DEBUG - Response for create_network : {'network': {'id': 'b97208ee-837c-48c8-9ead-c2693baa242c', 'name': 'ctest-left_vn-71500075', 'tenant_id': '5a616162254d44cd9d78bdd1b735b3df', 'project_id': '5a616162254d44cd9d78bdd1b735b3df', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcRegr-57366020', 'ctest-left_vn-71500075'], 'port_security_enabled': True, 'description': ''}} 2025-07-31 04:18:30,716 - DEBUG - Response for create_subnet : {'subnet': {'id': '0c67ede3-821b-4c6b-ac8a-1c5dd07bb876', 'name': '', 'tenant_id': '5a616162254d44cd9d78bdd1b735b3df', 'network_id': 'b97208ee-837c-48c8-9ead-c2693baa242c', 'ip_version': 4, 'cidr': '86.14.153.128/26', 'allocation_pools': [{'start': '86.14.153.130', 'end': '86.14.153.190'}], 'gateway_ip': '86.14.153.129', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '86.14.153.130', 'tags': [], 'project_id': '5a616162254d44cd9d78bdd1b735b3df'}} 2025-07-31 04:18:30,741 - INFO - Created VN ctest-left_vn-71500075 2025-07-31 04:18:30,764 - DEBUG - VN ctest-left_vn-71500075 UUID is b97208ee-837c-48c8-9ead-c2693baa242c 2025-07-31 04:18:30,880 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:18:30,890 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:18:30,914 - DEBUG - Requesting: http://10.0.0.41:8082/project/5a616162-254d-44cd-9d78-bdd1b735b3df 2025-07-31 04:18:31,067 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/b97208ee-837c-48c8-9ead-c2693baa242c 2025-07-31 04:18:31,087 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/b97208ee-837c-48c8-9ead-c2693baa242c 2025-07-31 04:18:31,104 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/c6a9347b-2108-44c3-bbb5-f135aaedd51b 2025-07-31 04:18:31,119 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/c6a9347b-2108-44c3-bbb5-f135aaedd51b 2025-07-31 04:18:31,131 - DEBUG - Requesting: http://10.0.0.41:8082/route-target/1ff839a7-c31b-4049-99c5-0eb45b78a627 2025-07-31 04:18:31,151 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-31 04:18:31,151 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/b97208ee-837c-48c8-9ead-c2693baa242c 2025-07-31 04:18:31,173 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/c6a9347b-2108-44c3-bbb5-f135aaedd51b 2025-07-31 04:18:31,210 - INFO - Verified VN network id 12 for VN b97208ee-837c-48c8-9ead-c2693baa242c 2025-07-31 04:18:31,211 - INFO - Verifications in API Server for VN ctest-left_vn-71500075 passed 2025-07-31 04:18:31,211 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/b97208ee-837c-48c8-9ead-c2693baa242c 2025-07-31 04:18:31,233 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/c6a9347b-2108-44c3-bbb5-f135aaedd51b 2025-07-31 04:18:31,254 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/c6a9347b-2108-44c3-bbb5-f135aaedd51b 2025-07-31 04:18:31,271 - DEBUG - Requesting: http://10.0.0.41:8082/route-target/1ff839a7-c31b-4049-99c5-0eb45b78a627 2025-07-31 04:18:31,318 - DEBUG - Control-node 10.0.0.41 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-57366020:ctest-left_vn-71500075', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-57366020', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '13362752864862881992', 'uuid-lslong': '11434008786191918124'}, 'enable': 'true', 'created': '2025-07-31T04:18:30', 'last-modified': '2025-07-31T04:18:30', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.671439'} 2025-07-31 04:18:31,339 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-31 04:18:31,350 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-57366020:ctest-left_vn-71500075', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-57366020', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '13362752864862881992', 'uuid-lslong': '11434008786191918124'}, 'enable': 'true', 'created': '2025-07-31T04:18:30', 'last-modified': '2025-07-31T04:18:30', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.698691'} 2025-07-31 04:18:31,364 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-31 04:18:31,375 - DEBUG - Control-node 10.0.0.61 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-57366020:ctest-left_vn-71500075', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-57366020', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '13362752864862881992', 'uuid-lslong': '11434008786191918124'}, 'enable': 'true', 'created': '2025-07-31T04:18:30', 'last-modified': '2025-07-31T04:18:30', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.744149'} 2025-07-31 04:18:31,396 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-31 04:18:31,396 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-left_vn-71500075 passed 2025-07-31 04:18:31,397 - DEBUG - ====Verifying policy data for ctest-left_vn-71500075 in API_Server ====== 2025-07-31 04:18:31,397 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:18:31,408 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:18:31,429 - DEBUG - Requesting: http://10.0.0.41:8082/project/5a616162-254d-44cd-9d78-bdd1b735b3df 2025-07-31 04:18:31,554 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/b97208ee-837c-48c8-9ead-c2693baa242c 2025-07-31 04:18:31,566 - DEBUG - =>VN ctest-left_vn-71500075 has no policy to be verified 2025-07-31 04:18:31,566 - DEBUG - Verifying the vn in opserver 2025-07-31 04:18:31,566 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-57366020:ctest-left_vn-71500075 virtual network link through opserver 10.0.0.41 2025-07-31 04:18:31,566 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/virtual-networks 2025-07-31 04:18:31,582 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-57366020:ctest-left_vn-71500075', 'href': 'http://10.0.0.41:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-57366020:ctest-left_vn-71500075?flat'} 2025-07-31 04:18:31,582 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-57366020:ctest-left_vn-71500075 virtual network link through opserver 10.0.0.38 2025-07-31 04:18:31,582 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-07-31 04:18:31,604 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-57366020:ctest-left_vn-71500075', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-57366020:ctest-left_vn-71500075?flat'} 2025-07-31 04:18:31,604 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-57366020:ctest-left_vn-71500075 virtual network link through opserver 10.0.0.61 2025-07-31 04:18:31,604 - DEBUG - Requesting: http://10.0.0.61:8081/analytics/uves/virtual-networks 2025-07-31 04:18:31,619 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-57366020:ctest-left_vn-71500075', 'href': 'http://10.0.0.61:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-57366020:ctest-left_vn-71500075?flat'} 2025-07-31 04:18:31,619 - INFO - Validated that VN default-domain:ctest-TestSvcRegr-57366020:ctest-left_vn-71500075 is found in opserver 2025-07-31 04:18:31,654 - DEBUG - Do not have enough data to verify VN in agent 2025-07-31 04:18:31,674 - DEBUG - VRF ids for VN ctest-left_vn-71500075: {} 2025-07-31 04:18:31,981 - DEBUG - Response for create_network : {'network': {'id': '70bd8de7-29ec-40a5-9b54-bd20621e2bce', 'name': 'ctest-right_vn-24873194', 'tenant_id': '5a616162254d44cd9d78bdd1b735b3df', 'project_id': '5a616162254d44cd9d78bdd1b735b3df', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcRegr-57366020', 'ctest-right_vn-24873194'], 'port_security_enabled': True, 'description': ''}} 2025-07-31 04:18:32,311 - DEBUG - Response for create_subnet : {'subnet': {'id': '4fddd8de-214e-49e5-a4a2-8bed02a37136', 'name': '', 'tenant_id': '5a616162254d44cd9d78bdd1b735b3df', 'network_id': '70bd8de7-29ec-40a5-9b54-bd20621e2bce', 'ip_version': 4, 'cidr': '86.128.46.64/26', 'allocation_pools': [{'start': '86.128.46.66', 'end': '86.128.46.126'}], 'gateway_ip': '86.128.46.65', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '86.128.46.66', 'tags': [], 'project_id': '5a616162254d44cd9d78bdd1b735b3df'}} 2025-07-31 04:18:32,338 - INFO - Created VN ctest-right_vn-24873194 2025-07-31 04:18:32,356 - DEBUG - VN ctest-right_vn-24873194 UUID is 70bd8de7-29ec-40a5-9b54-bd20621e2bce 2025-07-31 04:18:32,457 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:18:32,468 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:18:32,490 - DEBUG - Requesting: http://10.0.0.41:8082/project/5a616162-254d-44cd-9d78-bdd1b735b3df 2025-07-31 04:18:32,610 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/70bd8de7-29ec-40a5-9b54-bd20621e2bce 2025-07-31 04:18:32,625 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/70bd8de7-29ec-40a5-9b54-bd20621e2bce 2025-07-31 04:18:32,639 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/890f7a42-9d64-47f2-a5bb-bd63272b0487 2025-07-31 04:18:32,652 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/890f7a42-9d64-47f2-a5bb-bd63272b0487 2025-07-31 04:18:32,661 - DEBUG - Requesting: http://10.0.0.41:8082/route-target/a16f47b1-cee4-4953-a3cc-ad9b8e7f9051 2025-07-31 04:18:32,670 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-31 04:18:32,670 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/70bd8de7-29ec-40a5-9b54-bd20621e2bce 2025-07-31 04:18:32,683 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/890f7a42-9d64-47f2-a5bb-bd63272b0487 2025-07-31 04:18:32,703 - INFO - Verified VN network id 13 for VN 70bd8de7-29ec-40a5-9b54-bd20621e2bce 2025-07-31 04:18:32,703 - INFO - Verifications in API Server for VN ctest-right_vn-24873194 passed 2025-07-31 04:18:32,703 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/70bd8de7-29ec-40a5-9b54-bd20621e2bce 2025-07-31 04:18:32,714 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/890f7a42-9d64-47f2-a5bb-bd63272b0487 2025-07-31 04:18:32,724 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/890f7a42-9d64-47f2-a5bb-bd63272b0487 2025-07-31 04:18:32,732 - DEBUG - Requesting: http://10.0.0.41:8082/route-target/a16f47b1-cee4-4953-a3cc-ad9b8e7f9051 2025-07-31 04:18:32,753 - DEBUG - Control-node 10.0.0.41 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-57366020:ctest-right_vn-24873194', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-57366020', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '8123805326826553509', 'uuid-lslong': '11192778920705272782'}, 'enable': 'true', 'created': '2025-07-31T04:18:31', 'last-modified': '2025-07-31T04:18:32', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.509236'} 2025-07-31 04:18:32,762 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-31 04:18:32,769 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-57366020:ctest-right_vn-24873194', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-57366020', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '8123805326826553509', 'uuid-lslong': '11192778920705272782'}, 'enable': 'true', 'created': '2025-07-31T04:18:31', 'last-modified': '2025-07-31T04:18:32', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.529808'} 2025-07-31 04:18:32,778 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-31 04:18:32,786 - DEBUG - Control-node 10.0.0.61 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-57366020:ctest-right_vn-24873194', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-57366020', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '8123805326826553509', 'uuid-lslong': '11192778920705272782'}, 'enable': 'true', 'created': '2025-07-31T04:18:31', 'last-modified': '2025-07-31T04:18:32', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.551729'} 2025-07-31 04:18:32,795 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-31 04:18:32,795 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-right_vn-24873194 passed 2025-07-31 04:18:32,796 - DEBUG - ====Verifying policy data for ctest-right_vn-24873194 in API_Server ====== 2025-07-31 04:18:32,796 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:18:32,806 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:18:32,831 - DEBUG - Requesting: http://10.0.0.41:8082/project/5a616162-254d-44cd-9d78-bdd1b735b3df 2025-07-31 04:18:32,985 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/70bd8de7-29ec-40a5-9b54-bd20621e2bce 2025-07-31 04:18:33,007 - DEBUG - =>VN ctest-right_vn-24873194 has no policy to be verified 2025-07-31 04:18:33,007 - DEBUG - Verifying the vn in opserver 2025-07-31 04:18:33,007 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-57366020:ctest-right_vn-24873194 virtual network link through opserver 10.0.0.41 2025-07-31 04:18:33,007 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/virtual-networks 2025-07-31 04:18:33,026 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-57366020:ctest-right_vn-24873194', 'href': 'http://10.0.0.41:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-57366020:ctest-right_vn-24873194?flat'} 2025-07-31 04:18:33,026 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-57366020:ctest-right_vn-24873194 virtual network link through opserver 10.0.0.38 2025-07-31 04:18:33,026 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-07-31 04:18:33,043 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-57366020:ctest-right_vn-24873194', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-57366020:ctest-right_vn-24873194?flat'} 2025-07-31 04:18:33,043 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-57366020:ctest-right_vn-24873194 virtual network link through opserver 10.0.0.61 2025-07-31 04:18:33,043 - DEBUG - Requesting: http://10.0.0.61:8081/analytics/uves/virtual-networks 2025-07-31 04:18:33,059 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-57366020:ctest-right_vn-24873194', 'href': 'http://10.0.0.61:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-57366020:ctest-right_vn-24873194?flat'} 2025-07-31 04:18:33,059 - INFO - Validated that VN default-domain:ctest-TestSvcRegr-57366020:ctest-right_vn-24873194 is found in opserver 2025-07-31 04:18:33,093 - DEBUG - Do not have enough data to verify VN in agent 2025-07-31 04:18:33,116 - DEBUG - VRF ids for VN ctest-right_vn-24873194: {} 2025-07-31 04:18:33,471 - DEBUG - Response for create_network : {'network': {'id': '8931dc12-9220-4f92-bdf3-6e286d6fba2e', 'name': 'ctest-trans_left_vn-07057192', 'tenant_id': '5a616162254d44cd9d78bdd1b735b3df', 'project_id': '5a616162254d44cd9d78bdd1b735b3df', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcRegr-57366020', 'ctest-trans_left_vn-07057192'], 'port_security_enabled': True, 'description': ''}} 2025-07-31 04:18:33,754 - DEBUG - Response for create_subnet : {'subnet': {'id': '2987585c-0b5d-48ea-b066-d91ca46b5cec', 'name': '', 'tenant_id': '5a616162254d44cd9d78bdd1b735b3df', 'network_id': '8931dc12-9220-4f92-bdf3-6e286d6fba2e', 'ip_version': 4, 'cidr': '11.106.24.64/26', 'allocation_pools': [{'start': '11.106.24.66', 'end': '11.106.24.126'}], 'gateway_ip': '11.106.24.65', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '11.106.24.66', 'tags': [], 'project_id': '5a616162254d44cd9d78bdd1b735b3df'}} 2025-07-31 04:18:33,783 - INFO - Created VN ctest-trans_left_vn-07057192 2025-07-31 04:18:33,798 - DEBUG - VN ctest-trans_left_vn-07057192 UUID is 8931dc12-9220-4f92-bdf3-6e286d6fba2e 2025-07-31 04:18:33,887 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:18:33,896 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:18:33,915 - DEBUG - Requesting: http://10.0.0.41:8082/project/5a616162-254d-44cd-9d78-bdd1b735b3df 2025-07-31 04:18:34,050 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/8931dc12-9220-4f92-bdf3-6e286d6fba2e 2025-07-31 04:18:34,068 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/8931dc12-9220-4f92-bdf3-6e286d6fba2e 2025-07-31 04:18:34,081 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/5c86f0f3-8582-4d26-8de5-b74b2610b863 2025-07-31 04:18:34,093 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/5c86f0f3-8582-4d26-8de5-b74b2610b863 2025-07-31 04:18:34,103 - DEBUG - Requesting: http://10.0.0.41:8082/route-target/bb3971ab-0355-4e5d-be6d-1fce2781c5d2 2025-07-31 04:18:34,114 - DEBUG - Route Targets: ['target:64512:8000008'] 2025-07-31 04:18:34,114 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/8931dc12-9220-4f92-bdf3-6e286d6fba2e 2025-07-31 04:18:34,128 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/5c86f0f3-8582-4d26-8de5-b74b2610b863 2025-07-31 04:18:34,150 - INFO - Verified VN network id 14 for VN 8931dc12-9220-4f92-bdf3-6e286d6fba2e 2025-07-31 04:18:34,150 - INFO - Verifications in API Server for VN ctest-trans_left_vn-07057192 passed 2025-07-31 04:18:34,150 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/8931dc12-9220-4f92-bdf3-6e286d6fba2e 2025-07-31 04:18:34,168 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/5c86f0f3-8582-4d26-8de5-b74b2610b863 2025-07-31 04:18:34,180 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/5c86f0f3-8582-4d26-8de5-b74b2610b863 2025-07-31 04:18:34,195 - DEBUG - Requesting: http://10.0.0.41:8082/route-target/bb3971ab-0355-4e5d-be6d-1fce2781c5d2 2025-07-31 04:18:34,223 - DEBUG - Control-node 10.0.0.41 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-57366020:ctest-trans_left_vn-07057192', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-57366020', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '9885924629374062482', 'uuid-lslong': '13687404812422855214'}, 'enable': 'true', 'created': '2025-07-31T04:18:33', 'last-modified': '2025-07-31T04:18:33', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.532933'} 2025-07-31 04:18:34,238 - DEBUG - Route Targets: ['target:64512:8000008'] 2025-07-31 04:18:34,247 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-57366020:ctest-trans_left_vn-07057192', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-57366020', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '9885924629374062482', 'uuid-lslong': '13687404812422855214'}, 'enable': 'true', 'created': '2025-07-31T04:18:33', 'last-modified': '2025-07-31T04:18:33', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.558746'} 2025-07-31 04:18:34,260 - DEBUG - Route Targets: ['target:64512:8000008'] 2025-07-31 04:18:34,268 - DEBUG - Control-node 10.0.0.61 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-57366020:ctest-trans_left_vn-07057192', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-57366020', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '9885924629374062482', 'uuid-lslong': '13687404812422855214'}, 'enable': 'true', 'created': '2025-07-31T04:18:33', 'last-modified': '2025-07-31T04:18:33', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.578650'} 2025-07-31 04:18:34,280 - DEBUG - Route Targets: ['target:64512:8000008'] 2025-07-31 04:18:34,281 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-trans_left_vn-07057192 passed 2025-07-31 04:18:34,281 - DEBUG - ====Verifying policy data for ctest-trans_left_vn-07057192 in API_Server ====== 2025-07-31 04:18:34,281 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:18:34,297 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:18:34,327 - DEBUG - Requesting: http://10.0.0.41:8082/project/5a616162-254d-44cd-9d78-bdd1b735b3df 2025-07-31 04:18:34,469 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/8931dc12-9220-4f92-bdf3-6e286d6fba2e 2025-07-31 04:18:34,499 - DEBUG - =>VN ctest-trans_left_vn-07057192 has no policy to be verified 2025-07-31 04:18:34,499 - DEBUG - Verifying the vn in opserver 2025-07-31 04:18:34,499 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-57366020:ctest-trans_left_vn-07057192 virtual network link through opserver 10.0.0.41 2025-07-31 04:18:34,500 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/virtual-networks 2025-07-31 04:18:34,520 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-57366020:ctest-trans_left_vn-07057192', 'href': 'http://10.0.0.41:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-57366020:ctest-trans_left_vn-07057192?flat'} 2025-07-31 04:18:34,520 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-57366020:ctest-trans_left_vn-07057192 virtual network link through opserver 10.0.0.38 2025-07-31 04:18:34,520 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-07-31 04:18:34,535 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-57366020:ctest-trans_left_vn-07057192', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-57366020:ctest-trans_left_vn-07057192?flat'} 2025-07-31 04:18:34,535 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-57366020:ctest-trans_left_vn-07057192 virtual network link through opserver 10.0.0.61 2025-07-31 04:18:34,535 - DEBUG - Requesting: http://10.0.0.61:8081/analytics/uves/virtual-networks 2025-07-31 04:18:34,551 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-57366020:ctest-trans_left_vn-07057192', 'href': 'http://10.0.0.61:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-57366020:ctest-trans_left_vn-07057192?flat'} 2025-07-31 04:18:34,551 - INFO - Validated that VN default-domain:ctest-TestSvcRegr-57366020:ctest-trans_left_vn-07057192 is found in opserver 2025-07-31 04:18:34,578 - DEBUG - Do not have enough data to verify VN in agent 2025-07-31 04:18:34,592 - DEBUG - VRF ids for VN ctest-trans_left_vn-07057192: {} 2025-07-31 04:18:34,960 - DEBUG - Response for create_network : {'network': {'id': '0ad0979b-2c9f-47dd-8a59-a3476fe7f52c', 'name': 'ctest-trans_right_vn-54937609', 'tenant_id': '5a616162254d44cd9d78bdd1b735b3df', 'project_id': '5a616162254d44cd9d78bdd1b735b3df', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcRegr-57366020', 'ctest-trans_right_vn-54937609'], 'port_security_enabled': True, 'description': ''}} 2025-07-31 04:18:35,278 - DEBUG - Response for create_subnet : {'subnet': {'id': 'da0bc8b8-c3e6-419a-b57f-d00d0b1a274d', 'name': '', 'tenant_id': '5a616162254d44cd9d78bdd1b735b3df', 'network_id': '0ad0979b-2c9f-47dd-8a59-a3476fe7f52c', 'ip_version': 4, 'cidr': '134.248.219.192/26', 'allocation_pools': [{'start': '134.248.219.194', 'end': '134.248.219.254'}], 'gateway_ip': '134.248.219.193', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '134.248.219.194', 'tags': [], 'project_id': '5a616162254d44cd9d78bdd1b735b3df'}} 2025-07-31 04:18:35,305 - INFO - Created VN ctest-trans_right_vn-54937609 2025-07-31 04:18:35,321 - DEBUG - VN ctest-trans_right_vn-54937609 UUID is 0ad0979b-2c9f-47dd-8a59-a3476fe7f52c 2025-07-31 04:18:35,448 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:18:35,462 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:18:35,487 - DEBUG - Requesting: http://10.0.0.41:8082/project/5a616162-254d-44cd-9d78-bdd1b735b3df 2025-07-31 04:18:35,621 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/0ad0979b-2c9f-47dd-8a59-a3476fe7f52c 2025-07-31 04:18:35,639 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/0ad0979b-2c9f-47dd-8a59-a3476fe7f52c 2025-07-31 04:18:35,655 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/c4edae71-133f-43f9-bebe-483e4cd098e5 2025-07-31 04:18:35,673 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/c4edae71-133f-43f9-bebe-483e4cd098e5 2025-07-31 04:18:35,690 - DEBUG - Requesting: http://10.0.0.41:8082/route-target/45c594c1-9a11-49e4-ae00-aa1c7aea523b 2025-07-31 04:18:35,710 - DEBUG - Route Targets: ['target:64512:8000009'] 2025-07-31 04:18:35,710 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/0ad0979b-2c9f-47dd-8a59-a3476fe7f52c 2025-07-31 04:18:35,735 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/c4edae71-133f-43f9-bebe-483e4cd098e5 2025-07-31 04:18:35,763 - INFO - Verified VN network id 15 for VN 0ad0979b-2c9f-47dd-8a59-a3476fe7f52c 2025-07-31 04:18:35,764 - INFO - Verifications in API Server for VN ctest-trans_right_vn-54937609 passed 2025-07-31 04:18:35,764 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/0ad0979b-2c9f-47dd-8a59-a3476fe7f52c 2025-07-31 04:18:35,796 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/c4edae71-133f-43f9-bebe-483e4cd098e5 2025-07-31 04:18:35,819 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/c4edae71-133f-43f9-bebe-483e4cd098e5 2025-07-31 04:18:35,832 - DEBUG - Requesting: http://10.0.0.41:8082/route-target/45c594c1-9a11-49e4-ae00-aa1c7aea523b 2025-07-31 04:18:35,854 - DEBUG - Control-node 10.0.0.41 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-57366020:ctest-trans_right_vn-54937609', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-57366020', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '779289428259456989', 'uuid-lslong': '9969178777376781612'}, 'enable': 'true', 'created': '2025-07-31T04:18:34', 'last-modified': '2025-07-31T04:18:35', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.648479'} 2025-07-31 04:18:35,867 - DEBUG - Route Targets: ['target:64512:8000009'] 2025-07-31 04:18:35,879 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-57366020:ctest-trans_right_vn-54937609', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-57366020', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '779289428259456989', 'uuid-lslong': '9969178777376781612'}, 'enable': 'true', 'created': '2025-07-31T04:18:34', 'last-modified': '2025-07-31T04:18:35', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.675023'} 2025-07-31 04:18:35,892 - DEBUG - Route Targets: ['target:64512:8000009'] 2025-07-31 04:18:35,905 - DEBUG - Control-node 10.0.0.61 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-57366020:ctest-trans_right_vn-54937609', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-57366020', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '779289428259456989', 'uuid-lslong': '9969178777376781612'}, 'enable': 'true', 'created': '2025-07-31T04:18:34', 'last-modified': '2025-07-31T04:18:35', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.701068'} 2025-07-31 04:18:35,918 - DEBUG - Route Targets: ['target:64512:8000009'] 2025-07-31 04:18:35,918 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-trans_right_vn-54937609 passed 2025-07-31 04:18:35,919 - DEBUG - ====Verifying policy data for ctest-trans_right_vn-54937609 in API_Server ====== 2025-07-31 04:18:35,919 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:18:35,935 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:18:35,974 - DEBUG - Requesting: http://10.0.0.41:8082/project/5a616162-254d-44cd-9d78-bdd1b735b3df 2025-07-31 04:18:36,169 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/0ad0979b-2c9f-47dd-8a59-a3476fe7f52c 2025-07-31 04:18:36,183 - DEBUG - =>VN ctest-trans_right_vn-54937609 has no policy to be verified 2025-07-31 04:18:36,183 - DEBUG - Verifying the vn in opserver 2025-07-31 04:18:36,183 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-57366020:ctest-trans_right_vn-54937609 virtual network link through opserver 10.0.0.41 2025-07-31 04:18:36,184 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/virtual-networks 2025-07-31 04:18:36,200 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-57366020:ctest-trans_right_vn-54937609', 'href': 'http://10.0.0.41:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-57366020:ctest-trans_right_vn-54937609?flat'} 2025-07-31 04:18:36,200 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-57366020:ctest-trans_right_vn-54937609 virtual network link through opserver 10.0.0.38 2025-07-31 04:18:36,200 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-07-31 04:18:36,221 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-57366020:ctest-trans_right_vn-54937609', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-57366020:ctest-trans_right_vn-54937609?flat'} 2025-07-31 04:18:36,221 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-57366020:ctest-trans_right_vn-54937609 virtual network link through opserver 10.0.0.61 2025-07-31 04:18:36,221 - DEBUG - Requesting: http://10.0.0.61:8081/analytics/uves/virtual-networks 2025-07-31 04:18:36,237 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-57366020:ctest-trans_right_vn-54937609', 'href': 'http://10.0.0.61:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-57366020:ctest-trans_right_vn-54937609?flat'} 2025-07-31 04:18:36,238 - INFO - Validated that VN default-domain:ctest-TestSvcRegr-57366020:ctest-trans_right_vn-54937609 is found in opserver 2025-07-31 04:18:36,283 - DEBUG - Do not have enough data to verify VN in agent 2025-07-31 04:18:36,301 - DEBUG - VRF ids for VN ctest-trans_right_vn-54937609: {} 2025-07-31 04:18:37,634 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3732-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3732-1) 2025-07-31 04:18:39,041 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3732-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3732-2) 2025-07-31 04:18:39,041 - DEBUG - Creating service template: ['default-domain', 'ctest-service_template_1-67387577'] 2025-07-31 04:18:39,360 - DEBUG - Service template: ['default-domain', 'ctest-service_template_1-67387577'] created succesfully 2025-07-31 04:18:39,809 - DEBUG - Image by name tiny_trans_fw not found 2025-07-31 04:18:39,809 - DEBUG - Installing image tiny_trans_fw 2025-07-31 04:18:39,809 - DEBUG - Image info {'name': 'tinycore-net-transparent-fw.qcow2.gz', 'webserver': '', 'location': '/images/tinycore/', 'username': 'tc', 'password': 'secret', 'type': 'os', 'flavor': 'contrail_flavor_tiny', 'params': '--container-format bare --disk-format vmdk --property vmware_disktype="sparse" --property vmware_adaptertype="ide" --property hypervisor_type=qemu', 'name_docker': 'phusion-baseimage-enablesshd'} 2025-07-31 04:18:39,810 - DEBUG - Image build path http://nexus.opensdn.io/repository///images/tinycore//tinycore-net-transparent-fw.qcow2.gz 2025-07-31 04:18:39,810 - DEBUG - Download image from web http://nexus.opensdn.io/repository///images/tinycore//tinycore-net-transparent-fw.qcow2.gz 2025-07-31 04:18:39,828 - DEBUG - Image download (local=True) (proxy=None): Execute cmd: wget -nv http://nexus.opensdn.io/repository///images/tinycore//tinycore-net-transparent-fw.qcow2.gz -O /contrail-test/images/tinycore-net-transparent-fw.qcow2.gz 2025-07-31 04:18:39,861 - DEBUG - Image has been downloaded to /contrail-test/images/tinycore-net-transparent-fw.qcow2.gz 2025-07-31 04:18:39,862 - DEBUG - Unzip image 2025-07-31 04:18:39,862 - DEBUG - Image download (local=True) (proxy=None): Execute cmd: gunzip -f /contrail-test/images/tinycore-net-transparent-fw.qcow2.gz 2025-07-31 04:18:39,879 - DEBUG - Try to create image in user context 2025-07-31 04:18:39,952 - DEBUG - Deleting service template: ['default-domain', 'ctest-service_template_1-67387577'] 2025-07-31 04:18:40,048 - INFO - Service template: ['default-domain', 'ctest-service_template_1-67387577'] deleted successfully. 2025-07-31 04:18:40,049 - INFO - Skip interface_detach for VM ctest-right_vm-51308011 in state BUILD 2025-07-31 04:18:40,049 - INFO - Deleting VM ctest-right_vm-51308011 2025-07-31 04:18:40,049 - ERROR - VM ctest-right_vm-51308011 has failed to come up 2025-07-31 04:18:40,050 - ERROR - Nova failed to get host of the VM 2025-07-31 04:18:40,050 - ERROR - VM ctest-right_vm-51308011 has failed to come up 2025-07-31 04:18:40,050 - ERROR - Nova failed to get host of the VM 2025-07-31 04:18:40,174 - INFO - Skip interface_detach for VM ctest-left_vm-30507842 in state BUILD 2025-07-31 04:18:40,175 - INFO - Deleting VM ctest-left_vm-30507842 2025-07-31 04:18:40,175 - ERROR - VM ctest-left_vm-30507842 has failed to come up 2025-07-31 04:18:40,175 - ERROR - Nova failed to get host of the VM 2025-07-31 04:18:40,175 - ERROR - VM ctest-left_vm-30507842 has failed to come up 2025-07-31 04:18:40,175 - ERROR - Nova failed to get host of the VM 2025-07-31 04:18:40,236 - INFO - Deleting VN ctest-trans_right_vn-54937609 2025-07-31 04:18:40,409 - DEBUG - Response for deleting network () 2025-07-31 04:18:40,410 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/c4edae71-133f-43f9-bebe-483e4cd098e5 2025-07-31 04:18:40,416 - DEBUG - Response Code: 404 2025-07-31 04:18:40,416 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:18:40,423 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:18:40,451 - DEBUG - Requesting: http://10.0.0.41:8082/project/5a616162-254d-44cd-9d78-bdd1b735b3df 2025-07-31 04:18:40,566 - INFO - Validated that VN ctest-trans_right_vn-54937609 is not found in API Server 2025-07-31 04:18:40,581 - DEBUG - VN ctest-trans_right_vn-54937609 is not present in Agent 10.0.0.27 2025-07-31 04:18:40,616 - DEBUG - VN ctest-trans_right_vn-54937609 is not present in Agent 10.0.0.57 2025-07-31 04:18:40,616 - INFO - Validated that VN ctest-trans_right_vn-54937609 is not in any agent 2025-07-31 04:18:40,702 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-trans_right_vn-54937609 info 2025-07-31 04:18:40,702 - INFO - Deleting VN ctest-trans_left_vn-07057192 2025-07-31 04:18:40,888 - DEBUG - Response for deleting network () 2025-07-31 04:18:40,888 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/5c86f0f3-8582-4d26-8de5-b74b2610b863 2025-07-31 04:18:40,907 - DEBUG - Response Code: 404 2025-07-31 04:18:40,907 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:18:40,915 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:18:40,939 - DEBUG - Requesting: http://10.0.0.41:8082/project/5a616162-254d-44cd-9d78-bdd1b735b3df 2025-07-31 04:18:41,020 - INFO - Validated that VN ctest-trans_left_vn-07057192 is not found in API Server 2025-07-31 04:18:41,034 - DEBUG - VN ctest-trans_left_vn-07057192 is not present in Agent 10.0.0.27 2025-07-31 04:18:41,061 - DEBUG - VN ctest-trans_left_vn-07057192 is not present in Agent 10.0.0.57 2025-07-31 04:18:41,061 - INFO - Validated that VN ctest-trans_left_vn-07057192 is not in any agent 2025-07-31 04:18:41,125 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-trans_left_vn-07057192 info 2025-07-31 04:18:41,125 - INFO - Deleting VN ctest-right_vn-24873194 2025-07-31 04:18:41,176 - DEBUG - VN 70bd8de7-29ec-40a5-9b54-bd20621e2bce still in use: Unable to complete operation on network 70bd8de7-29ec-40a5-9b54-bd20621e2bce. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-c83152be-777c-4553-93d7-661cf28f74d1'] 2025-07-31 04:18:41,176 - WARNING - Deleting VN ctest-right_vn-24873194 failed..Will retry 2025-07-31 04:18:43,406 - DEBUG - Response for deleting network () 2025-07-31 04:18:43,406 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/890f7a42-9d64-47f2-a5bb-bd63272b0487 2025-07-31 04:18:43,417 - DEBUG - Response Code: 404 2025-07-31 04:18:43,417 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:18:43,424 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:18:43,480 - DEBUG - Requesting: http://10.0.0.41:8082/project/5a616162-254d-44cd-9d78-bdd1b735b3df 2025-07-31 04:18:43,594 - INFO - Validated that VN ctest-right_vn-24873194 is not found in API Server 2025-07-31 04:18:43,609 - DEBUG - VN ctest-right_vn-24873194 is not present in Agent 10.0.0.27 2025-07-31 04:18:43,624 - DEBUG - VN ctest-right_vn-24873194 is not present in Agent 10.0.0.57 2025-07-31 04:18:43,624 - INFO - Validated that VN ctest-right_vn-24873194 is not in any agent 2025-07-31 04:18:43,678 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-right_vn-24873194 info 2025-07-31 04:18:43,679 - INFO - Deleting VN ctest-left_vn-71500075 2025-07-31 04:18:43,894 - DEBUG - Response for deleting network () 2025-07-31 04:18:43,894 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/c6a9347b-2108-44c3-bbb5-f135aaedd51b 2025-07-31 04:18:43,902 - DEBUG - Response Code: 404 2025-07-31 04:18:43,902 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:18:43,924 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:18:43,945 - DEBUG - Requesting: http://10.0.0.41:8082/project/5a616162-254d-44cd-9d78-bdd1b735b3df 2025-07-31 04:18:44,047 - INFO - Validated that VN ctest-left_vn-71500075 is not found in API Server 2025-07-31 04:18:44,064 - DEBUG - VN ctest-left_vn-71500075 is not present in Agent 10.0.0.27 2025-07-31 04:18:44,083 - DEBUG - VN ctest-left_vn-71500075 is not present in Agent 10.0.0.57 2025-07-31 04:18:44,083 - INFO - Validated that VN ctest-left_vn-71500075 is not in any agent 2025-07-31 04:18:44,134 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-left_vn-71500075 info 2025-07-31 04:18:44,134 - INFO - Deleting VN ctest-mgmt_vn-01374105 2025-07-31 04:18:44,326 - DEBUG - Response for deleting network () 2025-07-31 04:18:44,326 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/ad15d8ab-07e3-4f60-b272-4df0348892de 2025-07-31 04:18:44,333 - DEBUG - Response Code: 404 2025-07-31 04:18:44,333 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:18:44,342 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:18:44,369 - DEBUG - Requesting: http://10.0.0.41:8082/project/5a616162-254d-44cd-9d78-bdd1b735b3df 2025-07-31 04:18:44,465 - INFO - Validated that VN ctest-mgmt_vn-01374105 is not found in API Server 2025-07-31 04:18:44,479 - DEBUG - VN ctest-mgmt_vn-01374105 is not present in Agent 10.0.0.27 2025-07-31 04:18:44,492 - DEBUG - VN ctest-mgmt_vn-01374105 is not present in Agent 10.0.0.57 2025-07-31 04:18:44,492 - INFO - Validated that VN ctest-mgmt_vn-01374105 is not in any agent 2025-07-31 04:18:44,528 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-mgmt_vn-01374105 info 2025-07-31 04:18:45,883 - ERROR - 
Traceback (most recent call last):
  File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper
    result = function(self, *args, **kwargs)
  File "/contrail-test/tcutils/util.py", line 1041, in wrapper
    return f(self, *func_args, **func_kwargs)
  File "/contrail-test/scripts/svc_firewall/test_svc_fw.py", line 34, in test_svc_v2_transparent_datapath
    create_svms=True)
  File "/contrail-test/common/servicechain/verify.py", line 320, in verify_svc_chain
    ret_dict = svc_chain_info or self.config_svc_chain(*args, **kwargs)
  File "/contrail-test/common/servicechain/config.py", line 533, in config_svc_chain
    max_inst=max_inst)
  File "/contrail-test/common/servicechain/config.py", line 335, in create_service_vms
    zone=random.choice(valid_zones))
  File "/contrail-test/common/servicechain/config.py", line 161, in config_vm_only
    flavor=flavor, zone=zone, node_name=node_name,**kwargs)
  File "/contrail-test/common/servicechain/config.py", line 203, in config_vm
    flavor=flavor, vn_objs=vns, count=count, zone=zone,**kwargs))
  File "/usr/local/lib/python3.6/site-packages/testtools/testcase.py", line 756, in useFixture
    reraise(*exc_info)
  File "/usr/local/lib/python3.6/site-packages/testtools/_compat3x.py", line 16, in reraise
    raise exc_obj.with_traceback(exc_tb)
  File "/usr/local/lib/python3.6/site-packages/testtools/testcase.py", line 731, in useFixture
    fixture.setUp()
  File "/contrail-test/fixtures/vm_test.py", line 188, in setUp
    self.create()
  File "/contrail-test/fixtures/vm_test.py", line 240, in create
    fixed_ips=self.fixed_ips)
  File "/contrail-test/fixtures/openstack.py", line 118, in create_vm
    return self.nova_h.create_vm(vm_name=vm_name, image_name=image_name, vn_ids=vn_ids, **kwargs)
  File "/contrail-test/fixtures/nova_test.py", line 577, in create_vm
    image = self.get_image(image_name=image_name)
  File "/contrail-test/fixtures/nova_test.py", line 198, in get_image
    self._install_image(image_name=image_name)
  File "/contrail-test/fixtures/nova_test.py", line 338, in _install_image
    return self.copy_and_glance(build_path, image_name, params)
  File "/contrail-test/fixtures/nova_test.py", line 407, in copy_and_glance
    image_id = self.glance_h.create_image(generic_image_name, image_path_real, **params)
  File "/contrail-test/fixtures/glance_test.py", line 50, in create_image
    self.upload_image(obj['id'], filename)
  File "/contrail-test/fixtures/glance_test.py", line 54, in upload_image
    self.obj.images.upload(uuid, open(filename, 'rb'))
FileNotFoundError: [Errno 2] No such file or directory: '/contrail-test/images/tinycore-net-transparent-fw.qcow2'
2025-07-31 04:18:45,902 - DEBUG - No XMPP flaps were noticed during the test 2025-07-31 04:18:45,902 - INFO - 2025-07-31 04:18:45,902 - INFO - END TEST : test_svc_v2_transparent_datapath : FAILED[0:00:18] 2025-07-31 04:18:45,903 - INFO - -------------------------------------------------------------------------------- 2025-07-31 04:18:46,871 - INFO - Deleted project: ctest-TestSvcRegr-57366020, ID : 5a616162-254d-44cd-9d78-bdd1b735b3df 2025-07-31 04:18:47,277 - INFO - Domain Default found not creating 2025-07-31 04:18:47,468 - INFO - Project ctest-TestSvcRegr-85309723 not found, creating it 2025-07-31 04:18:48,058 - INFO - Created Project:ctest-TestSvcRegr-85309723, ID : b1960ad8-31ff-4683-883f-7e53e7827c36 2025-07-31 04:18:50,149 - INFO - ================================================================================ 2025-07-31 04:18:50,149 - INFO - STARTING TEST : test_svc_CEM_22032 2025-07-31 04:18:51,535 - DEBUG - Nothing to compare xmpp stats {'10.0.0.27': {'10.20.0.193': '0', '10.20.0.18': '0'}, '10.0.0.57': {'10.20.0.18': '0', '10.20.0.193': '0'}} with 2025-07-31 04:18:51,536 - INFO - Initial checks done. Running the testcase now 2025-07-31 04:18:51,536 - INFO - 2025-07-31 04:18:52,528 - DEBUG - Response for create_network : {'network': {'id': '8cc68515-66e7-4188-9d07-4f9c9b219dec', 'name': 'ctest-mgmt_vn-23925910', 'tenant_id': 'b1960ad831ff4683883f7e53e7827c36', 'project_id': 'b1960ad831ff4683883f7e53e7827c36', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcRegr-85309723', 'ctest-mgmt_vn-23925910'], 'port_security_enabled': True, 'description': ''}} 2025-07-31 04:18:52,792 - DEBUG - Response for create_subnet : {'subnet': {'id': 'e6d4fcaa-58bd-444b-8aa6-b63fad4d8686', 'name': '', 'tenant_id': 'b1960ad831ff4683883f7e53e7827c36', 'network_id': '8cc68515-66e7-4188-9d07-4f9c9b219dec', 'ip_version': 4, 'cidr': '59.16.154.128/26', 'allocation_pools': [{'start': '59.16.154.130', 'end': '59.16.154.190'}], 'gateway_ip': '59.16.154.129', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '59.16.154.130', 'tags': [], 'project_id': 'b1960ad831ff4683883f7e53e7827c36'}} 2025-07-31 04:18:52,815 - INFO - Created VN ctest-mgmt_vn-23925910 2025-07-31 04:18:52,829 - DEBUG - VN ctest-mgmt_vn-23925910 UUID is 8cc68515-66e7-4188-9d07-4f9c9b219dec 2025-07-31 04:18:52,902 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:18:53,203 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:18:53,239 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:18:53,257 - DEBUG - Requesting: http://10.0.0.41:8082/project/b1960ad8-31ff-4683-883f-7e53e7827c36 2025-07-31 04:18:53,359 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/8cc68515-66e7-4188-9d07-4f9c9b219dec 2025-07-31 04:18:53,383 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/8cc68515-66e7-4188-9d07-4f9c9b219dec 2025-07-31 04:18:53,400 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/ecfec333-dac3-4ff7-a0aa-1b0e7b7e9a13 2025-07-31 04:18:53,411 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/ecfec333-dac3-4ff7-a0aa-1b0e7b7e9a13 2025-07-31 04:18:53,420 - DEBUG - Requesting: http://10.0.0.41:8082/route-target/4381723a-a0ac-45e9-abb6-81610fe6b4ab 2025-07-31 04:18:53,431 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-31 04:18:53,431 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/8cc68515-66e7-4188-9d07-4f9c9b219dec 2025-07-31 04:18:53,443 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/ecfec333-dac3-4ff7-a0aa-1b0e7b7e9a13 2025-07-31 04:18:53,459 - INFO - Verified VN network id 11 for VN 8cc68515-66e7-4188-9d07-4f9c9b219dec 2025-07-31 04:18:53,459 - INFO - Verifications in API Server for VN ctest-mgmt_vn-23925910 passed 2025-07-31 04:18:53,459 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/8cc68515-66e7-4188-9d07-4f9c9b219dec 2025-07-31 04:18:53,474 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/ecfec333-dac3-4ff7-a0aa-1b0e7b7e9a13 2025-07-31 04:18:53,493 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/ecfec333-dac3-4ff7-a0aa-1b0e7b7e9a13 2025-07-31 04:18:53,504 - DEBUG - Requesting: http://10.0.0.41:8082/route-target/4381723a-a0ac-45e9-abb6-81610fe6b4ab 2025-07-31 04:18:53,525 - DEBUG - Control-node 10.0.0.41 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-85309723:ctest-mgmt_vn-23925910', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-85309723', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '10143941537665859976', 'uuid-lslong': '11315100122827824620'}, 'enable': 'true', 'created': '2025-07-31T04:18:52', 'last-modified': '2025-07-31T04:18:52', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.791337'} 2025-07-31 04:18:53,534 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-31 04:18:53,544 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-85309723:ctest-mgmt_vn-23925910', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-85309723', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '10143941537665859976', 'uuid-lslong': '11315100122827824620'}, 'enable': 'true', 'created': '2025-07-31T04:18:52', 'last-modified': '2025-07-31T04:18:52', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.814205'} 2025-07-31 04:18:53,552 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-31 04:18:53,559 - DEBUG - Control-node 10.0.0.61 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-85309723:ctest-mgmt_vn-23925910', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-85309723', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '10143941537665859976', 'uuid-lslong': '11315100122827824620'}, 'enable': 'true', 'created': '2025-07-31T04:18:52', 'last-modified': '2025-07-31T04:18:52', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.829305'} 2025-07-31 04:18:53,567 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-31 04:18:53,567 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-mgmt_vn-23925910 passed 2025-07-31 04:18:53,567 - DEBUG - ====Verifying policy data for ctest-mgmt_vn-23925910 in API_Server ====== 2025-07-31 04:18:53,567 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:18:53,574 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:18:53,587 - DEBUG - Requesting: http://10.0.0.41:8082/project/b1960ad8-31ff-4683-883f-7e53e7827c36 2025-07-31 04:18:53,690 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/8cc68515-66e7-4188-9d07-4f9c9b219dec 2025-07-31 04:18:53,710 - DEBUG - =>VN ctest-mgmt_vn-23925910 has no policy to be verified 2025-07-31 04:18:53,710 - DEBUG - Verifying the vn in opserver 2025-07-31 04:18:53,710 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-85309723:ctest-mgmt_vn-23925910 virtual network link through opserver 10.0.0.41 2025-07-31 04:18:53,710 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/virtual-networks 2025-07-31 04:18:54,018 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/virtual-networks 2025-07-31 04:18:54,114 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-85309723:ctest-mgmt_vn-23925910', 'href': 'http://10.0.0.41:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-85309723:ctest-mgmt_vn-23925910?flat'} 2025-07-31 04:18:54,115 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-85309723:ctest-mgmt_vn-23925910 virtual network link through opserver 10.0.0.38 2025-07-31 04:18:54,115 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-07-31 04:18:54,429 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-07-31 04:18:54,493 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-85309723:ctest-mgmt_vn-23925910', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-85309723:ctest-mgmt_vn-23925910?flat'} 2025-07-31 04:18:54,493 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-85309723:ctest-mgmt_vn-23925910 virtual network link through opserver 10.0.0.61 2025-07-31 04:18:54,493 - DEBUG - Requesting: http://10.0.0.61:8081/analytics/uves/virtual-networks 2025-07-31 04:18:54,800 - DEBUG - Requesting: http://10.0.0.61:8081/analytics/uves/virtual-networks 2025-07-31 04:18:54,871 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-85309723:ctest-mgmt_vn-23925910', 'href': 'http://10.0.0.61:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-85309723:ctest-mgmt_vn-23925910?flat'} 2025-07-31 04:18:54,871 - INFO - Validated that VN default-domain:ctest-TestSvcRegr-85309723:ctest-mgmt_vn-23925910 is found in opserver 2025-07-31 04:18:54,903 - DEBUG - Do not have enough data to verify VN in agent 2025-07-31 04:18:54,917 - DEBUG - VRF ids for VN ctest-mgmt_vn-23925910: {} 2025-07-31 04:18:55,195 - DEBUG - Response for create_network : {'network': {'id': '3c7f83a2-02d6-44b6-8ddd-17745b73d9f4', 'name': 'service-left-vn-02539498', 'tenant_id': 'b1960ad831ff4683883f7e53e7827c36', 'project_id': 'b1960ad831ff4683883f7e53e7827c36', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcRegr-85309723', 'service-left-vn-02539498'], 'port_security_enabled': True, 'description': ''}} 2025-07-31 04:18:55,410 - DEBUG - Response for create_subnet : {'subnet': {'id': 'b48e8bd1-7e61-428d-8dee-c21e5aa165ca', 'name': '', 'tenant_id': 'b1960ad831ff4683883f7e53e7827c36', 'network_id': '3c7f83a2-02d6-44b6-8ddd-17745b73d9f4', 'ip_version': 4, 'cidr': '104.144.166.0/26', 'allocation_pools': [{'start': '104.144.166.2', 'end': '104.144.166.62'}], 'gateway_ip': '104.144.166.1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '104.144.166.2', 'tags': [], 'project_id': 'b1960ad831ff4683883f7e53e7827c36'}} 2025-07-31 04:18:55,444 - INFO - Created VN service-left-vn-02539498 2025-07-31 04:18:55,458 - DEBUG - VN service-left-vn-02539498 UUID is 3c7f83a2-02d6-44b6-8ddd-17745b73d9f4 2025-07-31 04:18:55,559 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:18:55,569 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:18:55,586 - DEBUG - Requesting: http://10.0.0.41:8082/project/b1960ad8-31ff-4683-883f-7e53e7827c36 2025-07-31 04:18:55,689 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/3c7f83a2-02d6-44b6-8ddd-17745b73d9f4 2025-07-31 04:18:55,707 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/3c7f83a2-02d6-44b6-8ddd-17745b73d9f4 2025-07-31 04:18:55,740 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/819fc72a-cf60-4ddc-9083-4e1404072253 2025-07-31 04:18:55,773 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/819fc72a-cf60-4ddc-9083-4e1404072253 2025-07-31 04:18:55,784 - DEBUG - Requesting: http://10.0.0.41:8082/route-target/807f3f01-a84f-4034-b292-45dad15813c3 2025-07-31 04:18:55,795 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-31 04:18:55,795 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/3c7f83a2-02d6-44b6-8ddd-17745b73d9f4 2025-07-31 04:18:55,806 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/819fc72a-cf60-4ddc-9083-4e1404072253 2025-07-31 04:18:55,826 - INFO - Verified VN network id 12 for VN 3c7f83a2-02d6-44b6-8ddd-17745b73d9f4 2025-07-31 04:18:55,826 - INFO - Verifications in API Server for VN service-left-vn-02539498 passed 2025-07-31 04:18:55,826 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/3c7f83a2-02d6-44b6-8ddd-17745b73d9f4 2025-07-31 04:18:55,839 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/819fc72a-cf60-4ddc-9083-4e1404072253 2025-07-31 04:18:55,851 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/819fc72a-cf60-4ddc-9083-4e1404072253 2025-07-31 04:18:55,860 - DEBUG - Requesting: http://10.0.0.41:8082/route-target/807f3f01-a84f-4034-b292-45dad15813c3 2025-07-31 04:18:55,876 - DEBUG - Control-node 10.0.0.41 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-85309723:service-left-vn-02539498', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-85309723', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '4359347696173466806', 'uuid-lslong': '10222352517718858228'}, 'enable': 'true', 'created': '2025-07-31T04:18:55', 'last-modified': '2025-07-31T04:18:55', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.517977'} 2025-07-31 04:18:55,886 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-31 04:18:55,895 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-85309723:service-left-vn-02539498', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-85309723', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '4359347696173466806', 'uuid-lslong': '10222352517718858228'}, 'enable': 'true', 'created': '2025-07-31T04:18:55', 'last-modified': '2025-07-31T04:18:55', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.537944'} 2025-07-31 04:18:55,902 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-31 04:18:55,910 - DEBUG - Control-node 10.0.0.61 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-85309723:service-left-vn-02539498', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-85309723', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '4359347696173466806', 'uuid-lslong': '10222352517718858228'}, 'enable': 'true', 'created': '2025-07-31T04:18:55', 'last-modified': '2025-07-31T04:18:55', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.545876'} 2025-07-31 04:18:55,917 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-31 04:18:55,918 - INFO - On all control nodes, Config, RI and RT verification for VN service-left-vn-02539498 passed 2025-07-31 04:18:55,918 - DEBUG - ====Verifying policy data for service-left-vn-02539498 in API_Server ====== 2025-07-31 04:18:55,918 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:18:55,927 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:18:55,946 - DEBUG - Requesting: http://10.0.0.41:8082/project/b1960ad8-31ff-4683-883f-7e53e7827c36 2025-07-31 04:18:56,048 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/3c7f83a2-02d6-44b6-8ddd-17745b73d9f4 2025-07-31 04:18:56,066 - DEBUG - =>VN service-left-vn-02539498 has no policy to be verified 2025-07-31 04:18:56,066 - DEBUG - Verifying the vn in opserver 2025-07-31 04:18:56,066 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-85309723:service-left-vn-02539498 virtual network link through opserver 10.0.0.41 2025-07-31 04:18:56,067 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/virtual-networks 2025-07-31 04:18:56,132 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-85309723:service-left-vn-02539498', 'href': 'http://10.0.0.41:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-85309723:service-left-vn-02539498?flat'} 2025-07-31 04:18:56,132 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-85309723:service-left-vn-02539498 virtual network link through opserver 10.0.0.38 2025-07-31 04:18:56,132 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-07-31 04:18:56,207 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-85309723:service-left-vn-02539498', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-85309723:service-left-vn-02539498?flat'} 2025-07-31 04:18:56,207 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-85309723:service-left-vn-02539498 virtual network link through opserver 10.0.0.61 2025-07-31 04:18:56,207 - DEBUG - Requesting: http://10.0.0.61:8081/analytics/uves/virtual-networks 2025-07-31 04:18:56,308 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-85309723:service-left-vn-02539498', 'href': 'http://10.0.0.61:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-85309723:service-left-vn-02539498?flat'} 2025-07-31 04:18:56,308 - INFO - Validated that VN default-domain:ctest-TestSvcRegr-85309723:service-left-vn-02539498 is found in opserver 2025-07-31 04:18:56,339 - DEBUG - Do not have enough data to verify VN in agent 2025-07-31 04:18:56,353 - DEBUG - VRF ids for VN service-left-vn-02539498: {} 2025-07-31 04:18:56,637 - DEBUG - Response for create_network : {'network': {'id': 'a5c1708a-27f3-46be-bbd9-63471f0af784', 'name': 'service-right-vn-46227450', 'tenant_id': 'b1960ad831ff4683883f7e53e7827c36', 'project_id': 'b1960ad831ff4683883f7e53e7827c36', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcRegr-85309723', 'service-right-vn-46227450'], 'port_security_enabled': True, 'description': ''}} 2025-07-31 04:18:56,944 - DEBUG - Response for create_subnet : {'subnet': {'id': 'a36eedb2-5131-487d-b4bf-b9eba0f8f9a5', 'name': '', 'tenant_id': 'b1960ad831ff4683883f7e53e7827c36', 'network_id': 'a5c1708a-27f3-46be-bbd9-63471f0af784', 'ip_version': 4, 'cidr': '74.29.40.64/26', 'allocation_pools': [{'start': '74.29.40.66', 'end': '74.29.40.126'}], 'gateway_ip': '74.29.40.65', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '74.29.40.66', 'tags': [], 'project_id': 'b1960ad831ff4683883f7e53e7827c36'}} 2025-07-31 04:18:56,970 - INFO - Created VN service-right-vn-46227450 2025-07-31 04:18:56,986 - DEBUG - VN service-right-vn-46227450 UUID is a5c1708a-27f3-46be-bbd9-63471f0af784 2025-07-31 04:18:57,065 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:18:57,075 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:18:57,093 - DEBUG - Requesting: http://10.0.0.41:8082/project/b1960ad8-31ff-4683-883f-7e53e7827c36 2025-07-31 04:18:57,211 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/a5c1708a-27f3-46be-bbd9-63471f0af784 2025-07-31 04:18:57,234 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/a5c1708a-27f3-46be-bbd9-63471f0af784 2025-07-31 04:18:57,251 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/03232e12-2bc0-4905-ba16-1cb4ebdb753a 2025-07-31 04:18:57,260 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/03232e12-2bc0-4905-ba16-1cb4ebdb753a 2025-07-31 04:18:57,270 - DEBUG - Requesting: http://10.0.0.41:8082/route-target/23ddbc79-f1c0-42b7-a358-9d49426f1a41 2025-07-31 04:18:57,279 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-31 04:18:57,279 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/a5c1708a-27f3-46be-bbd9-63471f0af784 2025-07-31 04:18:57,293 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/03232e12-2bc0-4905-ba16-1cb4ebdb753a 2025-07-31 04:18:57,314 - INFO - Verified VN network id 13 for VN a5c1708a-27f3-46be-bbd9-63471f0af784 2025-07-31 04:18:57,314 - INFO - Verifications in API Server for VN service-right-vn-46227450 passed 2025-07-31 04:18:57,314 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/a5c1708a-27f3-46be-bbd9-63471f0af784 2025-07-31 04:18:57,325 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/03232e12-2bc0-4905-ba16-1cb4ebdb753a 2025-07-31 04:18:57,335 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/03232e12-2bc0-4905-ba16-1cb4ebdb753a 2025-07-31 04:18:57,344 - DEBUG - Requesting: http://10.0.0.41:8082/route-target/23ddbc79-f1c0-42b7-a358-9d49426f1a41 2025-07-31 04:18:57,365 - DEBUG - Control-node 10.0.0.41 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-85309723:service-right-vn-46227450', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-85309723', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '11943951425441318590', 'uuid-lslong': '13535959312153376644'}, 'enable': 'true', 'created': '2025-07-31T04:18:56', 'last-modified': '2025-07-31T04:18:56', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.509005'} 2025-07-31 04:18:57,376 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-31 04:18:57,383 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-85309723:service-right-vn-46227450', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-85309723', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '11943951425441318590', 'uuid-lslong': '13535959312153376644'}, 'enable': 'true', 'created': '2025-07-31T04:18:56', 'last-modified': '2025-07-31T04:18:56', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.527171'} 2025-07-31 04:18:57,395 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-31 04:18:57,404 - DEBUG - Control-node 10.0.0.61 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-85309723:service-right-vn-46227450', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-85309723', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '11943951425441318590', 'uuid-lslong': '13535959312153376644'}, 'enable': 'true', 'created': '2025-07-31T04:18:56', 'last-modified': '2025-07-31T04:18:56', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.547769'} 2025-07-31 04:18:57,418 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-31 04:18:57,418 - INFO - On all control nodes, Config, RI and RT verification for VN service-right-vn-46227450 passed 2025-07-31 04:18:57,418 - DEBUG - ====Verifying policy data for service-right-vn-46227450 in API_Server ====== 2025-07-31 04:18:57,418 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:18:57,425 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:18:57,444 - DEBUG - Requesting: http://10.0.0.41:8082/project/b1960ad8-31ff-4683-883f-7e53e7827c36 2025-07-31 04:18:57,562 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/a5c1708a-27f3-46be-bbd9-63471f0af784 2025-07-31 04:18:57,589 - DEBUG - =>VN service-right-vn-46227450 has no policy to be verified 2025-07-31 04:18:57,589 - DEBUG - Verifying the vn in opserver 2025-07-31 04:18:57,589 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-85309723:service-right-vn-46227450 virtual network link through opserver 10.0.0.41 2025-07-31 04:18:57,589 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/virtual-networks 2025-07-31 04:18:57,657 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-85309723:service-right-vn-46227450', 'href': 'http://10.0.0.41:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-85309723:service-right-vn-46227450?flat'} 2025-07-31 04:18:57,657 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-85309723:service-right-vn-46227450 virtual network link through opserver 10.0.0.38 2025-07-31 04:18:57,657 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-07-31 04:18:57,751 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-85309723:service-right-vn-46227450', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-85309723:service-right-vn-46227450?flat'} 2025-07-31 04:18:57,751 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-85309723:service-right-vn-46227450 virtual network link through opserver 10.0.0.61 2025-07-31 04:18:57,751 - DEBUG - Requesting: http://10.0.0.61:8081/analytics/uves/virtual-networks 2025-07-31 04:18:57,816 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-85309723:service-right-vn-46227450', 'href': 'http://10.0.0.61:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-85309723:service-right-vn-46227450?flat'} 2025-07-31 04:18:57,816 - INFO - Validated that VN default-domain:ctest-TestSvcRegr-85309723:service-right-vn-46227450 is found in opserver 2025-07-31 04:18:57,847 - DEBUG - Do not have enough data to verify VN in agent 2025-07-31 04:18:57,864 - DEBUG - VRF ids for VN service-right-vn-46227450: {} 2025-07-31 04:18:57,931 - DEBUG - Services list from nova: [, , , ] 2025-07-31 04:18:59,161 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3732-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3732-1) 2025-07-31 04:19:00,690 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3732-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3732-2) 2025-07-31 04:19:00,690 - DEBUG - Creating service template: ['default-domain', 'ctest-service_template_1-91794768'] 2025-07-31 04:19:01,020 - DEBUG - Service template: ['default-domain', 'ctest-service_template_1-91794768'] created succesfully 2025-07-31 04:20:42,130 - DEBUG - Installing image tiny_nat_fw 2025-07-31 04:20:42,130 - DEBUG - Image info {'name': 'tinycore-in-network-nat.qcow2.gz', 'webserver': '', 'location': '/images/tinycore/', 'username': 'tc', 'password': 'secret', 'type': 'os', 'flavor': 'contrail_flavor_tiny', 'params': '--container-format bare --disk-format vmdk --property vmware_disktype="sparse" --property vmware_adaptertype="ide" --property hypervisor_type=qemu', 'name_docker': 'phusion-baseimage-enablesshd', 'vctmpl': 'service_chain_images/nat_service/tiny_nat_fw/tinycore-in-network-nat.vmtx', 'vcname': 'service_chain_images/nat_service/tiny_nat_fw/tinycore-in-network-nat.vmdk', 'vmdk': 'service_chain_images/nat_service/tiny_nat_fw/tinycore-in-network-nat-flat.vmdk', 'shrinked_vmdk': "'no'"} 2025-07-31 04:20:42,131 - DEBUG - Image build path file:///contrail-test/images/tinycore-in-network-nat.qcow2.gz 2025-07-31 04:20:42,131 - DEBUG - Download image from local file file:///contrail-test/images/tinycore-in-network-nat.qcow2.gz 2025-07-31 04:20:42,131 - DEBUG - Unzip image 2025-07-31 04:20:42,132 - DEBUG - Image download (local=True) (proxy=None): Execute cmd: gunzip -f /contrail-test/images/tinycore-in-network-nat.qcow2.gz 2025-07-31 04:20:42,158 - DEBUG - Try to create image in user context 2025-07-31 04:20:42,257 - DEBUG - Deleting service template: ['default-domain', 'ctest-service_template_1-91794768'] 2025-07-31 04:20:42,361 - INFO - Service template: ['default-domain', 'ctest-service_template_1-91794768'] deleted successfully. 2025-07-31 04:20:42,361 - INFO - Skip interface_detach for VM ctest-right_vm-95482399 in state BUILD 2025-07-31 04:20:42,361 - INFO - Deleting VM ctest-right_vm-95482399 2025-07-31 04:20:42,389 - ERROR - VM ctest-right_vm-95482399 has failed to come up 2025-07-31 04:20:42,390 - ERROR - Nova failed to get host of the VM 2025-07-31 04:20:42,390 - ERROR - VM ctest-right_vm-95482399 has failed to come up 2025-07-31 04:20:42,390 - ERROR - Nova failed to get host of the VM 2025-07-31 04:20:42,474 - INFO - Skip interface_detach for VM ctest-left_vm-75350884 in state BUILD 2025-07-31 04:20:42,475 - INFO - Deleting VM ctest-left_vm-75350884 2025-07-31 04:20:42,475 - ERROR - VM ctest-left_vm-75350884 has failed to come up 2025-07-31 04:20:42,475 - ERROR - Nova failed to get host of the VM 2025-07-31 04:20:42,475 - ERROR - VM ctest-left_vm-75350884 has failed to come up 2025-07-31 04:20:42,475 - ERROR - Nova failed to get host of the VM 2025-07-31 04:20:42,561 - INFO - Deleting VN service-right-vn-46227450 2025-07-31 04:20:42,628 - DEBUG - VN a5c1708a-27f3-46be-bbd9-63471f0af784 still in use: Unable to complete operation on network a5c1708a-27f3-46be-bbd9-63471f0af784. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-ed867632-24b9-49a8-bacb-67838f098edd'] 2025-07-31 04:20:42,628 - WARNING - Deleting VN service-right-vn-46227450 failed..Will retry 2025-07-31 04:20:44,843 - DEBUG - Response for deleting network () 2025-07-31 04:20:44,843 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/03232e12-2bc0-4905-ba16-1cb4ebdb753a 2025-07-31 04:20:44,858 - DEBUG - Response Code: 404 2025-07-31 04:20:44,858 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:20:44,868 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:20:44,922 - DEBUG - Requesting: http://10.0.0.41:8082/project/b1960ad8-31ff-4683-883f-7e53e7827c36 2025-07-31 04:20:45,041 - INFO - Validated that VN service-right-vn-46227450 is not found in API Server 2025-07-31 04:20:45,059 - DEBUG - VN service-right-vn-46227450 is not present in Agent 10.0.0.27 2025-07-31 04:20:45,077 - DEBUG - VN service-right-vn-46227450 is not present in Agent 10.0.0.57 2025-07-31 04:20:45,077 - INFO - Validated that VN service-right-vn-46227450 is not in any agent 2025-07-31 04:20:45,116 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN service-right-vn-46227450 info 2025-07-31 04:20:45,117 - INFO - Deleting VN service-left-vn-02539498 2025-07-31 04:20:45,319 - DEBUG - Response for deleting network () 2025-07-31 04:20:45,320 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/819fc72a-cf60-4ddc-9083-4e1404072253 2025-07-31 04:20:45,329 - DEBUG - Response Code: 404 2025-07-31 04:20:45,329 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:20:45,336 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:20:45,354 - DEBUG - Requesting: http://10.0.0.41:8082/project/b1960ad8-31ff-4683-883f-7e53e7827c36 2025-07-31 04:20:45,466 - INFO - Validated that VN service-left-vn-02539498 is not found in API Server 2025-07-31 04:20:45,480 - DEBUG - VN service-left-vn-02539498 is not present in Agent 10.0.0.27 2025-07-31 04:20:45,496 - DEBUG - VN service-left-vn-02539498 is not present in Agent 10.0.0.57 2025-07-31 04:20:45,497 - INFO - Validated that VN service-left-vn-02539498 is not in any agent 2025-07-31 04:20:45,549 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN service-left-vn-02539498 info 2025-07-31 04:20:45,549 - INFO - Deleting VN ctest-mgmt_vn-23925910 2025-07-31 04:20:45,759 - DEBUG - Response for deleting network () 2025-07-31 04:20:45,760 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/ecfec333-dac3-4ff7-a0aa-1b0e7b7e9a13 2025-07-31 04:20:45,767 - DEBUG - Response Code: 404 2025-07-31 04:20:45,767 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:20:45,774 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:20:45,842 - DEBUG - Requesting: http://10.0.0.41:8082/project/b1960ad8-31ff-4683-883f-7e53e7827c36 2025-07-31 04:20:45,954 - INFO - Validated that VN ctest-mgmt_vn-23925910 is not found in API Server 2025-07-31 04:20:45,969 - DEBUG - VN ctest-mgmt_vn-23925910 is not present in Agent 10.0.0.27 2025-07-31 04:20:45,984 - DEBUG - VN ctest-mgmt_vn-23925910 is not present in Agent 10.0.0.57 2025-07-31 04:20:45,984 - INFO - Validated that VN ctest-mgmt_vn-23925910 is not in any agent 2025-07-31 04:20:46,052 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-mgmt_vn-23925910 info 2025-07-31 04:20:47,408 - ERROR - 
Traceback (most recent call last):
  File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper
    result = function(self, *args, **kwargs)
  File "/contrail-test/scripts/svc_firewall/test_svc_fw_basic.py", line 35, in test_svc_CEM_22032
    right_vn_name=get_random_name('service-right-vn', ''))
  File "/contrail-test/common/servicechain/verify.py", line 320, in verify_svc_chain
    ret_dict = svc_chain_info or self.config_svc_chain(*args, **kwargs)
  File "/contrail-test/common/servicechain/config.py", line 533, in config_svc_chain
    max_inst=max_inst)
  File "/contrail-test/common/servicechain/config.py", line 335, in create_service_vms
    zone=random.choice(valid_zones))
  File "/contrail-test/common/servicechain/config.py", line 161, in config_vm_only
    flavor=flavor, zone=zone, node_name=node_name,**kwargs)
  File "/contrail-test/common/servicechain/config.py", line 203, in config_vm
    flavor=flavor, vn_objs=vns, count=count, zone=zone,**kwargs))
  File "/usr/local/lib/python3.6/site-packages/testtools/testcase.py", line 756, in useFixture
    reraise(*exc_info)
  File "/usr/local/lib/python3.6/site-packages/testtools/_compat3x.py", line 16, in reraise
    raise exc_obj.with_traceback(exc_tb)
  File "/usr/local/lib/python3.6/site-packages/testtools/testcase.py", line 731, in useFixture
    fixture.setUp()
  File "/contrail-test/fixtures/vm_test.py", line 188, in setUp
    self.create()
  File "/contrail-test/fixtures/vm_test.py", line 240, in create
    fixed_ips=self.fixed_ips)
  File "/contrail-test/fixtures/openstack.py", line 118, in create_vm
    return self.nova_h.create_vm(vm_name=vm_name, image_name=image_name, vn_ids=vn_ids, **kwargs)
  File "/contrail-test/fixtures/nova_test.py", line 577, in create_vm
    image = self.get_image(image_name=image_name)
  File "/contrail-test/fixtures/nova_test.py", line 198, in get_image
    self._install_image(image_name=image_name)
  File "/contrail-test/fixtures/nova_test.py", line 338, in _install_image
    return self.copy_and_glance(build_path, image_name, params)
  File "/contrail-test/fixtures/nova_test.py", line 407, in copy_and_glance
    image_id = self.glance_h.create_image(generic_image_name, image_path_real, **params)
  File "/contrail-test/fixtures/glance_test.py", line 50, in create_image
    self.upload_image(obj['id'], filename)
  File "/contrail-test/fixtures/glance_test.py", line 54, in upload_image
    self.obj.images.upload(uuid, open(filename, 'rb'))
FileNotFoundError: [Errno 2] No such file or directory: '/contrail-test/images/tinycore-in-network-nat.qcow2'
2025-07-31 04:20:47,433 - DEBUG - No XMPP flaps were noticed during the test 2025-07-31 04:20:47,433 - INFO - 2025-07-31 04:20:47,434 - INFO - END TEST : test_svc_CEM_22032 : FAILED[0:01:57] 2025-07-31 04:20:47,434 - INFO - -------------------------------------------------------------------------------- 2025-07-31 04:20:47,445 - INFO - ================================================================================ 2025-07-31 04:20:47,445 - INFO - STARTING TEST : test_svc_in_network_datapath 2025-07-31 04:20:48,833 - DEBUG - Nothing to compare xmpp stats {'10.0.0.27': {'10.20.0.193': '0', '10.20.0.18': '0'}, '10.0.0.57': {'10.20.0.18': '0', '10.20.0.193': '0'}} with 2025-07-31 04:20:48,834 - INFO - Initial checks done. Running the testcase now 2025-07-31 04:20:48,834 - INFO - 2025-07-31 04:20:49,152 - DEBUG - Response for create_network : {'network': {'id': '57d97065-cf8e-40f1-8eee-604fda40270e', 'name': 'ctest-mgmt_vn-51375819', 'tenant_id': 'b1960ad831ff4683883f7e53e7827c36', 'project_id': 'b1960ad831ff4683883f7e53e7827c36', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcRegr-85309723', 'ctest-mgmt_vn-51375819'], 'port_security_enabled': True, 'description': ''}} 2025-07-31 04:20:49,444 - DEBUG - Response for create_subnet : {'subnet': {'id': '1016ff1e-c779-4a12-8f4c-38a71101b312', 'name': '', 'tenant_id': 'b1960ad831ff4683883f7e53e7827c36', 'network_id': '57d97065-cf8e-40f1-8eee-604fda40270e', 'ip_version': 4, 'cidr': '163.228.54.128/26', 'allocation_pools': [{'start': '163.228.54.130', 'end': '163.228.54.190'}], 'gateway_ip': '163.228.54.129', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '163.228.54.130', 'tags': [], 'project_id': 'b1960ad831ff4683883f7e53e7827c36'}} 2025-07-31 04:20:49,472 - INFO - Created VN ctest-mgmt_vn-51375819 2025-07-31 04:20:49,492 - DEBUG - VN ctest-mgmt_vn-51375819 UUID is 57d97065-cf8e-40f1-8eee-604fda40270e 2025-07-31 04:20:49,587 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:20:49,595 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:20:49,614 - DEBUG - Requesting: http://10.0.0.41:8082/project/b1960ad8-31ff-4683-883f-7e53e7827c36 2025-07-31 04:20:49,755 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/57d97065-cf8e-40f1-8eee-604fda40270e 2025-07-31 04:20:49,774 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/57d97065-cf8e-40f1-8eee-604fda40270e 2025-07-31 04:20:49,791 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/7453c5c5-e821-4214-aff7-680c798697f2 2025-07-31 04:20:49,804 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/7453c5c5-e821-4214-aff7-680c798697f2 2025-07-31 04:20:49,820 - DEBUG - Requesting: http://10.0.0.41:8082/route-target/6de0efd4-936f-4c92-9d92-5bca1fbb019b 2025-07-31 04:20:49,836 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-31 04:20:49,836 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/57d97065-cf8e-40f1-8eee-604fda40270e 2025-07-31 04:20:49,861 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/7453c5c5-e821-4214-aff7-680c798697f2 2025-07-31 04:20:49,888 - INFO - Verified VN network id 11 for VN 57d97065-cf8e-40f1-8eee-604fda40270e 2025-07-31 04:20:49,888 - INFO - Verifications in API Server for VN ctest-mgmt_vn-51375819 passed 2025-07-31 04:20:49,888 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/57d97065-cf8e-40f1-8eee-604fda40270e 2025-07-31 04:20:49,908 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/7453c5c5-e821-4214-aff7-680c798697f2 2025-07-31 04:20:49,923 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/7453c5c5-e821-4214-aff7-680c798697f2 2025-07-31 04:20:49,935 - DEBUG - Requesting: http://10.0.0.41:8082/route-target/6de0efd4-936f-4c92-9d92-5bca1fbb019b 2025-07-31 04:20:49,961 - DEBUG - Control-node 10.0.0.41 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-85309723:ctest-mgmt_vn-51375819', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-85309723', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '6330214333822157041', 'uuid-lslong': '10299275293923223310'}, 'enable': 'true', 'created': '2025-07-31T04:20:49', 'last-modified': '2025-07-31T04:20:49', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.601711'} 2025-07-31 04:20:49,976 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-31 04:20:49,985 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-85309723:ctest-mgmt_vn-51375819', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-85309723', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '6330214333822157041', 'uuid-lslong': '10299275293923223310'}, 'enable': 'true', 'created': '2025-07-31T04:20:49', 'last-modified': '2025-07-31T04:20:49', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.624177'} 2025-07-31 04:20:49,994 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-31 04:20:50,002 - DEBUG - Control-node 10.0.0.61 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-85309723:ctest-mgmt_vn-51375819', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-85309723', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '6330214333822157041', 'uuid-lslong': '10299275293923223310'}, 'enable': 'true', 'created': '2025-07-31T04:20:49', 'last-modified': '2025-07-31T04:20:49', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.630515'} 2025-07-31 04:20:50,011 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-31 04:20:50,011 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-mgmt_vn-51375819 passed 2025-07-31 04:20:50,011 - DEBUG - ====Verifying policy data for ctest-mgmt_vn-51375819 in API_Server ====== 2025-07-31 04:20:50,011 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:20:50,020 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:20:50,035 - DEBUG - Requesting: http://10.0.0.41:8082/project/b1960ad8-31ff-4683-883f-7e53e7827c36 2025-07-31 04:20:50,145 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/57d97065-cf8e-40f1-8eee-604fda40270e 2025-07-31 04:20:50,169 - DEBUG - =>VN ctest-mgmt_vn-51375819 has no policy to be verified 2025-07-31 04:20:50,169 - DEBUG - Verifying the vn in opserver 2025-07-31 04:20:50,170 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-85309723:ctest-mgmt_vn-51375819 virtual network link through opserver 10.0.0.41 2025-07-31 04:20:50,170 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/virtual-networks 2025-07-31 04:20:50,189 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-85309723:ctest-mgmt_vn-51375819', 'href': 'http://10.0.0.41:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-85309723:ctest-mgmt_vn-51375819?flat'} 2025-07-31 04:20:50,189 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-85309723:ctest-mgmt_vn-51375819 virtual network link through opserver 10.0.0.38 2025-07-31 04:20:50,189 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-07-31 04:20:50,205 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-85309723:ctest-mgmt_vn-51375819', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-85309723:ctest-mgmt_vn-51375819?flat'} 2025-07-31 04:20:50,206 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-85309723:ctest-mgmt_vn-51375819 virtual network link through opserver 10.0.0.61 2025-07-31 04:20:50,206 - DEBUG - Requesting: http://10.0.0.61:8081/analytics/uves/virtual-networks 2025-07-31 04:20:50,224 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-85309723:ctest-mgmt_vn-51375819', 'href': 'http://10.0.0.61:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-85309723:ctest-mgmt_vn-51375819?flat'} 2025-07-31 04:20:50,224 - INFO - Validated that VN default-domain:ctest-TestSvcRegr-85309723:ctest-mgmt_vn-51375819 is found in opserver 2025-07-31 04:20:50,269 - DEBUG - Do not have enough data to verify VN in agent 2025-07-31 04:20:50,291 - DEBUG - VRF ids for VN ctest-mgmt_vn-51375819: {} 2025-07-31 04:20:50,623 - DEBUG - Response for create_network : {'network': {'id': '2cb915b3-b080-45ca-bf8f-7101f18377ed', 'name': 'ctest-left_vn-21558340', 'tenant_id': 'b1960ad831ff4683883f7e53e7827c36', 'project_id': 'b1960ad831ff4683883f7e53e7827c36', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcRegr-85309723', 'ctest-left_vn-21558340'], 'port_security_enabled': True, 'description': ''}} 2025-07-31 04:20:50,917 - DEBUG - Response for create_subnet : {'subnet': {'id': '65e558e9-b72c-484f-887b-ded302e65583', 'name': '', 'tenant_id': 'b1960ad831ff4683883f7e53e7827c36', 'network_id': '2cb915b3-b080-45ca-bf8f-7101f18377ed', 'ip_version': 4, 'cidr': '36.114.23.128/26', 'allocation_pools': [{'start': '36.114.23.130', 'end': '36.114.23.190'}], 'gateway_ip': '36.114.23.129', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '36.114.23.130', 'tags': [], 'project_id': 'b1960ad831ff4683883f7e53e7827c36'}} 2025-07-31 04:20:50,948 - INFO - Created VN ctest-left_vn-21558340 2025-07-31 04:20:50,964 - DEBUG - VN ctest-left_vn-21558340 UUID is 2cb915b3-b080-45ca-bf8f-7101f18377ed 2025-07-31 04:20:51,066 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:20:51,074 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:20:51,096 - DEBUG - Requesting: http://10.0.0.41:8082/project/b1960ad8-31ff-4683-883f-7e53e7827c36 2025-07-31 04:20:51,232 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/2cb915b3-b080-45ca-bf8f-7101f18377ed 2025-07-31 04:20:51,251 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/2cb915b3-b080-45ca-bf8f-7101f18377ed 2025-07-31 04:20:51,268 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/0cf02916-8be8-4cbf-8fa3-06c3e6d49773 2025-07-31 04:20:51,284 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/0cf02916-8be8-4cbf-8fa3-06c3e6d49773 2025-07-31 04:20:51,297 - DEBUG - Requesting: http://10.0.0.41:8082/route-target/cedf99cc-4a9b-4095-8ee0-b3185a053c37 2025-07-31 04:20:51,312 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-31 04:20:51,312 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/2cb915b3-b080-45ca-bf8f-7101f18377ed 2025-07-31 04:20:51,340 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/0cf02916-8be8-4cbf-8fa3-06c3e6d49773 2025-07-31 04:20:51,373 - INFO - Verified VN network id 12 for VN 2cb915b3-b080-45ca-bf8f-7101f18377ed 2025-07-31 04:20:51,373 - INFO - Verifications in API Server for VN ctest-left_vn-21558340 passed 2025-07-31 04:20:51,373 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/2cb915b3-b080-45ca-bf8f-7101f18377ed 2025-07-31 04:20:51,396 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/0cf02916-8be8-4cbf-8fa3-06c3e6d49773 2025-07-31 04:20:51,416 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/0cf02916-8be8-4cbf-8fa3-06c3e6d49773 2025-07-31 04:20:51,436 - DEBUG - Requesting: http://10.0.0.41:8082/route-target/cedf99cc-4a9b-4095-8ee0-b3185a053c37 2025-07-31 04:20:51,468 - DEBUG - Control-node 10.0.0.41 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-85309723:ctest-left_vn-21558340', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-85309723', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '3222630869864826314', 'uuid-lslong': '13803375636074690541'}, 'enable': 'true', 'created': '2025-07-31T04:20:50', 'last-modified': '2025-07-31T04:20:50', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.594588'} 2025-07-31 04:20:51,487 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-31 04:20:51,497 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-85309723:ctest-left_vn-21558340', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-85309723', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '3222630869864826314', 'uuid-lslong': '13803375636074690541'}, 'enable': 'true', 'created': '2025-07-31T04:20:50', 'last-modified': '2025-07-31T04:20:50', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.623148'} 2025-07-31 04:20:51,507 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-31 04:20:51,515 - DEBUG - Control-node 10.0.0.61 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-85309723:ctest-left_vn-21558340', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-85309723', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '3222630869864826314', 'uuid-lslong': '13803375636074690541'}, 'enable': 'true', 'created': '2025-07-31T04:20:50', 'last-modified': '2025-07-31T04:20:50', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.656102'} 2025-07-31 04:20:51,523 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-07-31 04:20:51,524 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-left_vn-21558340 passed 2025-07-31 04:20:51,524 - DEBUG - ====Verifying policy data for ctest-left_vn-21558340 in API_Server ====== 2025-07-31 04:20:51,524 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:20:51,532 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:20:51,554 - DEBUG - Requesting: http://10.0.0.41:8082/project/b1960ad8-31ff-4683-883f-7e53e7827c36 2025-07-31 04:20:51,668 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/2cb915b3-b080-45ca-bf8f-7101f18377ed 2025-07-31 04:20:51,682 - DEBUG - =>VN ctest-left_vn-21558340 has no policy to be verified 2025-07-31 04:20:51,682 - DEBUG - Verifying the vn in opserver 2025-07-31 04:20:51,682 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-85309723:ctest-left_vn-21558340 virtual network link through opserver 10.0.0.41 2025-07-31 04:20:51,682 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/virtual-networks 2025-07-31 04:20:51,704 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-85309723:ctest-left_vn-21558340', 'href': 'http://10.0.0.41:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-85309723:ctest-left_vn-21558340?flat'} 2025-07-31 04:20:51,704 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-85309723:ctest-left_vn-21558340 virtual network link through opserver 10.0.0.38 2025-07-31 04:20:51,704 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-07-31 04:20:51,724 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-85309723:ctest-left_vn-21558340', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-85309723:ctest-left_vn-21558340?flat'} 2025-07-31 04:20:51,724 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-85309723:ctest-left_vn-21558340 virtual network link through opserver 10.0.0.61 2025-07-31 04:20:51,725 - DEBUG - Requesting: http://10.0.0.61:8081/analytics/uves/virtual-networks 2025-07-31 04:20:51,747 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-85309723:ctest-left_vn-21558340', 'href': 'http://10.0.0.61:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-85309723:ctest-left_vn-21558340?flat'} 2025-07-31 04:20:51,747 - INFO - Validated that VN default-domain:ctest-TestSvcRegr-85309723:ctest-left_vn-21558340 is found in opserver 2025-07-31 04:20:51,797 - DEBUG - Do not have enough data to verify VN in agent 2025-07-31 04:20:51,812 - DEBUG - VRF ids for VN ctest-left_vn-21558340: {} 2025-07-31 04:20:52,124 - DEBUG - Response for create_network : {'network': {'id': '88fd211a-6141-47d7-8c52-a8cd313cda8d', 'name': 'ctest-right_vn-09349639', 'tenant_id': 'b1960ad831ff4683883f7e53e7827c36', 'project_id': 'b1960ad831ff4683883f7e53e7827c36', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcRegr-85309723', 'ctest-right_vn-09349639'], 'port_security_enabled': True, 'description': ''}} 2025-07-31 04:20:52,401 - DEBUG - Response for create_subnet : {'subnet': {'id': '98bda843-82dd-446d-9624-d26f750f70df', 'name': '', 'tenant_id': 'b1960ad831ff4683883f7e53e7827c36', 'network_id': '88fd211a-6141-47d7-8c52-a8cd313cda8d', 'ip_version': 4, 'cidr': '178.206.69.64/26', 'allocation_pools': [{'start': '178.206.69.66', 'end': '178.206.69.126'}], 'gateway_ip': '178.206.69.65', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '178.206.69.66', 'tags': [], 'project_id': 'b1960ad831ff4683883f7e53e7827c36'}} 2025-07-31 04:20:52,429 - INFO - Created VN ctest-right_vn-09349639 2025-07-31 04:20:52,444 - DEBUG - VN ctest-right_vn-09349639 UUID is 88fd211a-6141-47d7-8c52-a8cd313cda8d 2025-07-31 04:20:52,529 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:20:52,538 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:20:52,558 - DEBUG - Requesting: http://10.0.0.41:8082/project/b1960ad8-31ff-4683-883f-7e53e7827c36 2025-07-31 04:20:52,683 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/88fd211a-6141-47d7-8c52-a8cd313cda8d 2025-07-31 04:20:52,702 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/88fd211a-6141-47d7-8c52-a8cd313cda8d 2025-07-31 04:20:52,723 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/e0aee1f5-1222-4146-87a3-723fdc00814f 2025-07-31 04:20:52,735 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/e0aee1f5-1222-4146-87a3-723fdc00814f 2025-07-31 04:20:52,747 - DEBUG - Requesting: http://10.0.0.41:8082/route-target/fa787421-c8e2-402a-b76d-f266f8b1ee5d 2025-07-31 04:20:52,760 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-31 04:20:52,760 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/88fd211a-6141-47d7-8c52-a8cd313cda8d 2025-07-31 04:20:52,784 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/e0aee1f5-1222-4146-87a3-723fdc00814f 2025-07-31 04:20:52,814 - INFO - Verified VN network id 13 for VN 88fd211a-6141-47d7-8c52-a8cd313cda8d 2025-07-31 04:20:52,814 - INFO - Verifications in API Server for VN ctest-right_vn-09349639 passed 2025-07-31 04:20:52,815 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/88fd211a-6141-47d7-8c52-a8cd313cda8d 2025-07-31 04:20:52,830 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/e0aee1f5-1222-4146-87a3-723fdc00814f 2025-07-31 04:20:52,842 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/e0aee1f5-1222-4146-87a3-723fdc00814f 2025-07-31 04:20:52,852 - DEBUG - Requesting: http://10.0.0.41:8082/route-target/fa787421-c8e2-402a-b76d-f266f8b1ee5d 2025-07-31 04:20:52,874 - DEBUG - Control-node 10.0.0.41 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-85309723:ctest-right_vn-09349639', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-85309723', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '9871082355450529751', 'uuid-lslong': '10111329712648018573'}, 'enable': 'true', 'created': '2025-07-31T04:20:52', 'last-modified': '2025-07-31T04:20:52', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.534475'} 2025-07-31 04:20:52,884 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-31 04:20:52,894 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-85309723:ctest-right_vn-09349639', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-85309723', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '9871082355450529751', 'uuid-lslong': '10111329712648018573'}, 'enable': 'true', 'created': '2025-07-31T04:20:52', 'last-modified': '2025-07-31T04:20:52', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.552687'} 2025-07-31 04:20:52,901 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-31 04:20:52,909 - DEBUG - Control-node 10.0.0.61 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcRegr-85309723:ctest-right_vn-09349639', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcRegr-85309723', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '9871082355450529751', 'uuid-lslong': '10111329712648018573'}, 'enable': 'true', 'created': '2025-07-31T04:20:52', 'last-modified': '2025-07-31T04:20:52', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.562359'} 2025-07-31 04:20:52,919 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-07-31 04:20:52,919 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-right_vn-09349639 passed 2025-07-31 04:20:52,919 - DEBUG - ====Verifying policy data for ctest-right_vn-09349639 in API_Server ====== 2025-07-31 04:20:52,919 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:20:52,929 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:20:52,947 - DEBUG - Requesting: http://10.0.0.41:8082/project/b1960ad8-31ff-4683-883f-7e53e7827c36 2025-07-31 04:20:53,062 - DEBUG - Requesting: http://10.0.0.41:8082/virtual-network/88fd211a-6141-47d7-8c52-a8cd313cda8d 2025-07-31 04:20:53,084 - DEBUG - =>VN ctest-right_vn-09349639 has no policy to be verified 2025-07-31 04:20:53,084 - DEBUG - Verifying the vn in opserver 2025-07-31 04:20:53,084 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-85309723:ctest-right_vn-09349639 virtual network link through opserver 10.0.0.41 2025-07-31 04:20:53,084 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/virtual-networks 2025-07-31 04:20:53,101 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-85309723:ctest-right_vn-09349639', 'href': 'http://10.0.0.41:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-85309723:ctest-right_vn-09349639?flat'} 2025-07-31 04:20:53,101 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-85309723:ctest-right_vn-09349639 virtual network link through opserver 10.0.0.38 2025-07-31 04:20:53,101 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-07-31 04:20:53,117 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-85309723:ctest-right_vn-09349639', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-85309723:ctest-right_vn-09349639?flat'} 2025-07-31 04:20:53,117 - DEBUG - Verifying the default-domain:ctest-TestSvcRegr-85309723:ctest-right_vn-09349639 virtual network link through opserver 10.0.0.61 2025-07-31 04:20:53,117 - DEBUG - Requesting: http://10.0.0.61:8081/analytics/uves/virtual-networks 2025-07-31 04:20:53,133 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcRegr-85309723:ctest-right_vn-09349639', 'href': 'http://10.0.0.61:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcRegr-85309723:ctest-right_vn-09349639?flat'} 2025-07-31 04:20:53,133 - INFO - Validated that VN default-domain:ctest-TestSvcRegr-85309723:ctest-right_vn-09349639 is found in opserver 2025-07-31 04:20:53,167 - DEBUG - Do not have enough data to verify VN in agent 2025-07-31 04:20:53,181 - DEBUG - VRF ids for VN ctest-right_vn-09349639: {} 2025-07-31 04:20:54,514 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3732-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3732-2) 2025-07-31 04:20:55,962 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-3732-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3732-1) 2025-07-31 04:20:55,962 - DEBUG - Creating service template: ['default-domain', 'ctest-service_template_1-49825656'] 2025-07-31 04:20:56,294 - DEBUG - Service template: ['default-domain', 'ctest-service_template_1-49825656'] created succesfully 2025-07-31 04:22:37,304 - DEBUG - Installing image tiny_nat_fw 2025-07-31 04:22:37,304 - DEBUG - Image info {'name': 'tinycore-in-network-nat.qcow2.gz', 'webserver': '', 'location': '/images/tinycore/', 'username': 'tc', 'password': 'secret', 'type': 'os', 'flavor': 'contrail_flavor_tiny', 'params': '--container-format bare --disk-format vmdk --property vmware_disktype="sparse" --property vmware_adaptertype="ide" --property hypervisor_type=qemu', 'name_docker': 'phusion-baseimage-enablesshd', 'vctmpl': 'service_chain_images/nat_service/tiny_nat_fw/tinycore-in-network-nat.vmtx', 'vcname': 'service_chain_images/nat_service/tiny_nat_fw/tinycore-in-network-nat.vmdk', 'vmdk': 'service_chain_images/nat_service/tiny_nat_fw/tinycore-in-network-nat-flat.vmdk', 'shrinked_vmdk': "'no'"} 2025-07-31 04:22:37,304 - DEBUG - Image build path file:///contrail-test/images/tinycore-in-network-nat.qcow2.gz 2025-07-31 04:22:37,305 - DEBUG - Download image from local file file:///contrail-test/images/tinycore-in-network-nat.qcow2.gz 2025-07-31 04:22:37,305 - DEBUG - Unzip image 2025-07-31 04:22:37,305 - DEBUG - Image download (local=True) (proxy=None): Execute cmd: gunzip -f /contrail-test/images/tinycore-in-network-nat.qcow2.gz 2025-07-31 04:22:37,339 - DEBUG - Try to create image in user context 2025-07-31 04:22:37,496 - DEBUG - Deleting service template: ['default-domain', 'ctest-service_template_1-49825656'] 2025-07-31 04:22:37,609 - INFO - Service template: ['default-domain', 'ctest-service_template_1-49825656'] deleted successfully. 2025-07-31 04:22:37,610 - INFO - Skip interface_detach for VM ctest-right_vm-37512240 in state BUILD 2025-07-31 04:22:37,610 - INFO - Deleting VM ctest-right_vm-37512240 2025-07-31 04:22:37,610 - ERROR - VM ctest-right_vm-37512240 has failed to come up 2025-07-31 04:22:37,611 - ERROR - Nova failed to get host of the VM 2025-07-31 04:22:37,611 - ERROR - VM ctest-right_vm-37512240 has failed to come up 2025-07-31 04:22:37,611 - ERROR - Nova failed to get host of the VM 2025-07-31 04:22:37,781 - INFO - Skip interface_detach for VM ctest-left_vm-12939493 in state BUILD 2025-07-31 04:22:37,781 - INFO - Deleting VM ctest-left_vm-12939493 2025-07-31 04:22:37,781 - ERROR - VM ctest-left_vm-12939493 has failed to come up 2025-07-31 04:22:37,781 - ERROR - Nova failed to get host of the VM 2025-07-31 04:22:37,781 - ERROR - VM ctest-left_vm-12939493 has failed to come up 2025-07-31 04:22:37,781 - ERROR - Nova failed to get host of the VM 2025-07-31 04:22:37,889 - INFO - Deleting VN ctest-right_vn-09349639 2025-07-31 04:22:37,940 - DEBUG - VN 88fd211a-6141-47d7-8c52-a8cd313cda8d still in use: Unable to complete operation on network 88fd211a-6141-47d7-8c52-a8cd313cda8d. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-5f5b7625-a78d-4c60-9d10-6d24abda8d80'] 2025-07-31 04:22:37,941 - WARNING - Deleting VN ctest-right_vn-09349639 failed..Will retry 2025-07-31 04:22:40,220 - DEBUG - Response for deleting network () 2025-07-31 04:22:40,220 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/e0aee1f5-1222-4146-87a3-723fdc00814f 2025-07-31 04:22:40,227 - DEBUG - Response Code: 404 2025-07-31 04:22:40,227 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:22:40,242 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:22:40,304 - DEBUG - Requesting: http://10.0.0.41:8082/project/b1960ad8-31ff-4683-883f-7e53e7827c36 2025-07-31 04:22:40,437 - INFO - Validated that VN ctest-right_vn-09349639 is not found in API Server 2025-07-31 04:22:40,458 - DEBUG - VN ctest-right_vn-09349639 is not present in Agent 10.0.0.27 2025-07-31 04:22:40,473 - DEBUG - VN ctest-right_vn-09349639 is not present in Agent 10.0.0.57 2025-07-31 04:22:40,473 - INFO - Validated that VN ctest-right_vn-09349639 is not in any agent 2025-07-31 04:22:40,530 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-right_vn-09349639 info 2025-07-31 04:22:40,530 - INFO - Deleting VN ctest-left_vn-21558340 2025-07-31 04:22:40,738 - DEBUG - Response for deleting network () 2025-07-31 04:22:40,738 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/0cf02916-8be8-4cbf-8fa3-06c3e6d49773 2025-07-31 04:22:40,749 - DEBUG - Response Code: 404 2025-07-31 04:22:40,750 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:22:40,762 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:22:40,780 - DEBUG - Requesting: http://10.0.0.41:8082/project/b1960ad8-31ff-4683-883f-7e53e7827c36 2025-07-31 04:22:40,874 - INFO - Validated that VN ctest-left_vn-21558340 is not found in API Server 2025-07-31 04:22:40,886 - DEBUG - VN ctest-left_vn-21558340 is not present in Agent 10.0.0.27 2025-07-31 04:22:40,903 - DEBUG - VN ctest-left_vn-21558340 is not present in Agent 10.0.0.57 2025-07-31 04:22:40,903 - INFO - Validated that VN ctest-left_vn-21558340 is not in any agent 2025-07-31 04:22:40,948 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-left_vn-21558340 info 2025-07-31 04:22:40,948 - INFO - Deleting VN ctest-mgmt_vn-51375819 2025-07-31 04:22:41,128 - DEBUG - Response for deleting network () 2025-07-31 04:22:41,128 - DEBUG - Requesting: http://10.0.0.41:8082/routing-instance/7453c5c5-e821-4214-aff7-680c798697f2 2025-07-31 04:22:41,136 - DEBUG - Response Code: 404 2025-07-31 04:22:41,136 - DEBUG - Requesting: http://10.0.0.41:8082/domains 2025-07-31 04:22:41,145 - DEBUG - Requesting: http://10.0.0.41:8082/domain/34dad5b7-e30d-42c7-8ff4-7b0d7fa576d7 2025-07-31 04:22:41,196 - DEBUG - Requesting: http://10.0.0.41:8082/project/b1960ad8-31ff-4683-883f-7e53e7827c36 2025-07-31 04:22:41,295 - INFO - Validated that VN ctest-mgmt_vn-51375819 is not found in API Server 2025-07-31 04:22:41,308 - DEBUG - VN ctest-mgmt_vn-51375819 is not present in Agent 10.0.0.27 2025-07-31 04:22:41,322 - DEBUG - VN ctest-mgmt_vn-51375819 is not present in Agent 10.0.0.57 2025-07-31 04:22:41,323 - INFO - Validated that VN ctest-mgmt_vn-51375819 is not in any agent 2025-07-31 04:22:41,361 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-mgmt_vn-51375819 info 2025-07-31 04:22:42,717 - ERROR - 
Traceback (most recent call last):
  File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper
    result = function(self, *args, **kwargs)
  File "/contrail-test/scripts/svc_firewall/test_svc_fw_basic.py", line 26, in test_svc_in_network_datapath
    create_svms=True)
  File "/contrail-test/common/servicechain/verify.py", line 320, in verify_svc_chain
    ret_dict = svc_chain_info or self.config_svc_chain(*args, **kwargs)
  File "/contrail-test/common/servicechain/config.py", line 533, in config_svc_chain
    max_inst=max_inst)
  File "/contrail-test/common/servicechain/config.py", line 335, in create_service_vms
    zone=random.choice(valid_zones))
  File "/contrail-test/common/servicechain/config.py", line 161, in config_vm_only
    flavor=flavor, zone=zone, node_name=node_name,**kwargs)
  File "/contrail-test/common/servicechain/config.py", line 203, in config_vm
    flavor=flavor, vn_objs=vns, count=count, zone=zone,**kwargs))
  File "/usr/local/lib/python3.6/site-packages/testtools/testcase.py", line 756, in useFixture
    reraise(*exc_info)
  File "/usr/local/lib/python3.6/site-packages/testtools/_compat3x.py", line 16, in reraise
    raise exc_obj.with_traceback(exc_tb)
  File "/usr/local/lib/python3.6/site-packages/testtools/testcase.py", line 731, in useFixture
    fixture.setUp()
  File "/contrail-test/fixtures/vm_test.py", line 188, in setUp
    self.create()
  File "/contrail-test/fixtures/vm_test.py", line 240, in create
    fixed_ips=self.fixed_ips)
  File "/contrail-test/fixtures/openstack.py", line 118, in create_vm
    return self.nova_h.create_vm(vm_name=vm_name, image_name=image_name, vn_ids=vn_ids, **kwargs)
  File "/contrail-test/fixtures/nova_test.py", line 577, in create_vm
    image = self.get_image(image_name=image_name)
  File "/contrail-test/fixtures/nova_test.py", line 198, in get_image
    self._install_image(image_name=image_name)
  File "/contrail-test/fixtures/nova_test.py", line 338, in _install_image
    return self.copy_and_glance(build_path, image_name, params)
  File "/contrail-test/fixtures/nova_test.py", line 407, in copy_and_glance
    image_id = self.glance_h.create_image(generic_image_name, image_path_real, **params)
  File "/contrail-test/fixtures/glance_test.py", line 50, in create_image
    self.upload_image(obj['id'], filename)
  File "/contrail-test/fixtures/glance_test.py", line 54, in upload_image
    self.obj.images.upload(uuid, open(filename, 'rb'))
FileNotFoundError: [Errno 2] No such file or directory: '/contrail-test/images/tinycore-in-network-nat.qcow2'
2025-07-31 04:22:42,741 - DEBUG - No XMPP flaps were noticed during the test 2025-07-31 04:22:42,741 - INFO - 2025-07-31 04:22:42,742 - INFO - END TEST : test_svc_in_network_datapath : FAILED[0:01:55] 2025-07-31 04:22:42,742 - INFO - -------------------------------------------------------------------------------- 2025-07-31 04:22:43,624 - INFO - Deleted project: ctest-TestSvcRegr-85309723, ID : b1960ad8-31ff-4683-883f-7e53e7827c36