2025-09-18 03:03:23,881 - INFO - Domain Default found not creating 2025-09-18 03:03:24,050 - INFO - Project ctest-TestSvcHC-70360481 not found, creating it 2025-09-18 03:03:24,593 - INFO - Created Project:ctest-TestSvcHC-70360481, ID : 4f4e4097-dcce-4868-a9c2-29d74f1abd03 2025-09-18 03:03:26,386 - INFO - ================================================================================ 2025-09-18 03:03:26,386 - INFO - STARTING TEST : test_svc_hc_basic 2025-09-18 03:03:27,707 - DEBUG - Nothing to compare xmpp stats {'10.0.0.61': {'10.20.0.17': '0', '10.20.0.25': '0'}, '10.0.0.27': {'10.20.0.17': '0', '10.20.0.25': '0'}} with 2025-09-18 03:03:27,707 - INFO - Initial checks done. Running the testcase now 2025-09-18 03:03:27,707 - INFO - 2025-09-18 03:03:28,478 - DEBUG - Response for create_network : {'network': {'id': '823d0493-1ba1-4718-bedb-ffef63c75b2b', 'name': 'ctest-mgmt_vn-87643786', 'tenant_id': '4f4e4097dcce4868a9c229d74f1abd03', 'project_id': '4f4e4097dcce4868a9c229d74f1abd03', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcHC-70360481', 'ctest-mgmt_vn-87643786'], 'port_security_enabled': True, 'description': ''}} 2025-09-18 03:03:28,795 - DEBUG - Response for create_subnet : {'subnet': {'id': 'fb6c93bc-03b2-4306-bc45-c600b15f2c7d', 'name': '', 'tenant_id': '4f4e4097dcce4868a9c229d74f1abd03', 'network_id': '823d0493-1ba1-4718-bedb-ffef63c75b2b', 'ip_version': 4, 'cidr': '129.51.141.128/26', 'allocation_pools': [{'start': '129.51.141.130', 'end': '129.51.141.190'}], 'gateway_ip': '129.51.141.129', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '129.51.141.130', 'tags': [], 'project_id': '4f4e4097dcce4868a9c229d74f1abd03'}} 2025-09-18 03:03:28,822 - INFO - Created VN ctest-mgmt_vn-87643786 2025-09-18 03:03:28,834 - DEBUG - VN ctest-mgmt_vn-87643786 UUID is 823d0493-1ba1-4718-bedb-ffef63c75b2b 2025-09-18 03:03:28,925 - DEBUG - Requesting: http://10.0.0.56:8082/domains 2025-09-18 03:03:29,276 - DEBUG - Requesting: http://10.0.0.56:8082/domains 2025-09-18 03:03:29,312 - DEBUG - Requesting: http://10.0.0.56:8082/domain/245899c5-19ba-4930-be6a-859f116860e1 2025-09-18 03:03:29,329 - DEBUG - Requesting: http://10.0.0.56:8082/project/4f4e4097-dcce-4868-a9c2-29d74f1abd03 2025-09-18 03:03:29,436 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/823d0493-1ba1-4718-bedb-ffef63c75b2b 2025-09-18 03:03:29,456 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/823d0493-1ba1-4718-bedb-ffef63c75b2b 2025-09-18 03:03:29,467 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/93b87f9e-f07f-492a-a935-dfde2868ce7a 2025-09-18 03:03:29,477 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/93b87f9e-f07f-492a-a935-dfde2868ce7a 2025-09-18 03:03:29,485 - DEBUG - Requesting: http://10.0.0.56:8082/route-target/35b905c7-d421-42d0-918a-f4b2f91145b4 2025-09-18 03:03:29,493 - DEBUG - Route Targets: ['target:64512:8000004'] 2025-09-18 03:03:29,493 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/823d0493-1ba1-4718-bedb-ffef63c75b2b 2025-09-18 03:03:29,503 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/93b87f9e-f07f-492a-a935-dfde2868ce7a 2025-09-18 03:03:29,520 - INFO - Verified VN network id 12 for VN 823d0493-1ba1-4718-bedb-ffef63c75b2b 2025-09-18 03:03:29,520 - INFO - Verifications in API Server for VN ctest-mgmt_vn-87643786 passed 2025-09-18 03:03:29,520 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/823d0493-1ba1-4718-bedb-ffef63c75b2b 2025-09-18 03:03:29,532 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/93b87f9e-f07f-492a-a935-dfde2868ce7a 2025-09-18 03:03:29,540 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/93b87f9e-f07f-492a-a935-dfde2868ce7a 2025-09-18 03:03:29,548 - DEBUG - Requesting: http://10.0.0.56:8082/route-target/35b905c7-d421-42d0-918a-f4b2f91145b4 2025-09-18 03:03:29,562 - DEBUG - Control-node 10.0.0.56 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-70360481:ctest-mgmt_vn-87643786', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-70360481', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '9384662228380239640', 'uuid-lslong': '13752867290742217515'}, 'enable': 'true', 'created': '2025-09-18T03:03:28', 'last-modified': '2025-09-18T03:03:28', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.833009'} 2025-09-18 03:03:29,570 - DEBUG - Route Targets: ['target:64512:8000004'] 2025-09-18 03:03:29,576 - DEBUG - Control-node 10.0.0.22 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-70360481:ctest-mgmt_vn-87643786', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-70360481', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '9384662228380239640', 'uuid-lslong': '13752867290742217515'}, 'enable': 'true', 'created': '2025-09-18T03:03:28', 'last-modified': '2025-09-18T03:03:28', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.831478'} 2025-09-18 03:03:29,583 - DEBUG - Route Targets: ['target:64512:8000004'] 2025-09-18 03:03:29,591 - DEBUG - Control-node 10.0.0.41 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-70360481:ctest-mgmt_vn-87643786', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-70360481', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '9384662228380239640', 'uuid-lslong': '13752867290742217515'}, 'enable': 'true', 'created': '2025-09-18T03:03:28', 'last-modified': '2025-09-18T03:03:28', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.859860'} 2025-09-18 03:03:29,600 - DEBUG - Route Targets: ['target:64512:8000004'] 2025-09-18 03:03:29,600 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-mgmt_vn-87643786 passed 2025-09-18 03:03:29,601 - DEBUG - ====Verifying policy data for ctest-mgmt_vn-87643786 in API_Server ====== 2025-09-18 03:03:29,601 - DEBUG - Requesting: http://10.0.0.56:8082/domains 2025-09-18 03:03:29,607 - DEBUG - Requesting: http://10.0.0.56:8082/domain/245899c5-19ba-4930-be6a-859f116860e1 2025-09-18 03:03:29,618 - DEBUG - Requesting: http://10.0.0.56:8082/project/4f4e4097-dcce-4868-a9c2-29d74f1abd03 2025-09-18 03:03:29,717 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/823d0493-1ba1-4718-bedb-ffef63c75b2b 2025-09-18 03:03:29,728 - DEBUG - =>VN ctest-mgmt_vn-87643786 has no policy to be verified 2025-09-18 03:03:29,728 - DEBUG - Verifying the vn in opserver 2025-09-18 03:03:29,729 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-70360481:ctest-mgmt_vn-87643786 virtual network link through opserver 10.0.0.56 2025-09-18 03:03:29,729 - DEBUG - Requesting: http://10.0.0.56:8081/analytics/uves/virtual-networks 2025-09-18 03:03:30,077 - DEBUG - Requesting: http://10.0.0.56:8081/analytics/uves/virtual-networks 2025-09-18 03:03:30,149 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-70360481:ctest-mgmt_vn-87643786', 'href': 'http://10.0.0.56:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-70360481:ctest-mgmt_vn-87643786?flat'} 2025-09-18 03:03:30,150 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-70360481:ctest-mgmt_vn-87643786 virtual network link through opserver 10.0.0.22 2025-09-18 03:03:30,150 - DEBUG - Requesting: http://10.0.0.22:8081/analytics/uves/virtual-networks 2025-09-18 03:03:30,495 - DEBUG - Requesting: http://10.0.0.22:8081/analytics/uves/virtual-networks 2025-09-18 03:03:30,566 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-70360481:ctest-mgmt_vn-87643786', 'href': 'http://10.0.0.22:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-70360481:ctest-mgmt_vn-87643786?flat'} 2025-09-18 03:03:30,566 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-70360481:ctest-mgmt_vn-87643786 virtual network link through opserver 10.0.0.41 2025-09-18 03:03:30,566 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/virtual-networks 2025-09-18 03:03:30,906 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/virtual-networks 2025-09-18 03:03:30,974 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-70360481:ctest-mgmt_vn-87643786', 'href': 'http://10.0.0.41:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-70360481:ctest-mgmt_vn-87643786?flat'} 2025-09-18 03:03:30,974 - INFO - Validated that VN default-domain:ctest-TestSvcHC-70360481:ctest-mgmt_vn-87643786 is found in opserver 2025-09-18 03:03:30,995 - DEBUG - Do not have enough data to verify VN in agent 2025-09-18 03:03:31,004 - DEBUG - VRF ids for VN ctest-mgmt_vn-87643786: {} 2025-09-18 03:03:31,308 - DEBUG - Response for create_network : {'network': {'id': '19db378d-b94e-4caa-bf4d-ca65e35e29d5', 'name': 'ctest-left_vn-56166373', 'tenant_id': '4f4e4097dcce4868a9c229d74f1abd03', 'project_id': '4f4e4097dcce4868a9c229d74f1abd03', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcHC-70360481', 'ctest-left_vn-56166373'], 'port_security_enabled': True, 'description': ''}} 2025-09-18 03:03:31,551 - DEBUG - Response for create_subnet : {'subnet': {'id': 'ce6e0da5-ec84-4dd6-b399-62415e57f80e', 'name': '', 'tenant_id': '4f4e4097dcce4868a9c229d74f1abd03', 'network_id': '19db378d-b94e-4caa-bf4d-ca65e35e29d5', 'ip_version': 4, 'cidr': '66.44.28.128/26', 'allocation_pools': [{'start': '66.44.28.130', 'end': '66.44.28.190'}], 'gateway_ip': '66.44.28.129', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '66.44.28.130', 'tags': [], 'project_id': '4f4e4097dcce4868a9c229d74f1abd03'}} 2025-09-18 03:03:31,582 - INFO - Created VN ctest-left_vn-56166373 2025-09-18 03:03:31,595 - DEBUG - VN ctest-left_vn-56166373 UUID is 19db378d-b94e-4caa-bf4d-ca65e35e29d5 2025-09-18 03:03:31,690 - DEBUG - Requesting: http://10.0.0.56:8082/domains 2025-09-18 03:03:31,698 - DEBUG - Requesting: http://10.0.0.56:8082/domain/245899c5-19ba-4930-be6a-859f116860e1 2025-09-18 03:03:31,713 - DEBUG - Requesting: http://10.0.0.56:8082/project/4f4e4097-dcce-4868-a9c2-29d74f1abd03 2025-09-18 03:03:31,827 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/19db378d-b94e-4caa-bf4d-ca65e35e29d5 2025-09-18 03:03:31,840 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/19db378d-b94e-4caa-bf4d-ca65e35e29d5 2025-09-18 03:03:31,851 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/6958e486-8193-4e43-a4b3-2f842c28df2a 2025-09-18 03:03:31,864 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/6958e486-8193-4e43-a4b3-2f842c28df2a 2025-09-18 03:03:31,874 - DEBUG - Requesting: http://10.0.0.56:8082/route-target/9e9edc4d-c007-44fb-9c5a-dcd4a499ffc1 2025-09-18 03:03:31,883 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-09-18 03:03:31,883 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/19db378d-b94e-4caa-bf4d-ca65e35e29d5 2025-09-18 03:03:31,892 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/6958e486-8193-4e43-a4b3-2f842c28df2a 2025-09-18 03:03:31,911 - INFO - Verified VN network id 13 for VN 19db378d-b94e-4caa-bf4d-ca65e35e29d5 2025-09-18 03:03:31,911 - INFO - Verifications in API Server for VN ctest-left_vn-56166373 passed 2025-09-18 03:03:31,911 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/19db378d-b94e-4caa-bf4d-ca65e35e29d5 2025-09-18 03:03:31,927 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/6958e486-8193-4e43-a4b3-2f842c28df2a 2025-09-18 03:03:31,935 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/6958e486-8193-4e43-a4b3-2f842c28df2a 2025-09-18 03:03:31,944 - DEBUG - Requesting: http://10.0.0.56:8082/route-target/9e9edc4d-c007-44fb-9c5a-dcd4a499ffc1 2025-09-18 03:03:31,964 - DEBUG - Control-node 10.0.0.56 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-70360481:ctest-left_vn-56166373', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-70360481', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '1863143952686664874', 'uuid-lslong': '13784896573406063061'}, 'enable': 'true', 'created': '2025-09-18T03:03:31', 'last-modified': '2025-09-18T03:03:31', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.467655'} 2025-09-18 03:03:31,981 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-09-18 03:03:31,987 - DEBUG - Control-node 10.0.0.22 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-70360481:ctest-left_vn-56166373', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-70360481', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '1863143952686664874', 'uuid-lslong': '13784896573406063061'}, 'enable': 'true', 'created': '2025-09-18T03:03:31', 'last-modified': '2025-09-18T03:03:31', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.495541'} 2025-09-18 03:03:31,992 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-09-18 03:03:31,998 - DEBUG - Control-node 10.0.0.41 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-70360481:ctest-left_vn-56166373', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-70360481', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '1863143952686664874', 'uuid-lslong': '13784896573406063061'}, 'enable': 'true', 'created': '2025-09-18T03:03:31', 'last-modified': '2025-09-18T03:03:31', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.509545'} 2025-09-18 03:03:32,006 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-09-18 03:03:32,006 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-left_vn-56166373 passed 2025-09-18 03:03:32,006 - DEBUG - ====Verifying policy data for ctest-left_vn-56166373 in API_Server ====== 2025-09-18 03:03:32,006 - DEBUG - Requesting: http://10.0.0.56:8082/domains 2025-09-18 03:03:32,013 - DEBUG - Requesting: http://10.0.0.56:8082/domain/245899c5-19ba-4930-be6a-859f116860e1 2025-09-18 03:03:32,025 - DEBUG - Requesting: http://10.0.0.56:8082/project/4f4e4097-dcce-4868-a9c2-29d74f1abd03 2025-09-18 03:03:32,126 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/19db378d-b94e-4caa-bf4d-ca65e35e29d5 2025-09-18 03:03:32,137 - DEBUG - =>VN ctest-left_vn-56166373 has no policy to be verified 2025-09-18 03:03:32,137 - DEBUG - Verifying the vn in opserver 2025-09-18 03:03:32,137 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-70360481:ctest-left_vn-56166373 virtual network link through opserver 10.0.0.56 2025-09-18 03:03:32,137 - DEBUG - Requesting: http://10.0.0.56:8081/analytics/uves/virtual-networks 2025-09-18 03:03:32,204 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-70360481:ctest-left_vn-56166373', 'href': 'http://10.0.0.56:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-70360481:ctest-left_vn-56166373?flat'} 2025-09-18 03:03:32,204 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-70360481:ctest-left_vn-56166373 virtual network link through opserver 10.0.0.22 2025-09-18 03:03:32,204 - DEBUG - Requesting: http://10.0.0.22:8081/analytics/uves/virtual-networks 2025-09-18 03:03:32,270 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-70360481:ctest-left_vn-56166373', 'href': 'http://10.0.0.22:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-70360481:ctest-left_vn-56166373?flat'} 2025-09-18 03:03:32,270 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-70360481:ctest-left_vn-56166373 virtual network link through opserver 10.0.0.41 2025-09-18 03:03:32,270 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/virtual-networks 2025-09-18 03:03:32,341 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-70360481:ctest-left_vn-56166373', 'href': 'http://10.0.0.41:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-70360481:ctest-left_vn-56166373?flat'} 2025-09-18 03:03:32,341 - INFO - Validated that VN default-domain:ctest-TestSvcHC-70360481:ctest-left_vn-56166373 is found in opserver 2025-09-18 03:03:32,363 - DEBUG - Do not have enough data to verify VN in agent 2025-09-18 03:03:32,374 - DEBUG - VRF ids for VN ctest-left_vn-56166373: {} 2025-09-18 03:03:32,645 - DEBUG - Response for create_network : {'network': {'id': '8e0f9026-164a-4353-a72e-b0f6a47f2bf7', 'name': 'ctest-right_vn-34613860', 'tenant_id': '4f4e4097dcce4868a9c229d74f1abd03', 'project_id': '4f4e4097dcce4868a9c229d74f1abd03', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcHC-70360481', 'ctest-right_vn-34613860'], 'port_security_enabled': True, 'description': ''}} 2025-09-18 03:03:32,923 - DEBUG - Response for create_subnet : {'subnet': {'id': '8eafcc5b-a55c-4a71-95c3-c2b58e3c3ac9', 'name': '', 'tenant_id': '4f4e4097dcce4868a9c229d74f1abd03', 'network_id': '8e0f9026-164a-4353-a72e-b0f6a47f2bf7', 'ip_version': 4, 'cidr': '15.165.162.192/26', 'allocation_pools': [{'start': '15.165.162.194', 'end': '15.165.162.254'}], 'gateway_ip': '15.165.162.193', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '15.165.162.194', 'tags': [], 'project_id': '4f4e4097dcce4868a9c229d74f1abd03'}} 2025-09-18 03:03:32,947 - INFO - Created VN ctest-right_vn-34613860 2025-09-18 03:03:32,962 - DEBUG - VN ctest-right_vn-34613860 UUID is 8e0f9026-164a-4353-a72e-b0f6a47f2bf7 2025-09-18 03:03:33,039 - DEBUG - Requesting: http://10.0.0.56:8082/domains 2025-09-18 03:03:33,047 - DEBUG - Requesting: http://10.0.0.56:8082/domain/245899c5-19ba-4930-be6a-859f116860e1 2025-09-18 03:03:33,060 - DEBUG - Requesting: http://10.0.0.56:8082/project/4f4e4097-dcce-4868-a9c2-29d74f1abd03 2025-09-18 03:03:33,160 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/8e0f9026-164a-4353-a72e-b0f6a47f2bf7 2025-09-18 03:03:33,178 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/8e0f9026-164a-4353-a72e-b0f6a47f2bf7 2025-09-18 03:03:33,189 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/608b47b7-6037-400f-8b12-cea26700262f 2025-09-18 03:03:33,198 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/608b47b7-6037-400f-8b12-cea26700262f 2025-09-18 03:03:33,207 - DEBUG - Requesting: http://10.0.0.56:8082/route-target/af99e639-8ffe-432c-a8a0-1c980a10eafc 2025-09-18 03:03:33,215 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-09-18 03:03:33,215 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/8e0f9026-164a-4353-a72e-b0f6a47f2bf7 2025-09-18 03:03:33,225 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/608b47b7-6037-400f-8b12-cea26700262f 2025-09-18 03:03:33,243 - INFO - Verified VN network id 14 for VN 8e0f9026-164a-4353-a72e-b0f6a47f2bf7 2025-09-18 03:03:33,244 - INFO - Verifications in API Server for VN ctest-right_vn-34613860 passed 2025-09-18 03:03:33,244 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/8e0f9026-164a-4353-a72e-b0f6a47f2bf7 2025-09-18 03:03:33,255 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/608b47b7-6037-400f-8b12-cea26700262f 2025-09-18 03:03:33,264 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/608b47b7-6037-400f-8b12-cea26700262f 2025-09-18 03:03:33,272 - DEBUG - Requesting: http://10.0.0.56:8082/route-target/af99e639-8ffe-432c-a8a0-1c980a10eafc 2025-09-18 03:03:33,287 - DEBUG - Control-node 10.0.0.56 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-70360481:ctest-right_vn-34613860', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-70360481', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '10236558971293549395', 'uuid-lslong': '12046760626630896631'}, 'enable': 'true', 'created': '2025-09-18T03:03:32', 'last-modified': '2025-09-18T03:03:32', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.441159'} 2025-09-18 03:03:33,294 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-09-18 03:03:33,300 - DEBUG - Control-node 10.0.0.22 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-70360481:ctest-right_vn-34613860', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-70360481', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '10236558971293549395', 'uuid-lslong': '12046760626630896631'}, 'enable': 'true', 'created': '2025-09-18T03:03:32', 'last-modified': '2025-09-18T03:03:32', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.460367'} 2025-09-18 03:03:33,306 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-09-18 03:03:33,314 - DEBUG - Control-node 10.0.0.41 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-70360481:ctest-right_vn-34613860', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-70360481', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '10236558971293549395', 'uuid-lslong': '12046760626630896631'}, 'enable': 'true', 'created': '2025-09-18T03:03:32', 'last-modified': '2025-09-18T03:03:32', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.470054'} 2025-09-18 03:03:33,323 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-09-18 03:03:33,323 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-right_vn-34613860 passed 2025-09-18 03:03:33,324 - DEBUG - ====Verifying policy data for ctest-right_vn-34613860 in API_Server ====== 2025-09-18 03:03:33,324 - DEBUG - Requesting: http://10.0.0.56:8082/domains 2025-09-18 03:03:33,332 - DEBUG - Requesting: http://10.0.0.56:8082/domain/245899c5-19ba-4930-be6a-859f116860e1 2025-09-18 03:03:33,344 - DEBUG - Requesting: http://10.0.0.56:8082/project/4f4e4097-dcce-4868-a9c2-29d74f1abd03 2025-09-18 03:03:33,468 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/8e0f9026-164a-4353-a72e-b0f6a47f2bf7 2025-09-18 03:03:33,485 - DEBUG - =>VN ctest-right_vn-34613860 has no policy to be verified 2025-09-18 03:03:33,485 - DEBUG - Verifying the vn in opserver 2025-09-18 03:03:33,485 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-70360481:ctest-right_vn-34613860 virtual network link through opserver 10.0.0.56 2025-09-18 03:03:33,485 - DEBUG - Requesting: http://10.0.0.56:8081/analytics/uves/virtual-networks 2025-09-18 03:03:33,552 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-70360481:ctest-right_vn-34613860', 'href': 'http://10.0.0.56:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-70360481:ctest-right_vn-34613860?flat'} 2025-09-18 03:03:33,552 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-70360481:ctest-right_vn-34613860 virtual network link through opserver 10.0.0.22 2025-09-18 03:03:33,552 - DEBUG - Requesting: http://10.0.0.22:8081/analytics/uves/virtual-networks 2025-09-18 03:03:33,623 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-70360481:ctest-right_vn-34613860', 'href': 'http://10.0.0.22:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-70360481:ctest-right_vn-34613860?flat'} 2025-09-18 03:03:33,623 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-70360481:ctest-right_vn-34613860 virtual network link through opserver 10.0.0.41 2025-09-18 03:03:33,623 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/virtual-networks 2025-09-18 03:03:33,695 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-70360481:ctest-right_vn-34613860', 'href': 'http://10.0.0.41:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-70360481:ctest-right_vn-34613860?flat'} 2025-09-18 03:03:33,695 - INFO - Validated that VN default-domain:ctest-TestSvcHC-70360481:ctest-right_vn-34613860 is found in opserver 2025-09-18 03:03:33,722 - DEBUG - Do not have enough data to verify VN in agent 2025-09-18 03:03:33,735 - DEBUG - VRF ids for VN ctest-right_vn-34613860: {} 2025-09-18 03:03:33,809 - DEBUG - Services list from nova: [, , , ] 2025-09-18 03:03:35,053 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4130-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4130-1) 2025-09-18 03:03:36,556 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4130-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4130-2) 2025-09-18 03:03:36,556 - DEBUG - Creating service template: ['default-domain', 'ctest-service_template_1-56227649'] 2025-09-18 03:03:36,869 - DEBUG - Service template: ['default-domain', 'ctest-service_template_1-56227649'] created succesfully 2025-09-18 03:03:37,290 - DEBUG - Image by name tiny_nat_fw not found 2025-09-18 03:03:37,290 - DEBUG - Installing image tiny_nat_fw 2025-09-18 03:03:37,290 - DEBUG - Image info {'name': 'tinycore-in-network-nat.qcow2.gz', 'webserver': '', 'location': '/images/tinycore/', 'username': 'tc', 'password': 'secret', 'type': 'os', 'flavor': 'contrail_flavor_tiny', 'params': '--container-format bare --disk-format vmdk --property vmware_disktype="sparse" --property vmware_adaptertype="ide" --property hypervisor_type=qemu', 'name_docker': 'phusion-baseimage-enablesshd', 'vmdk': 'service_chain_images/nat_service/tiny_nat_fw/tinycore-in-network-nat-flat.vmdk', 'shrinked_vmdk': "'no'"} 2025-09-18 03:03:37,290 - DEBUG - Image build path file:///contrail-test/images/tinycore-in-network-nat.qcow2.gz 2025-09-18 03:03:37,290 - DEBUG - Download image from local file file:///contrail-test/images/tinycore-in-network-nat.qcow2.gz 2025-09-18 03:03:37,291 - DEBUG - Unzip image 2025-09-18 03:03:37,291 - DEBUG - Image download (local=True) (proxy=None): Execute cmd: gunzip -f /contrail-test/images/tinycore-in-network-nat.qcow2.gz 2025-09-18 03:03:37,336 - INFO - image /contrail-test/images/tinycore-in-network-nat.qcow2 is not found 2025-09-18 03:03:37,336 - DEBUG - Deleting service template: ['default-domain', 'ctest-service_template_1-56227649'] 2025-09-18 03:03:37,425 - INFO - Service template: ['default-domain', 'ctest-service_template_1-56227649'] deleted successfully. 2025-09-18 03:03:37,426 - INFO - Skip interface_detach for VM ctest-right_vm-93492538 in state BUILD 2025-09-18 03:03:37,426 - INFO - Deleting VM ctest-right_vm-93492538 2025-09-18 03:03:37,477 - ERROR - VM ctest-right_vm-93492538 has failed to come up 2025-09-18 03:03:37,478 - ERROR - Nova failed to get host of the VM 2025-09-18 03:03:37,478 - ERROR - VM ctest-right_vm-93492538 has failed to come up 2025-09-18 03:03:37,478 - ERROR - Nova failed to get host of the VM 2025-09-18 03:03:37,582 - INFO - Skip interface_detach for VM ctest-left_vm-66101284 in state BUILD 2025-09-18 03:03:37,582 - INFO - Deleting VM ctest-left_vm-66101284 2025-09-18 03:03:37,582 - ERROR - VM ctest-left_vm-66101284 has failed to come up 2025-09-18 03:03:37,582 - ERROR - Nova failed to get host of the VM 2025-09-18 03:03:37,582 - ERROR - VM ctest-left_vm-66101284 has failed to come up 2025-09-18 03:03:37,582 - ERROR - Nova failed to get host of the VM 2025-09-18 03:03:37,672 - INFO - Deleting VN ctest-right_vn-34613860 2025-09-18 03:03:37,720 - DEBUG - VN 8e0f9026-164a-4353-a72e-b0f6a47f2bf7 still in use: Unable to complete operation on network 8e0f9026-164a-4353-a72e-b0f6a47f2bf7. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-8a8a732f-ff7f-47e7-8e60-9dcb32204a3c'] 2025-09-18 03:03:37,720 - WARNING - Deleting VN ctest-right_vn-34613860 failed..Will retry 2025-09-18 03:03:39,766 - DEBUG - VN 8e0f9026-164a-4353-a72e-b0f6a47f2bf7 still in use: Unable to complete operation on network 8e0f9026-164a-4353-a72e-b0f6a47f2bf7. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-b89cdc72-c7f9-40d3-a200-f1babae80856'] 2025-09-18 03:03:39,766 - WARNING - Deleting VN ctest-right_vn-34613860 failed..Will retry 2025-09-18 03:03:41,820 - DEBUG - VN 8e0f9026-164a-4353-a72e-b0f6a47f2bf7 still in use: Unable to complete operation on network 8e0f9026-164a-4353-a72e-b0f6a47f2bf7. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-99bbe5e0-d097-43ff-82fc-43fddb6f3efc'] 2025-09-18 03:03:41,820 - WARNING - Deleting VN ctest-right_vn-34613860 failed..Will retry 2025-09-18 03:03:43,867 - DEBUG - VN 8e0f9026-164a-4353-a72e-b0f6a47f2bf7 still in use: Unable to complete operation on network 8e0f9026-164a-4353-a72e-b0f6a47f2bf7. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-23486d9a-3919-4ba3-a330-815b87732096'] 2025-09-18 03:03:43,868 - WARNING - Deleting VN ctest-right_vn-34613860 failed..Will retry 2025-09-18 03:03:45,910 - DEBUG - VN 8e0f9026-164a-4353-a72e-b0f6a47f2bf7 still in use: Unable to complete operation on network 8e0f9026-164a-4353-a72e-b0f6a47f2bf7. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-a4202f77-8bd5-4cee-a1de-11e165961cd0'] 2025-09-18 03:03:45,910 - WARNING - Deleting VN ctest-right_vn-34613860 failed..Will retry 2025-09-18 03:03:48,127 - DEBUG - Response for deleting network () 2025-09-18 03:03:48,127 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/608b47b7-6037-400f-8b12-cea26700262f 2025-09-18 03:03:48,138 - DEBUG - Response Code: 404 2025-09-18 03:03:48,138 - DEBUG - Requesting: http://10.0.0.56:8082/domains 2025-09-18 03:03:48,146 - DEBUG - Requesting: http://10.0.0.56:8082/domain/245899c5-19ba-4930-be6a-859f116860e1 2025-09-18 03:03:48,213 - DEBUG - Requesting: http://10.0.0.56:8082/project/4f4e4097-dcce-4868-a9c2-29d74f1abd03 2025-09-18 03:03:48,307 - INFO - Validated that VN ctest-right_vn-34613860 is not found in API Server 2025-09-18 03:03:48,322 - DEBUG - VN ctest-right_vn-34613860 is not present in Agent 10.0.0.61 2025-09-18 03:03:48,347 - DEBUG - VN ctest-right_vn-34613860 is not present in Agent 10.0.0.27 2025-09-18 03:03:48,347 - INFO - Validated that VN ctest-right_vn-34613860 is not in any agent 2025-09-18 03:03:48,394 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-right_vn-34613860 info 2025-09-18 03:03:48,394 - INFO - Deleting VN ctest-left_vn-56166373 2025-09-18 03:03:48,624 - DEBUG - Response for deleting network () 2025-09-18 03:03:48,624 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/6958e486-8193-4e43-a4b3-2f842c28df2a 2025-09-18 03:03:48,633 - DEBUG - Response Code: 404 2025-09-18 03:03:48,634 - DEBUG - Requesting: http://10.0.0.56:8082/domains 2025-09-18 03:03:48,640 - DEBUG - Requesting: http://10.0.0.56:8082/domain/245899c5-19ba-4930-be6a-859f116860e1 2025-09-18 03:03:48,699 - DEBUG - Requesting: http://10.0.0.56:8082/project/4f4e4097-dcce-4868-a9c2-29d74f1abd03 2025-09-18 03:03:48,796 - INFO - Validated that VN ctest-left_vn-56166373 is not found in API Server 2025-09-18 03:03:48,811 - DEBUG - VN ctest-left_vn-56166373 is not present in Agent 10.0.0.61 2025-09-18 03:03:48,828 - DEBUG - VN ctest-left_vn-56166373 is not present in Agent 10.0.0.27 2025-09-18 03:03:48,828 - INFO - Validated that VN ctest-left_vn-56166373 is not in any agent 2025-09-18 03:03:48,863 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-left_vn-56166373 info 2025-09-18 03:03:48,863 - INFO - Deleting VN ctest-mgmt_vn-87643786 2025-09-18 03:03:49,034 - DEBUG - Response for deleting network () 2025-09-18 03:03:49,034 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/93b87f9e-f07f-492a-a935-dfde2868ce7a 2025-09-18 03:03:49,040 - DEBUG - Response Code: 404 2025-09-18 03:03:49,041 - DEBUG - Requesting: http://10.0.0.56:8082/domains 2025-09-18 03:03:49,067 - DEBUG - Requesting: http://10.0.0.56:8082/domain/245899c5-19ba-4930-be6a-859f116860e1 2025-09-18 03:03:49,087 - DEBUG - Requesting: http://10.0.0.56:8082/project/4f4e4097-dcce-4868-a9c2-29d74f1abd03 2025-09-18 03:03:49,210 - INFO - Validated that VN ctest-mgmt_vn-87643786 is not found in API Server 2025-09-18 03:03:49,223 - DEBUG - VN ctest-mgmt_vn-87643786 is not present in Agent 10.0.0.61 2025-09-18 03:03:49,238 - DEBUG - VN ctest-mgmt_vn-87643786 is not present in Agent 10.0.0.27 2025-09-18 03:03:49,238 - INFO - Validated that VN ctest-mgmt_vn-87643786 is not in any agent 2025-09-18 03:03:49,287 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-mgmt_vn-87643786 info 2025-09-18 03:03:50,593 - DEBUG - No XMPP flaps were noticed during the test 2025-09-18 03:03:50,593 - INFO - 2025-09-18 03:03:50,594 - INFO - END TEST : test_svc_hc_basic : SKIPPED[0:00:24] 2025-09-18 03:03:50,594 - INFO - -------------------------------------------------------------------------------- 2025-09-18 03:03:50,597 - INFO - ================================================================================ 2025-09-18 03:03:50,597 - INFO - STARTING TEST : test_svc_trans_segment_right_hc_fail_svm 2025-09-18 03:03:51,885 - DEBUG - Nothing to compare xmpp stats {'10.0.0.61': {'10.20.0.17': '0', '10.20.0.25': '0'}, '10.0.0.27': {'10.20.0.17': '0', '10.20.0.25': '0'}} with 2025-09-18 03:03:51,885 - INFO - Initial checks done. Running the testcase now 2025-09-18 03:03:51,886 - INFO - 2025-09-18 03:03:52,170 - DEBUG - Response for create_network : {'network': {'id': 'd48fe0de-3594-42b7-a18e-d973fe397fba', 'name': 'ctest-mgmt_vn-93094349', 'tenant_id': '4f4e4097dcce4868a9c229d74f1abd03', 'project_id': '4f4e4097dcce4868a9c229d74f1abd03', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcHC-70360481', 'ctest-mgmt_vn-93094349'], 'port_security_enabled': True, 'description': ''}} 2025-09-18 03:03:52,454 - DEBUG - Response for create_subnet : {'subnet': {'id': 'd5c6f465-1753-4117-a1e2-369194f6573d', 'name': '', 'tenant_id': '4f4e4097dcce4868a9c229d74f1abd03', 'network_id': 'd48fe0de-3594-42b7-a18e-d973fe397fba', 'ip_version': 4, 'cidr': '91.169.158.0/26', 'allocation_pools': [{'start': '91.169.158.2', 'end': '91.169.158.62'}], 'gateway_ip': '91.169.158.1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '91.169.158.2', 'tags': [], 'project_id': '4f4e4097dcce4868a9c229d74f1abd03'}} 2025-09-18 03:03:52,527 - INFO - Created VN ctest-mgmt_vn-93094349 2025-09-18 03:03:52,539 - DEBUG - VN ctest-mgmt_vn-93094349 UUID is d48fe0de-3594-42b7-a18e-d973fe397fba 2025-09-18 03:03:52,685 - DEBUG - Requesting: http://10.0.0.56:8082/domains 2025-09-18 03:03:52,694 - DEBUG - Requesting: http://10.0.0.56:8082/domain/245899c5-19ba-4930-be6a-859f116860e1 2025-09-18 03:03:52,756 - DEBUG - Requesting: http://10.0.0.56:8082/project/4f4e4097-dcce-4868-a9c2-29d74f1abd03 2025-09-18 03:03:52,867 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/d48fe0de-3594-42b7-a18e-d973fe397fba 2025-09-18 03:03:52,903 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/d48fe0de-3594-42b7-a18e-d973fe397fba 2025-09-18 03:03:52,926 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/27dc4a88-2eec-48d3-97e9-3711f4c010e9 2025-09-18 03:03:52,952 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/27dc4a88-2eec-48d3-97e9-3711f4c010e9 2025-09-18 03:03:52,980 - DEBUG - Requesting: http://10.0.0.56:8082/route-target/fe0a01dd-e069-42ad-8bef-a8464084186a 2025-09-18 03:03:53,018 - DEBUG - Route Targets: ['target:64512:8000004'] 2025-09-18 03:03:53,018 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/d48fe0de-3594-42b7-a18e-d973fe397fba 2025-09-18 03:03:53,056 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/27dc4a88-2eec-48d3-97e9-3711f4c010e9 2025-09-18 03:03:53,080 - INFO - Verified VN network id 12 for VN d48fe0de-3594-42b7-a18e-d973fe397fba 2025-09-18 03:03:53,080 - INFO - Verifications in API Server for VN ctest-mgmt_vn-93094349 passed 2025-09-18 03:03:53,080 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/d48fe0de-3594-42b7-a18e-d973fe397fba 2025-09-18 03:03:53,098 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/27dc4a88-2eec-48d3-97e9-3711f4c010e9 2025-09-18 03:03:53,116 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/27dc4a88-2eec-48d3-97e9-3711f4c010e9 2025-09-18 03:03:53,133 - DEBUG - Requesting: http://10.0.0.56:8082/route-target/fe0a01dd-e069-42ad-8bef-a8464084186a 2025-09-18 03:03:53,156 - DEBUG - Control-node 10.0.0.56 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-70360481:ctest-mgmt_vn-93094349', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-70360481', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '15316708102696616631', 'uuid-lslong': '11641481179008958394'}, 'enable': 'true', 'created': '2025-09-18T03:03:52', 'last-modified': '2025-09-18T03:03:52', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.757668'} 2025-09-18 03:03:53,175 - DEBUG - Route Targets: ['target:64512:8000004'] 2025-09-18 03:03:53,193 - DEBUG - Control-node 10.0.0.22 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-70360481:ctest-mgmt_vn-93094349', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-70360481', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '15316708102696616631', 'uuid-lslong': '11641481179008958394'}, 'enable': 'true', 'created': '2025-09-18T03:03:52', 'last-modified': '2025-09-18T03:03:52', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.801983'} 2025-09-18 03:03:53,202 - DEBUG - Route Targets: ['target:64512:8000004'] 2025-09-18 03:03:53,209 - DEBUG - Control-node 10.0.0.41 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-70360481:ctest-mgmt_vn-93094349', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-70360481', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '15316708102696616631', 'uuid-lslong': '11641481179008958394'}, 'enable': 'true', 'created': '2025-09-18T03:03:52', 'last-modified': '2025-09-18T03:03:52', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.817524'} 2025-09-18 03:03:53,219 - DEBUG - Route Targets: ['target:64512:8000004'] 2025-09-18 03:03:53,220 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-mgmt_vn-93094349 passed 2025-09-18 03:03:53,221 - DEBUG - ====Verifying policy data for ctest-mgmt_vn-93094349 in API_Server ====== 2025-09-18 03:03:53,221 - DEBUG - Requesting: http://10.0.0.56:8082/domains 2025-09-18 03:03:53,239 - DEBUG - Requesting: http://10.0.0.56:8082/domain/245899c5-19ba-4930-be6a-859f116860e1 2025-09-18 03:03:53,273 - DEBUG - Requesting: http://10.0.0.56:8082/project/4f4e4097-dcce-4868-a9c2-29d74f1abd03 2025-09-18 03:03:53,394 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/d48fe0de-3594-42b7-a18e-d973fe397fba 2025-09-18 03:03:53,416 - DEBUG - =>VN ctest-mgmt_vn-93094349 has no policy to be verified 2025-09-18 03:03:53,416 - DEBUG - Verifying the vn in opserver 2025-09-18 03:03:53,416 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-70360481:ctest-mgmt_vn-93094349 virtual network link through opserver 10.0.0.56 2025-09-18 03:03:53,416 - DEBUG - Requesting: http://10.0.0.56:8081/analytics/uves/virtual-networks 2025-09-18 03:03:53,431 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-70360481:ctest-mgmt_vn-93094349', 'href': 'http://10.0.0.56:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-70360481:ctest-mgmt_vn-93094349?flat'} 2025-09-18 03:03:53,431 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-70360481:ctest-mgmt_vn-93094349 virtual network link through opserver 10.0.0.22 2025-09-18 03:03:53,431 - DEBUG - Requesting: http://10.0.0.22:8081/analytics/uves/virtual-networks 2025-09-18 03:03:53,447 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-70360481:ctest-mgmt_vn-93094349', 'href': 'http://10.0.0.22:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-70360481:ctest-mgmt_vn-93094349?flat'} 2025-09-18 03:03:53,447 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-70360481:ctest-mgmt_vn-93094349 virtual network link through opserver 10.0.0.41 2025-09-18 03:03:53,448 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/virtual-networks 2025-09-18 03:03:53,461 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-70360481:ctest-mgmt_vn-93094349', 'href': 'http://10.0.0.41:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-70360481:ctest-mgmt_vn-93094349?flat'} 2025-09-18 03:03:53,461 - INFO - Validated that VN default-domain:ctest-TestSvcHC-70360481:ctest-mgmt_vn-93094349 is found in opserver 2025-09-18 03:03:53,492 - DEBUG - Do not have enough data to verify VN in agent 2025-09-18 03:03:53,507 - DEBUG - VRF ids for VN ctest-mgmt_vn-93094349: {} 2025-09-18 03:03:53,828 - DEBUG - Response for create_network : {'network': {'id': '62fd1d84-7e43-4606-b5ac-649496d14ee0', 'name': 'ctest-left_vn-19206875', 'tenant_id': '4f4e4097dcce4868a9c229d74f1abd03', 'project_id': '4f4e4097dcce4868a9c229d74f1abd03', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcHC-70360481', 'ctest-left_vn-19206875'], 'port_security_enabled': True, 'description': ''}} 2025-09-18 03:03:54,125 - DEBUG - Response for create_subnet : {'subnet': {'id': '88e4b7bb-46ee-443d-844e-0ba96c2ac61f', 'name': '', 'tenant_id': '4f4e4097dcce4868a9c229d74f1abd03', 'network_id': '62fd1d84-7e43-4606-b5ac-649496d14ee0', 'ip_version': 4, 'cidr': '154.9.83.0/26', 'allocation_pools': [{'start': '154.9.83.2', 'end': '154.9.83.62'}], 'gateway_ip': '154.9.83.1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '154.9.83.2', 'tags': [], 'project_id': '4f4e4097dcce4868a9c229d74f1abd03'}} 2025-09-18 03:03:54,149 - INFO - Created VN ctest-left_vn-19206875 2025-09-18 03:03:54,172 - DEBUG - VN ctest-left_vn-19206875 UUID is 62fd1d84-7e43-4606-b5ac-649496d14ee0 2025-09-18 03:03:54,253 - DEBUG - Requesting: http://10.0.0.56:8082/domains 2025-09-18 03:03:54,261 - DEBUG - Requesting: http://10.0.0.56:8082/domain/245899c5-19ba-4930-be6a-859f116860e1 2025-09-18 03:03:54,275 - DEBUG - Requesting: http://10.0.0.56:8082/project/4f4e4097-dcce-4868-a9c2-29d74f1abd03 2025-09-18 03:03:54,383 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/62fd1d84-7e43-4606-b5ac-649496d14ee0 2025-09-18 03:03:54,396 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/62fd1d84-7e43-4606-b5ac-649496d14ee0 2025-09-18 03:03:54,411 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/3b7fc812-9e79-4183-ad38-39b4c999b2cb 2025-09-18 03:03:54,422 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/3b7fc812-9e79-4183-ad38-39b4c999b2cb 2025-09-18 03:03:54,431 - DEBUG - Requesting: http://10.0.0.56:8082/route-target/5be389b6-0e7a-4be7-b3b5-182c45dbfadf 2025-09-18 03:03:54,439 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-09-18 03:03:54,439 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/62fd1d84-7e43-4606-b5ac-649496d14ee0 2025-09-18 03:03:54,449 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/3b7fc812-9e79-4183-ad38-39b4c999b2cb 2025-09-18 03:03:54,467 - INFO - Verified VN network id 13 for VN 62fd1d84-7e43-4606-b5ac-649496d14ee0 2025-09-18 03:03:54,467 - INFO - Verifications in API Server for VN ctest-left_vn-19206875 passed 2025-09-18 03:03:54,467 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/62fd1d84-7e43-4606-b5ac-649496d14ee0 2025-09-18 03:03:54,478 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/3b7fc812-9e79-4183-ad38-39b4c999b2cb 2025-09-18 03:03:54,487 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/3b7fc812-9e79-4183-ad38-39b4c999b2cb 2025-09-18 03:03:54,495 - DEBUG - Requesting: http://10.0.0.56:8082/route-target/5be389b6-0e7a-4be7-b3b5-182c45dbfadf 2025-09-18 03:03:54,512 - DEBUG - Control-node 10.0.0.56 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-70360481:ctest-left_vn-19206875', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-70360481', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '7132889839715960326', 'uuid-lslong': '13090948806207426272'}, 'enable': 'true', 'created': '2025-09-18T03:03:53', 'last-modified': '2025-09-18T03:03:54', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.462150'} 2025-09-18 03:03:54,518 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-09-18 03:03:54,524 - DEBUG - Control-node 10.0.0.22 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-70360481:ctest-left_vn-19206875', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-70360481', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '7132889839715960326', 'uuid-lslong': '13090948806207426272'}, 'enable': 'true', 'created': '2025-09-18T03:03:53', 'last-modified': '2025-09-18T03:03:54', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.486505'} 2025-09-18 03:03:54,531 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-09-18 03:03:54,536 - DEBUG - Control-node 10.0.0.41 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-70360481:ctest-left_vn-19206875', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-70360481', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '7132889839715960326', 'uuid-lslong': '13090948806207426272'}, 'enable': 'true', 'created': '2025-09-18T03:03:53', 'last-modified': '2025-09-18T03:03:54', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.492449'} 2025-09-18 03:03:54,542 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-09-18 03:03:54,542 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-left_vn-19206875 passed 2025-09-18 03:03:54,542 - DEBUG - ====Verifying policy data for ctest-left_vn-19206875 in API_Server ====== 2025-09-18 03:03:54,542 - DEBUG - Requesting: http://10.0.0.56:8082/domains 2025-09-18 03:03:54,549 - DEBUG - Requesting: http://10.0.0.56:8082/domain/245899c5-19ba-4930-be6a-859f116860e1 2025-09-18 03:03:54,561 - DEBUG - Requesting: http://10.0.0.56:8082/project/4f4e4097-dcce-4868-a9c2-29d74f1abd03 2025-09-18 03:03:54,674 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/62fd1d84-7e43-4606-b5ac-649496d14ee0 2025-09-18 03:03:54,700 - DEBUG - =>VN ctest-left_vn-19206875 has no policy to be verified 2025-09-18 03:03:54,701 - DEBUG - Verifying the vn in opserver 2025-09-18 03:03:54,701 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-70360481:ctest-left_vn-19206875 virtual network link through opserver 10.0.0.56 2025-09-18 03:03:54,701 - DEBUG - Requesting: http://10.0.0.56:8081/analytics/uves/virtual-networks 2025-09-18 03:03:54,715 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-70360481:ctest-left_vn-19206875', 'href': 'http://10.0.0.56:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-70360481:ctest-left_vn-19206875?flat'} 2025-09-18 03:03:54,715 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-70360481:ctest-left_vn-19206875 virtual network link through opserver 10.0.0.22 2025-09-18 03:03:54,715 - DEBUG - Requesting: http://10.0.0.22:8081/analytics/uves/virtual-networks 2025-09-18 03:03:54,728 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-70360481:ctest-left_vn-19206875', 'href': 'http://10.0.0.22:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-70360481:ctest-left_vn-19206875?flat'} 2025-09-18 03:03:54,728 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-70360481:ctest-left_vn-19206875 virtual network link through opserver 10.0.0.41 2025-09-18 03:03:54,728 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/virtual-networks 2025-09-18 03:03:54,743 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-70360481:ctest-left_vn-19206875', 'href': 'http://10.0.0.41:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-70360481:ctest-left_vn-19206875?flat'} 2025-09-18 03:03:54,743 - INFO - Validated that VN default-domain:ctest-TestSvcHC-70360481:ctest-left_vn-19206875 is found in opserver 2025-09-18 03:03:54,770 - DEBUG - Do not have enough data to verify VN in agent 2025-09-18 03:03:54,784 - DEBUG - VRF ids for VN ctest-left_vn-19206875: {} 2025-09-18 03:03:55,135 - DEBUG - Response for create_network : {'network': {'id': '93d70779-f831-4133-a848-6c77306e09fa', 'name': 'ctest-right_vn-29803132', 'tenant_id': '4f4e4097dcce4868a9c229d74f1abd03', 'project_id': '4f4e4097dcce4868a9c229d74f1abd03', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcHC-70360481', 'ctest-right_vn-29803132'], 'port_security_enabled': True, 'description': ''}} 2025-09-18 03:03:55,396 - DEBUG - Response for create_subnet : {'subnet': {'id': 'def55a09-a794-4b3d-aa94-1e7a8f213ae0', 'name': '', 'tenant_id': '4f4e4097dcce4868a9c229d74f1abd03', 'network_id': '93d70779-f831-4133-a848-6c77306e09fa', 'ip_version': 4, 'cidr': '111.74.133.64/26', 'allocation_pools': [{'start': '111.74.133.66', 'end': '111.74.133.126'}], 'gateway_ip': '111.74.133.65', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '111.74.133.66', 'tags': [], 'project_id': '4f4e4097dcce4868a9c229d74f1abd03'}} 2025-09-18 03:03:55,430 - INFO - Created VN ctest-right_vn-29803132 2025-09-18 03:03:55,442 - DEBUG - VN ctest-right_vn-29803132 UUID is 93d70779-f831-4133-a848-6c77306e09fa 2025-09-18 03:03:55,556 - DEBUG - Requesting: http://10.0.0.56:8082/domains 2025-09-18 03:03:55,565 - DEBUG - Requesting: http://10.0.0.56:8082/domain/245899c5-19ba-4930-be6a-859f116860e1 2025-09-18 03:03:55,583 - DEBUG - Requesting: http://10.0.0.56:8082/project/4f4e4097-dcce-4868-a9c2-29d74f1abd03 2025-09-18 03:03:55,712 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/93d70779-f831-4133-a848-6c77306e09fa 2025-09-18 03:03:55,729 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/93d70779-f831-4133-a848-6c77306e09fa 2025-09-18 03:03:55,742 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/edce6ac8-4ce5-48ca-8207-fdd22fa72150 2025-09-18 03:03:55,754 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/edce6ac8-4ce5-48ca-8207-fdd22fa72150 2025-09-18 03:03:55,766 - DEBUG - Requesting: http://10.0.0.56:8082/route-target/ea89ba92-64d8-46ec-8361-0ec78665d644 2025-09-18 03:03:55,775 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-09-18 03:03:55,775 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/93d70779-f831-4133-a848-6c77306e09fa 2025-09-18 03:03:55,786 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/edce6ac8-4ce5-48ca-8207-fdd22fa72150 2025-09-18 03:03:55,807 - INFO - Verified VN network id 14 for VN 93d70779-f831-4133-a848-6c77306e09fa 2025-09-18 03:03:55,807 - INFO - Verifications in API Server for VN ctest-right_vn-29803132 passed 2025-09-18 03:03:55,807 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/93d70779-f831-4133-a848-6c77306e09fa 2025-09-18 03:03:55,819 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/edce6ac8-4ce5-48ca-8207-fdd22fa72150 2025-09-18 03:03:55,827 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/edce6ac8-4ce5-48ca-8207-fdd22fa72150 2025-09-18 03:03:55,835 - DEBUG - Requesting: http://10.0.0.56:8082/route-target/ea89ba92-64d8-46ec-8361-0ec78665d644 2025-09-18 03:03:55,851 - DEBUG - Control-node 10.0.0.56 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-70360481:ctest-right_vn-29803132', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-70360481', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '10652991664004612403', 'uuid-lslong': '12126061255864486394'}, 'enable': 'true', 'created': '2025-09-18T03:03:55', 'last-modified': '2025-09-18T03:03:55', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.501858'} 2025-09-18 03:03:55,858 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-09-18 03:03:55,865 - DEBUG - Control-node 10.0.0.22 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-70360481:ctest-right_vn-29803132', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-70360481', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '10652991664004612403', 'uuid-lslong': '12126061255864486394'}, 'enable': 'true', 'created': '2025-09-18T03:03:55', 'last-modified': '2025-09-18T03:03:55', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.524930'} 2025-09-18 03:03:55,871 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-09-18 03:03:55,877 - DEBUG - Control-node 10.0.0.41 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-70360481:ctest-right_vn-29803132', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-70360481', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '10652991664004612403', 'uuid-lslong': '12126061255864486394'}, 'enable': 'true', 'created': '2025-09-18T03:03:55', 'last-modified': '2025-09-18T03:03:55', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.534504'} 2025-09-18 03:03:55,883 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-09-18 03:03:55,884 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-right_vn-29803132 passed 2025-09-18 03:03:55,884 - DEBUG - ====Verifying policy data for ctest-right_vn-29803132 in API_Server ====== 2025-09-18 03:03:55,884 - DEBUG - Requesting: http://10.0.0.56:8082/domains 2025-09-18 03:03:55,891 - DEBUG - Requesting: http://10.0.0.56:8082/domain/245899c5-19ba-4930-be6a-859f116860e1 2025-09-18 03:03:55,905 - DEBUG - Requesting: http://10.0.0.56:8082/project/4f4e4097-dcce-4868-a9c2-29d74f1abd03 2025-09-18 03:03:56,008 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/93d70779-f831-4133-a848-6c77306e09fa 2025-09-18 03:03:56,024 - DEBUG - =>VN ctest-right_vn-29803132 has no policy to be verified 2025-09-18 03:03:56,024 - DEBUG - Verifying the vn in opserver 2025-09-18 03:03:56,024 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-70360481:ctest-right_vn-29803132 virtual network link through opserver 10.0.0.56 2025-09-18 03:03:56,024 - DEBUG - Requesting: http://10.0.0.56:8081/analytics/uves/virtual-networks 2025-09-18 03:03:56,037 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-70360481:ctest-right_vn-29803132', 'href': 'http://10.0.0.56:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-70360481:ctest-right_vn-29803132?flat'} 2025-09-18 03:03:56,037 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-70360481:ctest-right_vn-29803132 virtual network link through opserver 10.0.0.22 2025-09-18 03:03:56,037 - DEBUG - Requesting: http://10.0.0.22:8081/analytics/uves/virtual-networks 2025-09-18 03:03:56,051 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-70360481:ctest-right_vn-29803132', 'href': 'http://10.0.0.22:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-70360481:ctest-right_vn-29803132?flat'} 2025-09-18 03:03:56,051 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-70360481:ctest-right_vn-29803132 virtual network link through opserver 10.0.0.41 2025-09-18 03:03:56,051 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/virtual-networks 2025-09-18 03:03:56,064 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-70360481:ctest-right_vn-29803132', 'href': 'http://10.0.0.41:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-70360481:ctest-right_vn-29803132?flat'} 2025-09-18 03:03:56,064 - INFO - Validated that VN default-domain:ctest-TestSvcHC-70360481:ctest-right_vn-29803132 is found in opserver 2025-09-18 03:03:56,094 - DEBUG - Do not have enough data to verify VN in agent 2025-09-18 03:03:56,113 - DEBUG - VRF ids for VN ctest-right_vn-29803132: {} 2025-09-18 03:03:56,383 - DEBUG - Response for create_network : {'network': {'id': 'c7679b79-d870-43c4-a025-ccb5bb6f19fc', 'name': 'ctest-trans_left_vn-61271344', 'tenant_id': '4f4e4097dcce4868a9c229d74f1abd03', 'project_id': '4f4e4097dcce4868a9c229d74f1abd03', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcHC-70360481', 'ctest-trans_left_vn-61271344'], 'port_security_enabled': True, 'description': ''}} 2025-09-18 03:03:56,599 - DEBUG - Response for create_subnet : {'subnet': {'id': '16ede626-b7f1-4e6d-bc0a-28d20d34135c', 'name': '', 'tenant_id': '4f4e4097dcce4868a9c229d74f1abd03', 'network_id': 'c7679b79-d870-43c4-a025-ccb5bb6f19fc', 'ip_version': 4, 'cidr': '71.228.160.192/26', 'allocation_pools': [{'start': '71.228.160.194', 'end': '71.228.160.254'}], 'gateway_ip': '71.228.160.193', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '71.228.160.194', 'tags': [], 'project_id': '4f4e4097dcce4868a9c229d74f1abd03'}} 2025-09-18 03:03:56,627 - INFO - Created VN ctest-trans_left_vn-61271344 2025-09-18 03:03:56,638 - DEBUG - VN ctest-trans_left_vn-61271344 UUID is c7679b79-d870-43c4-a025-ccb5bb6f19fc 2025-09-18 03:03:56,721 - DEBUG - Requesting: http://10.0.0.56:8082/domains 2025-09-18 03:03:56,727 - DEBUG - Requesting: http://10.0.0.56:8082/domain/245899c5-19ba-4930-be6a-859f116860e1 2025-09-18 03:03:56,740 - DEBUG - Requesting: http://10.0.0.56:8082/project/4f4e4097-dcce-4868-a9c2-29d74f1abd03 2025-09-18 03:03:56,845 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/c7679b79-d870-43c4-a025-ccb5bb6f19fc 2025-09-18 03:03:56,859 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/c7679b79-d870-43c4-a025-ccb5bb6f19fc 2025-09-18 03:03:56,872 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/3266be91-1009-4384-af0a-d7696014fb48 2025-09-18 03:03:56,888 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/3266be91-1009-4384-af0a-d7696014fb48 2025-09-18 03:03:56,897 - DEBUG - Requesting: http://10.0.0.56:8082/route-target/ce4948fe-806b-40dc-bfd4-da40b9d97a43 2025-09-18 03:03:56,905 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-09-18 03:03:56,905 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/c7679b79-d870-43c4-a025-ccb5bb6f19fc 2025-09-18 03:03:56,915 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/3266be91-1009-4384-af0a-d7696014fb48 2025-09-18 03:03:56,934 - INFO - Verified VN network id 15 for VN c7679b79-d870-43c4-a025-ccb5bb6f19fc 2025-09-18 03:03:56,934 - INFO - Verifications in API Server for VN ctest-trans_left_vn-61271344 passed 2025-09-18 03:03:56,934 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/c7679b79-d870-43c4-a025-ccb5bb6f19fc 2025-09-18 03:03:56,946 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/3266be91-1009-4384-af0a-d7696014fb48 2025-09-18 03:03:56,954 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/3266be91-1009-4384-af0a-d7696014fb48 2025-09-18 03:03:56,962 - DEBUG - Requesting: http://10.0.0.56:8082/route-target/ce4948fe-806b-40dc-bfd4-da40b9d97a43 2025-09-18 03:03:56,976 - DEBUG - Control-node 10.0.0.56 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-70360481:ctest-trans_left_vn-61271344', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-70360481', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '14368624083773440964', 'uuid-lslong': '11539854701112531452'}, 'enable': 'true', 'created': '2025-09-18T03:03:56', 'last-modified': '2025-09-18T03:03:56', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.450853'} 2025-09-18 03:03:56,982 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-09-18 03:03:56,987 - DEBUG - Control-node 10.0.0.22 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-70360481:ctest-trans_left_vn-61271344', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-70360481', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '14368624083773440964', 'uuid-lslong': '11539854701112531452'}, 'enable': 'true', 'created': '2025-09-18T03:03:56', 'last-modified': '2025-09-18T03:03:56', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.447528'} 2025-09-18 03:03:56,994 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-09-18 03:03:56,999 - DEBUG - Control-node 10.0.0.41 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-70360481:ctest-trans_left_vn-61271344', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-70360481', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '14368624083773440964', 'uuid-lslong': '11539854701112531452'}, 'enable': 'true', 'created': '2025-09-18T03:03:56', 'last-modified': '2025-09-18T03:03:56', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.460498'} 2025-09-18 03:03:57,006 - DEBUG - Route Targets: ['target:64512:8000007'] 2025-09-18 03:03:57,006 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-trans_left_vn-61271344 passed 2025-09-18 03:03:57,006 - DEBUG - ====Verifying policy data for ctest-trans_left_vn-61271344 in API_Server ====== 2025-09-18 03:03:57,007 - DEBUG - Requesting: http://10.0.0.56:8082/domains 2025-09-18 03:03:57,013 - DEBUG - Requesting: http://10.0.0.56:8082/domain/245899c5-19ba-4930-be6a-859f116860e1 2025-09-18 03:03:57,025 - DEBUG - Requesting: http://10.0.0.56:8082/project/4f4e4097-dcce-4868-a9c2-29d74f1abd03 2025-09-18 03:03:57,136 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/c7679b79-d870-43c4-a025-ccb5bb6f19fc 2025-09-18 03:03:57,153 - DEBUG - =>VN ctest-trans_left_vn-61271344 has no policy to be verified 2025-09-18 03:03:57,153 - DEBUG - Verifying the vn in opserver 2025-09-18 03:03:57,154 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-70360481:ctest-trans_left_vn-61271344 virtual network link through opserver 10.0.0.56 2025-09-18 03:03:57,154 - DEBUG - Requesting: http://10.0.0.56:8081/analytics/uves/virtual-networks 2025-09-18 03:03:57,171 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-70360481:ctest-trans_left_vn-61271344', 'href': 'http://10.0.0.56:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-70360481:ctest-trans_left_vn-61271344?flat'} 2025-09-18 03:03:57,171 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-70360481:ctest-trans_left_vn-61271344 virtual network link through opserver 10.0.0.22 2025-09-18 03:03:57,171 - DEBUG - Requesting: http://10.0.0.22:8081/analytics/uves/virtual-networks 2025-09-18 03:03:57,185 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-70360481:ctest-trans_left_vn-61271344', 'href': 'http://10.0.0.22:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-70360481:ctest-trans_left_vn-61271344?flat'} 2025-09-18 03:03:57,185 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-70360481:ctest-trans_left_vn-61271344 virtual network link through opserver 10.0.0.41 2025-09-18 03:03:57,185 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/virtual-networks 2025-09-18 03:03:57,200 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-70360481:ctest-trans_left_vn-61271344', 'href': 'http://10.0.0.41:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-70360481:ctest-trans_left_vn-61271344?flat'} 2025-09-18 03:03:57,200 - INFO - Validated that VN default-domain:ctest-TestSvcHC-70360481:ctest-trans_left_vn-61271344 is found in opserver 2025-09-18 03:03:57,221 - DEBUG - Do not have enough data to verify VN in agent 2025-09-18 03:03:57,233 - DEBUG - VRF ids for VN ctest-trans_left_vn-61271344: {} 2025-09-18 03:03:57,508 - DEBUG - Response for create_network : {'network': {'id': '4e32f6ec-31b5-48f2-b85f-91a690946ae8', 'name': 'ctest-trans_right_vn-42807411', 'tenant_id': '4f4e4097dcce4868a9c229d74f1abd03', 'project_id': '4f4e4097dcce4868a9c229d74f1abd03', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestSvcHC-70360481', 'ctest-trans_right_vn-42807411'], 'port_security_enabled': True, 'description': ''}} 2025-09-18 03:03:57,786 - DEBUG - Response for create_subnet : {'subnet': {'id': '01a526e5-a320-47a3-b6bc-ce67e802d9ec', 'name': '', 'tenant_id': '4f4e4097dcce4868a9c229d74f1abd03', 'network_id': '4e32f6ec-31b5-48f2-b85f-91a690946ae8', 'ip_version': 4, 'cidr': '111.23.190.0/26', 'allocation_pools': [{'start': '111.23.190.2', 'end': '111.23.190.62'}], 'gateway_ip': '111.23.190.1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '111.23.190.2', 'tags': [], 'project_id': '4f4e4097dcce4868a9c229d74f1abd03'}} 2025-09-18 03:03:57,814 - INFO - Created VN ctest-trans_right_vn-42807411 2025-09-18 03:03:57,830 - DEBUG - VN ctest-trans_right_vn-42807411 UUID is 4e32f6ec-31b5-48f2-b85f-91a690946ae8 2025-09-18 03:03:57,913 - DEBUG - Requesting: http://10.0.0.56:8082/domains 2025-09-18 03:03:57,921 - DEBUG - Requesting: http://10.0.0.56:8082/domain/245899c5-19ba-4930-be6a-859f116860e1 2025-09-18 03:03:57,936 - DEBUG - Requesting: http://10.0.0.56:8082/project/4f4e4097-dcce-4868-a9c2-29d74f1abd03 2025-09-18 03:03:58,041 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/4e32f6ec-31b5-48f2-b85f-91a690946ae8 2025-09-18 03:03:58,053 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/4e32f6ec-31b5-48f2-b85f-91a690946ae8 2025-09-18 03:03:58,062 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/6165cea8-0942-4a53-8420-e055ccf9c30a 2025-09-18 03:03:58,070 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/6165cea8-0942-4a53-8420-e055ccf9c30a 2025-09-18 03:03:58,077 - DEBUG - Requesting: http://10.0.0.56:8082/route-target/1d334fbc-66be-41e9-9e3f-f188115e0074 2025-09-18 03:03:58,084 - DEBUG - Route Targets: ['target:64512:8000008'] 2025-09-18 03:03:58,084 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/4e32f6ec-31b5-48f2-b85f-91a690946ae8 2025-09-18 03:03:58,094 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/6165cea8-0942-4a53-8420-e055ccf9c30a 2025-09-18 03:03:58,110 - INFO - Verified VN network id 16 for VN 4e32f6ec-31b5-48f2-b85f-91a690946ae8 2025-09-18 03:03:58,110 - INFO - Verifications in API Server for VN ctest-trans_right_vn-42807411 passed 2025-09-18 03:03:58,110 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/4e32f6ec-31b5-48f2-b85f-91a690946ae8 2025-09-18 03:03:58,121 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/6165cea8-0942-4a53-8420-e055ccf9c30a 2025-09-18 03:03:58,130 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/6165cea8-0942-4a53-8420-e055ccf9c30a 2025-09-18 03:03:58,139 - DEBUG - Requesting: http://10.0.0.56:8082/route-target/1d334fbc-66be-41e9-9e3f-f188115e0074 2025-09-18 03:03:58,154 - DEBUG - Control-node 10.0.0.56 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-70360481:ctest-trans_right_vn-42807411', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-70360481', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '5634837578100590834', 'uuid-lslong': '13285497570342497000'}, 'enable': 'true', 'created': '2025-09-18T03:03:57', 'last-modified': '2025-09-18T03:03:57', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.418201'} 2025-09-18 03:03:58,160 - DEBUG - Route Targets: ['target:64512:8000008'] 2025-09-18 03:03:58,167 - DEBUG - Control-node 10.0.0.22 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-70360481:ctest-trans_right_vn-42807411', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-70360481', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '5634837578100590834', 'uuid-lslong': '13285497570342497000'}, 'enable': 'true', 'created': '2025-09-18T03:03:57', 'last-modified': '2025-09-18T03:03:57', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.435328'} 2025-09-18 03:03:58,174 - DEBUG - Route Targets: ['target:64512:8000008'] 2025-09-18 03:03:58,180 - DEBUG - Control-node 10.0.0.41 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-TestSvcHC-70360481:ctest-trans_right_vn-42807411', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-TestSvcHC-70360481', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '5634837578100590834', 'uuid-lslong': '13285497570342497000'}, 'enable': 'true', 'created': '2025-09-18T03:03:57', 'last-modified': '2025-09-18T03:03:57', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.447922'} 2025-09-18 03:03:58,188 - DEBUG - Route Targets: ['target:64512:8000008'] 2025-09-18 03:03:58,188 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-trans_right_vn-42807411 passed 2025-09-18 03:03:58,188 - DEBUG - ====Verifying policy data for ctest-trans_right_vn-42807411 in API_Server ====== 2025-09-18 03:03:58,188 - DEBUG - Requesting: http://10.0.0.56:8082/domains 2025-09-18 03:03:58,195 - DEBUG - Requesting: http://10.0.0.56:8082/domain/245899c5-19ba-4930-be6a-859f116860e1 2025-09-18 03:03:58,207 - DEBUG - Requesting: http://10.0.0.56:8082/project/4f4e4097-dcce-4868-a9c2-29d74f1abd03 2025-09-18 03:03:58,307 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/4e32f6ec-31b5-48f2-b85f-91a690946ae8 2025-09-18 03:03:58,318 - DEBUG - =>VN ctest-trans_right_vn-42807411 has no policy to be verified 2025-09-18 03:03:58,318 - DEBUG - Verifying the vn in opserver 2025-09-18 03:03:58,318 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-70360481:ctest-trans_right_vn-42807411 virtual network link through opserver 10.0.0.56 2025-09-18 03:03:58,318 - DEBUG - Requesting: http://10.0.0.56:8081/analytics/uves/virtual-networks 2025-09-18 03:03:58,331 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-70360481:ctest-trans_right_vn-42807411', 'href': 'http://10.0.0.56:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-70360481:ctest-trans_right_vn-42807411?flat'} 2025-09-18 03:03:58,331 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-70360481:ctest-trans_right_vn-42807411 virtual network link through opserver 10.0.0.22 2025-09-18 03:03:58,331 - DEBUG - Requesting: http://10.0.0.22:8081/analytics/uves/virtual-networks 2025-09-18 03:03:58,344 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-70360481:ctest-trans_right_vn-42807411', 'href': 'http://10.0.0.22:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-70360481:ctest-trans_right_vn-42807411?flat'} 2025-09-18 03:03:58,344 - DEBUG - Verifying the default-domain:ctest-TestSvcHC-70360481:ctest-trans_right_vn-42807411 virtual network link through opserver 10.0.0.41 2025-09-18 03:03:58,344 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/virtual-networks 2025-09-18 03:03:58,358 - DEBUG - vn link and name as {'name': 'default-domain:ctest-TestSvcHC-70360481:ctest-trans_right_vn-42807411', 'href': 'http://10.0.0.41:8081/analytics/uves/virtual-network/default-domain:ctest-TestSvcHC-70360481:ctest-trans_right_vn-42807411?flat'} 2025-09-18 03:03:58,358 - INFO - Validated that VN default-domain:ctest-TestSvcHC-70360481:ctest-trans_right_vn-42807411 is found in opserver 2025-09-18 03:03:58,382 - DEBUG - Do not have enough data to verify VN in agent 2025-09-18 03:03:58,397 - DEBUG - VRF ids for VN ctest-trans_right_vn-42807411: {} 2025-09-18 03:03:59,605 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4130-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4130-2) 2025-09-18 03:04:01,161 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4130-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4130-1) 2025-09-18 03:04:01,161 - DEBUG - Creating service template: ['default-domain', 'ctest-service_template_1-13812778'] 2025-09-18 03:04:01,495 - DEBUG - Service template: ['default-domain', 'ctest-service_template_1-13812778'] created succesfully 2025-09-18 03:04:01,927 - DEBUG - Image by name tiny_trans_fw not found 2025-09-18 03:04:01,927 - DEBUG - Installing image tiny_trans_fw 2025-09-18 03:04:01,928 - DEBUG - Image info {'name': 'tinycore-net-transparent-fw.qcow2.gz', 'webserver': '', 'location': '/images/tinycore/', 'username': 'tc', 'password': 'secret', 'type': 'os', 'flavor': 'contrail_flavor_tiny', 'params': '--container-format bare --disk-format vmdk --property vmware_disktype="sparse" --property vmware_adaptertype="ide" --property hypervisor_type=qemu', 'name_docker': 'phusion-baseimage-enablesshd'} 2025-09-18 03:04:01,928 - DEBUG - Image build path file:///contrail-test/images/tinycore-net-transparent-fw.qcow2.gz 2025-09-18 03:04:01,928 - DEBUG - Download image from local file file:///contrail-test/images/tinycore-net-transparent-fw.qcow2.gz 2025-09-18 03:04:01,928 - DEBUG - Unzip image 2025-09-18 03:04:01,928 - DEBUG - Image download (local=True) (proxy=None): Execute cmd: gunzip -f /contrail-test/images/tinycore-net-transparent-fw.qcow2.gz 2025-09-18 03:04:01,949 - INFO - image /contrail-test/images/tinycore-net-transparent-fw.qcow2 is not found 2025-09-18 03:04:01,950 - DEBUG - Deleting service template: ['default-domain', 'ctest-service_template_1-13812778'] 2025-09-18 03:04:02,085 - INFO - Service template: ['default-domain', 'ctest-service_template_1-13812778'] deleted successfully. 2025-09-18 03:04:02,086 - INFO - Skip interface_detach for VM ctest-right_vm-41224649 in state BUILD 2025-09-18 03:04:02,086 - INFO - Deleting VM ctest-right_vm-41224649 2025-09-18 03:04:02,086 - ERROR - VM ctest-right_vm-41224649 has failed to come up 2025-09-18 03:04:02,086 - ERROR - Nova failed to get host of the VM 2025-09-18 03:04:02,086 - ERROR - VM ctest-right_vm-41224649 has failed to come up 2025-09-18 03:04:02,086 - ERROR - Nova failed to get host of the VM 2025-09-18 03:04:02,179 - INFO - Skip interface_detach for VM ctest-left_vm-78489667 in state BUILD 2025-09-18 03:04:02,179 - INFO - Deleting VM ctest-left_vm-78489667 2025-09-18 03:04:02,180 - ERROR - VM ctest-left_vm-78489667 has failed to come up 2025-09-18 03:04:02,180 - ERROR - Nova failed to get host of the VM 2025-09-18 03:04:02,180 - ERROR - VM ctest-left_vm-78489667 has failed to come up 2025-09-18 03:04:02,180 - ERROR - Nova failed to get host of the VM 2025-09-18 03:04:02,306 - INFO - Deleting VN ctest-trans_right_vn-42807411 2025-09-18 03:04:02,487 - DEBUG - Response for deleting network () 2025-09-18 03:04:02,487 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/6165cea8-0942-4a53-8420-e055ccf9c30a 2025-09-18 03:04:02,495 - DEBUG - Response Code: 404 2025-09-18 03:04:02,495 - DEBUG - Requesting: http://10.0.0.56:8082/domains 2025-09-18 03:04:02,503 - DEBUG - Requesting: http://10.0.0.56:8082/domain/245899c5-19ba-4930-be6a-859f116860e1 2025-09-18 03:04:02,535 - DEBUG - Requesting: http://10.0.0.56:8082/project/4f4e4097-dcce-4868-a9c2-29d74f1abd03 2025-09-18 03:04:02,629 - INFO - Validated that VN ctest-trans_right_vn-42807411 is not found in API Server 2025-09-18 03:04:02,651 - DEBUG - VN ctest-trans_right_vn-42807411 is not present in Agent 10.0.0.61 2025-09-18 03:04:02,662 - DEBUG - VN ctest-trans_right_vn-42807411 is not present in Agent 10.0.0.27 2025-09-18 03:04:02,662 - INFO - Validated that VN ctest-trans_right_vn-42807411 is not in any agent 2025-09-18 03:04:02,728 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-trans_right_vn-42807411 info 2025-09-18 03:04:02,728 - INFO - Deleting VN ctest-trans_left_vn-61271344 2025-09-18 03:04:02,895 - DEBUG - Response for deleting network () 2025-09-18 03:04:02,895 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/3266be91-1009-4384-af0a-d7696014fb48 2025-09-18 03:04:02,908 - DEBUG - Response Code: 404 2025-09-18 03:04:02,908 - DEBUG - Requesting: http://10.0.0.56:8082/domains 2025-09-18 03:04:02,916 - DEBUG - Requesting: http://10.0.0.56:8082/domain/245899c5-19ba-4930-be6a-859f116860e1 2025-09-18 03:04:02,927 - DEBUG - Requesting: http://10.0.0.56:8082/project/4f4e4097-dcce-4868-a9c2-29d74f1abd03 2025-09-18 03:04:03,027 - INFO - Validated that VN ctest-trans_left_vn-61271344 is not found in API Server 2025-09-18 03:04:03,047 - DEBUG - VN ctest-trans_left_vn-61271344 is not present in Agent 10.0.0.61 2025-09-18 03:04:03,059 - DEBUG - VN ctest-trans_left_vn-61271344 is not present in Agent 10.0.0.27 2025-09-18 03:04:03,059 - INFO - Validated that VN ctest-trans_left_vn-61271344 is not in any agent 2025-09-18 03:04:03,094 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-trans_left_vn-61271344 info 2025-09-18 03:04:03,094 - INFO - Deleting VN ctest-right_vn-29803132 2025-09-18 03:04:03,151 - DEBUG - VN 93d70779-f831-4133-a848-6c77306e09fa still in use: Unable to complete operation on network 93d70779-f831-4133-a848-6c77306e09fa. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-cf2f4a95-46b4-4591-87ec-af3bd68a195d'] 2025-09-18 03:04:03,151 - WARNING - Deleting VN ctest-right_vn-29803132 failed..Will retry 2025-09-18 03:04:05,340 - DEBUG - Response for deleting network () 2025-09-18 03:04:05,340 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/edce6ac8-4ce5-48ca-8207-fdd22fa72150 2025-09-18 03:04:05,358 - DEBUG - Response Code: 404 2025-09-18 03:04:05,358 - DEBUG - Requesting: http://10.0.0.56:8082/domains 2025-09-18 03:04:05,370 - DEBUG - Requesting: http://10.0.0.56:8082/domain/245899c5-19ba-4930-be6a-859f116860e1 2025-09-18 03:04:05,383 - DEBUG - Requesting: http://10.0.0.56:8082/project/4f4e4097-dcce-4868-a9c2-29d74f1abd03 2025-09-18 03:04:05,477 - INFO - Validated that VN ctest-right_vn-29803132 is not found in API Server 2025-09-18 03:04:05,491 - DEBUG - VN ctest-right_vn-29803132 is not present in Agent 10.0.0.61 2025-09-18 03:04:05,506 - DEBUG - VN ctest-right_vn-29803132 is not present in Agent 10.0.0.27 2025-09-18 03:04:05,506 - INFO - Validated that VN ctest-right_vn-29803132 is not in any agent 2025-09-18 03:04:05,539 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-right_vn-29803132 info 2025-09-18 03:04:05,539 - INFO - Deleting VN ctest-left_vn-19206875 2025-09-18 03:04:05,585 - DEBUG - VN 62fd1d84-7e43-4606-b5ac-649496d14ee0 still in use: Unable to complete operation on network 62fd1d84-7e43-4606-b5ac-649496d14ee0. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-cef4fb82-656a-4035-ae56-9ab3be47cb12'] 2025-09-18 03:04:05,585 - WARNING - Deleting VN ctest-left_vn-19206875 failed..Will retry 2025-09-18 03:04:07,629 - DEBUG - VN 62fd1d84-7e43-4606-b5ac-649496d14ee0 still in use: Unable to complete operation on network 62fd1d84-7e43-4606-b5ac-649496d14ee0. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-f86eae94-f0e1-476f-bc55-fd3dad20275e'] 2025-09-18 03:04:07,629 - WARNING - Deleting VN ctest-left_vn-19206875 failed..Will retry 2025-09-18 03:04:09,680 - DEBUG - VN 62fd1d84-7e43-4606-b5ac-649496d14ee0 still in use: Unable to complete operation on network 62fd1d84-7e43-4606-b5ac-649496d14ee0. There are one or more ports still in use on the network. Neutron server returns request_ids: ['req-bb37ac59-3071-4f86-bd55-316ff93075f4'] 2025-09-18 03:04:09,680 - WARNING - Deleting VN ctest-left_vn-19206875 failed..Will retry 2025-09-18 03:04:11,867 - DEBUG - Response for deleting network () 2025-09-18 03:04:11,867 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/3b7fc812-9e79-4183-ad38-39b4c999b2cb 2025-09-18 03:04:11,893 - DEBUG - Response Code: 404 2025-09-18 03:04:11,893 - DEBUG - Requesting: http://10.0.0.56:8082/domains 2025-09-18 03:04:11,909 - DEBUG - Requesting: http://10.0.0.56:8082/domain/245899c5-19ba-4930-be6a-859f116860e1 2025-09-18 03:04:11,925 - DEBUG - Requesting: http://10.0.0.56:8082/project/4f4e4097-dcce-4868-a9c2-29d74f1abd03 2025-09-18 03:04:12,036 - INFO - Validated that VN ctest-left_vn-19206875 is not found in API Server 2025-09-18 03:04:12,056 - DEBUG - VN ctest-left_vn-19206875 is not present in Agent 10.0.0.61 2025-09-18 03:04:12,066 - DEBUG - VN ctest-left_vn-19206875 is not present in Agent 10.0.0.27 2025-09-18 03:04:12,066 - INFO - Validated that VN ctest-left_vn-19206875 is not in any agent 2025-09-18 03:04:12,106 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-left_vn-19206875 info 2025-09-18 03:04:12,106 - INFO - Deleting VN ctest-mgmt_vn-93094349 2025-09-18 03:04:12,284 - DEBUG - Response for deleting network () 2025-09-18 03:04:12,284 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/27dc4a88-2eec-48d3-97e9-3711f4c010e9 2025-09-18 03:04:12,295 - DEBUG - Response Code: 404 2025-09-18 03:04:12,295 - DEBUG - Requesting: http://10.0.0.56:8082/domains 2025-09-18 03:04:12,305 - DEBUG - Requesting: http://10.0.0.56:8082/domain/245899c5-19ba-4930-be6a-859f116860e1 2025-09-18 03:04:12,320 - DEBUG - Requesting: http://10.0.0.56:8082/project/4f4e4097-dcce-4868-a9c2-29d74f1abd03 2025-09-18 03:04:12,419 - INFO - Validated that VN ctest-mgmt_vn-93094349 is not found in API Server 2025-09-18 03:04:12,432 - DEBUG - VN ctest-mgmt_vn-93094349 is not present in Agent 10.0.0.61 2025-09-18 03:04:12,443 - DEBUG - VN ctest-mgmt_vn-93094349 is not present in Agent 10.0.0.27 2025-09-18 03:04:12,443 - INFO - Validated that VN ctest-mgmt_vn-93094349 is not in any agent 2025-09-18 03:04:12,479 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-mgmt_vn-93094349 info 2025-09-18 03:04:13,779 - DEBUG - No XMPP flaps were noticed during the test 2025-09-18 03:04:13,779 - INFO - 2025-09-18 03:04:13,779 - INFO - END TEST : test_svc_trans_segment_right_hc_fail_svm : SKIPPED[0:00:23] 2025-09-18 03:04:13,779 - INFO - -------------------------------------------------------------------------------- 2025-09-18 03:04:14,603 - INFO - Deleted project: ctest-TestSvcHC-70360481, ID : 4f4e4097-dcce-4868-a9c2-29d74f1abd03