The above is a description of an error in a Python program. Here is
the original traceback:
Traceback (most recent call last):
File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper
result = function(self, *args, **kwargs)
File "/contrail-test/serial_scripts/analytics/test_analytics_resource.py", line 1375, in test_verify_session_record_table_inter_vn
self.setup_and_create_streams(self.res.vn1_vm1_fixture, self.res.vn2_vm2_fixture)
File "/contrail-test/serial_scripts/analytics/base.py", line 170, in setup_and_create_streams
self.stop_traffic(traffic_obj)
File "/contrail-test/common/base.py", line 798, in stop_traffic
assert sent and recv, "Packets not %s"%msg
AssertionError: Packets not transferred between 214.167.83.195 and 141.70.171.3, proto udp sport 8000 and dport 9000
2025-08-04 03:58:30,946 - DEBUG - Skipping xmpp flap check
2025-08-04 03:58:30,946 - INFO -
2025-08-04 03:58:30,947 - INFO - END TEST : test_verify_session_record_table_inter_vn : FAILED[0:00:49]
2025-08-04 03:58:30,947 - INFO - --------------------------------------------------------------------------------
2025-08-04 03:58:30,956 - INFO - ================================================================================
2025-08-04 03:58:30,956 - INFO - STARTING TEST : test_verify_session_sampling_teardown
2025-08-04 03:58:30,956 - INFO - TEST DESCRIPTION :
1.query client session samples
2.query server session samples
3.query client session to get number of sessions exported
4.query session record table for teardown bytes/pkts
5.query sample count after teardown on server side
2025-08-04 03:58:32,244 - DEBUG - Skipping xmpp flap check
2025-08-04 03:58:32,245 - INFO - Initial checks done. Running the testcase now
2025-08-04 03:58:32,245 - INFO -
2025-08-04 03:58:32,464 - INFO - Setting flow export rate: 100
2025-08-04 03:58:32,512 - DEBUG - Policy np_rules : [rule_sequence = None, rule_uuid = None, direction = <>, protocol = icmp, src_addresses = [subnet = None, virtual_network = any, security_group = None, network_policy = None, subnet_list = []], src_ports = [start_port = -1, end_port = -1], application = [], dst_addresses = [subnet = None, virtual_network = any, security_group = None, network_policy = None, subnet_list = []], dst_ports = [start_port = -1, end_port = -1], action_list = simple_action = pass, gateway_name = None, apply_service = [], service_properties = None, mirror_to = None, assign_routing_instance = None, log = False, alert = False, qos_action = None, host_based_service = False, ethertype = None, created = None, last_modified = None, description = None]
2025-08-04 03:58:32,891 - DEBUG - Response for mapping policy(s) with vn {'network': {'id': 'f4f2b026-bd63-4f2f-b7d1-2cc78ff80148', 'name': 'ctest-vn1-08779962', 'tenant_id': 'a48ab5304ea04d359e730dcdaaab3e63', 'project_id': 'a48ab5304ea04d359e730dcdaaab3e63', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': ['62dad8db-c361-4be0-ba79-889d78e2cc4f'], 'fq_name': ['default-domain', 'ctest-AnalyticsTestSanityWithResource-33294378', 'ctest-vn1-08779962'], 'policys': [['default-domain', 'ctest-AnalyticsTestSanityWithResource-33294378', 'policy1']], 'subnet_ipam': [{'subnet_cidr': '214.167.83.192/26', 'ipam_fq_name': ['default-domain', 'default-project', 'default-network-ipam']}], 'port_security_enabled': True, 'description': ''}}
2025-08-04 03:58:33,189 - DEBUG - Response for mapping policy(s) with vn {'network': {'id': '2b6cfeea-fcc2-463e-b59a-9ff2a1d5442d', 'name': 'ctest-vn2-60124453', 'tenant_id': 'a48ab5304ea04d359e730dcdaaab3e63', 'project_id': 'a48ab5304ea04d359e730dcdaaab3e63', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': ['9b4e8e41-f3bf-4647-9138-eee7df38405e'], 'fq_name': ['default-domain', 'ctest-AnalyticsTestSanityWithResource-33294378', 'ctest-vn2-60124453'], 'policys': [['default-domain', 'ctest-AnalyticsTestSanityWithResource-33294378', 'policy1']], 'subnet_ipam': [{'subnet_cidr': '141.70.171.0/26', 'ipam_fq_name': ['default-domain', 'default-project', 'default-network-ipam']}], 'port_security_enabled': True, 'description': ''}}
2025-08-04 03:58:33,348 - DEBUG - Requesting: http://10.0.0.27:8082/domains
2025-08-04 03:58:33,356 - DEBUG - Requesting: http://10.0.0.27:8082/domain/8d00e278-1e00-4abc-a644-cc23d080d5d1
2025-08-04 03:58:33,370 - DEBUG - Requesting: http://10.0.0.27:8082/project/a48ab530-4ea0-4d35-9e73-0dcdaaab3e63
2025-08-04 03:58:33,471 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/f4f2b026-bd63-4f2f-b7d1-2cc78ff80148
2025-08-04 03:58:33,499 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/f4f2b026-bd63-4f2f-b7d1-2cc78ff80148
2025-08-04 03:58:33,519 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/e99be9e0-5910-4043-aa77-dbeaa2b7ac52
2025-08-04 03:58:33,534 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/e99be9e0-5910-4043-aa77-dbeaa2b7ac52
2025-08-04 03:58:33,548 - DEBUG - Requesting: http://10.0.0.27:8082/route-target/ef14356b-eec4-430c-a633-8e3efe436700
2025-08-04 03:58:33,559 - DEBUG - Route Targets: ['target:64512:8000005']
2025-08-04 03:58:33,559 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/f4f2b026-bd63-4f2f-b7d1-2cc78ff80148
2025-08-04 03:58:33,572 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/e99be9e0-5910-4043-aa77-dbeaa2b7ac52
2025-08-04 03:58:33,597 - INFO - Verified VN network id 13 for VN f4f2b026-bd63-4f2f-b7d1-2cc78ff80148
2025-08-04 03:58:33,598 - INFO - Verifications in API Server for VN ctest-vn1-08779962 passed
2025-08-04 03:58:33,598 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/f4f2b026-bd63-4f2f-b7d1-2cc78ff80148
2025-08-04 03:58:33,616 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/e99be9e0-5910-4043-aa77-dbeaa2b7ac52
2025-08-04 03:58:33,630 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/e99be9e0-5910-4043-aa77-dbeaa2b7ac52
2025-08-04 03:58:33,645 - DEBUG - Requesting: http://10.0.0.27:8082/route-target/ef14356b-eec4-430c-a633-8e3efe436700
2025-08-04 03:58:33,665 - DEBUG - Control-node 10.0.0.27 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962', 'interests': '0', 'advertised': '0', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-33294378', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '17650363570051043119', 'uuid-lslong': '13245417214698848584'}, 'enable': 'true', 'created': '2025-08-04T03:56:17', 'last-modified': '2025-08-04T03:58:32', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.825900'}
2025-08-04 03:58:33,675 - DEBUG - Route Targets: ['target:64512:8000005']
2025-08-04 03:58:33,683 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-33294378', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '17650363570051043119', 'uuid-lslong': '13245417214698848584'}, 'enable': 'true', 'created': '2025-08-04T03:56:17', 'last-modified': '2025-08-04T03:58:32', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.844380'}
2025-08-04 03:58:33,693 - DEBUG - Route Targets: ['target:64512:8000005']
2025-08-04 03:58:33,702 - DEBUG - Control-node 10.0.0.61 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-33294378', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '17650363570051043119', 'uuid-lslong': '13245417214698848584'}, 'enable': 'true', 'created': '2025-08-04T03:56:17', 'last-modified': '2025-08-04T03:58:32', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.864094'}
2025-08-04 03:58:33,713 - DEBUG - Route Targets: ['target:64512:8000005']
2025-08-04 03:58:33,713 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn1-08779962 passed
2025-08-04 03:58:33,713 - DEBUG - ====Verifying policy data for ctest-vn1-08779962 in API_Server ======
2025-08-04 03:58:33,713 - DEBUG - Requesting: http://10.0.0.27:8082/domains
2025-08-04 03:58:33,722 - DEBUG - Requesting: http://10.0.0.27:8082/domain/8d00e278-1e00-4abc-a644-cc23d080d5d1
2025-08-04 03:58:33,745 - DEBUG - Requesting: http://10.0.0.27:8082/project/a48ab530-4ea0-4d35-9e73-0dcdaaab3e63
2025-08-04 03:58:33,852 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/f4f2b026-bd63-4f2f-b7d1-2cc78ff80148
2025-08-04 03:58:33,871 - DEBUG - ==>Verifying data for policy with id: 72d67ac5-2721-4bbe-9543-d92285436821, fqn: ['default-domain', 'ctest-AnalyticsTestSanityWithResource-33294378', 'policy1']
2025-08-04 03:58:33,871 - INFO - VN ctest-vn1-08779962 Policy verification: verify_vn_policy_in_api_server, status: True
2025-08-04 03:58:33,871 - DEBUG - Verifying the vn in opserver
2025-08-04 03:58:33,871 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962 virtual network link through opserver 10.0.0.27
2025-08-04 03:58:33,871 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/virtual-networks
2025-08-04 03:58:33,892 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962', 'href': 'http://10.0.0.27:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962?flat'}
2025-08-04 03:58:33,892 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962 virtual network link through opserver 10.0.0.38
2025-08-04 03:58:33,892 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks
2025-08-04 03:58:33,908 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962?flat'}
2025-08-04 03:58:33,908 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962 virtual network link through opserver 10.0.0.61
2025-08-04 03:58:33,908 - DEBUG - Requesting: http://10.0.0.61:8081/analytics/uves/virtual-networks
2025-08-04 03:58:33,927 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962', 'href': 'http://10.0.0.61:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962?flat'}
2025-08-04 03:58:33,927 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962 is found in opserver
2025-08-04 03:58:33,941 - DEBUG - VRF ids for VN ctest-vn1-08779962: {'10.0.0.57': '2'}
2025-08-04 03:58:33,942 - DEBUG - Requesting: http://10.0.0.27:8082/domains
2025-08-04 03:58:33,949 - DEBUG - Requesting: http://10.0.0.27:8082/domain/8d00e278-1e00-4abc-a644-cc23d080d5d1
2025-08-04 03:58:33,982 - DEBUG - Requesting: http://10.0.0.27:8082/project/a48ab530-4ea0-4d35-9e73-0dcdaaab3e63
2025-08-04 03:58:34,091 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/2b6cfeea-fcc2-463e-b59a-9ff2a1d5442d
2025-08-04 03:58:34,112 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/2b6cfeea-fcc2-463e-b59a-9ff2a1d5442d
2025-08-04 03:58:34,127 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/30e0c87d-9965-4af0-9284-aaf0f0fe02c3
2025-08-04 03:58:34,144 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/30e0c87d-9965-4af0-9284-aaf0f0fe02c3
2025-08-04 03:58:34,154 - DEBUG - Requesting: http://10.0.0.27:8082/route-target/5c49b08b-a708-46d0-a73c-8dea330b70de
2025-08-04 03:58:34,167 - DEBUG - Route Targets: ['target:64512:8000006']
2025-08-04 03:58:34,167 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/2b6cfeea-fcc2-463e-b59a-9ff2a1d5442d
2025-08-04 03:58:34,182 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/30e0c87d-9965-4af0-9284-aaf0f0fe02c3
2025-08-04 03:58:34,209 - INFO - Verified VN network id 14 for VN 2b6cfeea-fcc2-463e-b59a-9ff2a1d5442d
2025-08-04 03:58:34,210 - INFO - Verifications in API Server for VN ctest-vn2-60124453 passed
2025-08-04 03:58:34,210 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/2b6cfeea-fcc2-463e-b59a-9ff2a1d5442d
2025-08-04 03:58:34,228 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/30e0c87d-9965-4af0-9284-aaf0f0fe02c3
2025-08-04 03:58:34,240 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/30e0c87d-9965-4af0-9284-aaf0f0fe02c3
2025-08-04 03:58:34,252 - DEBUG - Requesting: http://10.0.0.27:8082/route-target/5c49b08b-a708-46d0-a73c-8dea330b70de
2025-08-04 03:58:34,274 - DEBUG - Control-node 10.0.0.27 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453', 'interests': '0', 'advertised': '0', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-33294378', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '3129156126332044862', 'uuid-lslong': '13085947531724407853'}, 'enable': 'true', 'created': '2025-08-04T03:56:53', 'last-modified': '2025-08-04T03:58:33', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:01.153003'}
2025-08-04 03:58:34,286 - DEBUG - Route Targets: ['target:64512:8000006']
2025-08-04 03:58:34,294 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-33294378', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '3129156126332044862', 'uuid-lslong': '13085947531724407853'}, 'enable': 'true', 'created': '2025-08-04T03:56:53', 'last-modified': '2025-08-04T03:58:33', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:01.175769'}
2025-08-04 03:58:34,305 - DEBUG - Route Targets: ['target:64512:8000006']
2025-08-04 03:58:34,313 - DEBUG - Control-node 10.0.0.61 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453', 'interests': '0', 'advertised': '0', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-33294378', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '3129156126332044862', 'uuid-lslong': '13085947531724407853'}, 'enable': 'true', 'created': '2025-08-04T03:56:53', 'last-modified': '2025-08-04T03:58:33', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:01.192660'}
2025-08-04 03:58:34,325 - DEBUG - Route Targets: ['target:64512:8000006']
2025-08-04 03:58:34,326 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn2-60124453 passed
2025-08-04 03:58:34,326 - DEBUG - ====Verifying policy data for ctest-vn2-60124453 in API_Server ======
2025-08-04 03:58:34,326 - DEBUG - Requesting: http://10.0.0.27:8082/domains
2025-08-04 03:58:34,334 - DEBUG - Requesting: http://10.0.0.27:8082/domain/8d00e278-1e00-4abc-a644-cc23d080d5d1
2025-08-04 03:58:34,351 - DEBUG - Requesting: http://10.0.0.27:8082/project/a48ab530-4ea0-4d35-9e73-0dcdaaab3e63
2025-08-04 03:58:34,462 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/2b6cfeea-fcc2-463e-b59a-9ff2a1d5442d
2025-08-04 03:58:34,477 - DEBUG - ==>Verifying data for policy with id: 72d67ac5-2721-4bbe-9543-d92285436821, fqn: ['default-domain', 'ctest-AnalyticsTestSanityWithResource-33294378', 'policy1']
2025-08-04 03:58:34,477 - INFO - VN ctest-vn2-60124453 Policy verification: verify_vn_policy_in_api_server, status: True
2025-08-04 03:58:34,478 - DEBUG - Verifying the vn in opserver
2025-08-04 03:58:34,478 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453 virtual network link through opserver 10.0.0.27
2025-08-04 03:58:34,478 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/virtual-networks
2025-08-04 03:58:34,494 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453', 'href': 'http://10.0.0.27:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453?flat'}
2025-08-04 03:58:34,494 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453 virtual network link through opserver 10.0.0.38
2025-08-04 03:58:34,494 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks
2025-08-04 03:58:34,511 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453?flat'}
2025-08-04 03:58:34,511 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453 virtual network link through opserver 10.0.0.61
2025-08-04 03:58:34,511 - DEBUG - Requesting: http://10.0.0.61:8081/analytics/uves/virtual-networks
2025-08-04 03:58:34,535 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453', 'href': 'http://10.0.0.61:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453?flat'}
2025-08-04 03:58:34,535 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453 is found in opserver
2025-08-04 03:58:34,551 - DEBUG - VRF ids for VN ctest-vn2-60124453: {'10.0.0.41': '2', '10.0.0.57': '3'}
2025-08-04 03:58:34,551 - DEBUG - Requesting: http://10.0.0.27:8082/domains
2025-08-04 03:58:34,560 - DEBUG - Requesting: http://10.0.0.27:8082/domain/8d00e278-1e00-4abc-a644-cc23d080d5d1
2025-08-04 03:58:34,579 - DEBUG - Requesting: http://10.0.0.27:8082/project/a48ab530-4ea0-4d35-9e73-0dcdaaab3e63
2025-08-04 03:58:34,682 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/f4f2b026-bd63-4f2f-b7d1-2cc78ff80148
2025-08-04 03:58:34,709 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/f4f2b026-bd63-4f2f-b7d1-2cc78ff80148
2025-08-04 03:58:34,727 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/e99be9e0-5910-4043-aa77-dbeaa2b7ac52
2025-08-04 03:58:34,738 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/e99be9e0-5910-4043-aa77-dbeaa2b7ac52
2025-08-04 03:58:34,748 - DEBUG - Requesting: http://10.0.0.27:8082/route-target/ef14356b-eec4-430c-a633-8e3efe436700
2025-08-04 03:58:34,761 - DEBUG - Route Targets: ['target:64512:8000005']
2025-08-04 03:58:34,761 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/f4f2b026-bd63-4f2f-b7d1-2cc78ff80148
2025-08-04 03:58:34,776 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/e99be9e0-5910-4043-aa77-dbeaa2b7ac52
2025-08-04 03:58:34,801 - INFO - Verified VN network id 13 for VN f4f2b026-bd63-4f2f-b7d1-2cc78ff80148
2025-08-04 03:58:34,801 - INFO - Verifications in API Server for VN ctest-vn1-08779962 passed
2025-08-04 03:58:34,802 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/f4f2b026-bd63-4f2f-b7d1-2cc78ff80148
2025-08-04 03:58:34,816 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/e99be9e0-5910-4043-aa77-dbeaa2b7ac52
2025-08-04 03:58:34,828 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/e99be9e0-5910-4043-aa77-dbeaa2b7ac52
2025-08-04 03:58:34,843 - DEBUG - Requesting: http://10.0.0.27:8082/route-target/ef14356b-eec4-430c-a633-8e3efe436700
2025-08-04 03:58:34,873 - DEBUG - Control-node 10.0.0.27 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962', 'interests': '0', 'advertised': '0', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-33294378', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '17650363570051043119', 'uuid-lslong': '13245417214698848584'}, 'enable': 'true', 'created': '2025-08-04T03:56:17', 'last-modified': '2025-08-04T03:58:32', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:02.028725'}
2025-08-04 03:58:34,886 - DEBUG - Route Targets: ['target:64512:8000005']
2025-08-04 03:58:34,895 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-33294378', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '17650363570051043119', 'uuid-lslong': '13245417214698848584'}, 'enable': 'true', 'created': '2025-08-04T03:56:17', 'last-modified': '2025-08-04T03:58:32', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:02.055930'}
2025-08-04 03:58:34,903 - DEBUG - Route Targets: ['target:64512:8000005']
2025-08-04 03:58:34,918 - DEBUG - Control-node 10.0.0.61 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-33294378', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '17650363570051043119', 'uuid-lslong': '13245417214698848584'}, 'enable': 'true', 'created': '2025-08-04T03:56:17', 'last-modified': '2025-08-04T03:58:32', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:02.079984'}
2025-08-04 03:58:34,930 - DEBUG - Route Targets: ['target:64512:8000005']
2025-08-04 03:58:34,930 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn1-08779962 passed
2025-08-04 03:58:34,930 - DEBUG - ====Verifying policy data for ctest-vn1-08779962 in API_Server ======
2025-08-04 03:58:34,930 - DEBUG - Requesting: http://10.0.0.27:8082/domains
2025-08-04 03:58:34,945 - DEBUG - Requesting: http://10.0.0.27:8082/domain/8d00e278-1e00-4abc-a644-cc23d080d5d1
2025-08-04 03:58:34,963 - DEBUG - Requesting: http://10.0.0.27:8082/project/a48ab530-4ea0-4d35-9e73-0dcdaaab3e63
2025-08-04 03:58:35,073 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/f4f2b026-bd63-4f2f-b7d1-2cc78ff80148
2025-08-04 03:58:35,087 - DEBUG - ==>Verifying data for policy with id: 72d67ac5-2721-4bbe-9543-d92285436821, fqn: ['default-domain', 'ctest-AnalyticsTestSanityWithResource-33294378', 'policy1']
2025-08-04 03:58:35,088 - INFO - VN ctest-vn1-08779962 Policy verification: verify_vn_policy_in_api_server, status: True
2025-08-04 03:58:35,088 - DEBUG - Verifying the vn in opserver
2025-08-04 03:58:35,088 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962 virtual network link through opserver 10.0.0.27
2025-08-04 03:58:35,088 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/virtual-networks
2025-08-04 03:58:35,106 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962', 'href': 'http://10.0.0.27:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962?flat'}
2025-08-04 03:58:35,106 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962 virtual network link through opserver 10.0.0.38
2025-08-04 03:58:35,106 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks
2025-08-04 03:58:35,121 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962?flat'}
2025-08-04 03:58:35,121 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962 virtual network link through opserver 10.0.0.61
2025-08-04 03:58:35,121 - DEBUG - Requesting: http://10.0.0.61:8081/analytics/uves/virtual-networks
2025-08-04 03:58:35,137 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962', 'href': 'http://10.0.0.61:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962?flat'}
2025-08-04 03:58:35,137 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962 is found in opserver
2025-08-04 03:58:35,155 - DEBUG - VRF ids for VN ctest-vn1-08779962: {'10.0.0.57': '2'}
2025-08-04 03:58:35,155 - DEBUG - Requesting: http://10.0.0.27:8082/domains
2025-08-04 03:58:35,163 - DEBUG - Requesting: http://10.0.0.27:8082/domain/8d00e278-1e00-4abc-a644-cc23d080d5d1
2025-08-04 03:58:35,183 - DEBUG - Requesting: http://10.0.0.27:8082/project/a48ab530-4ea0-4d35-9e73-0dcdaaab3e63
2025-08-04 03:58:35,288 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/2b6cfeea-fcc2-463e-b59a-9ff2a1d5442d
2025-08-04 03:58:35,304 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/2b6cfeea-fcc2-463e-b59a-9ff2a1d5442d
2025-08-04 03:58:35,318 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/30e0c87d-9965-4af0-9284-aaf0f0fe02c3
2025-08-04 03:58:35,329 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/30e0c87d-9965-4af0-9284-aaf0f0fe02c3
2025-08-04 03:58:35,340 - DEBUG - Requesting: http://10.0.0.27:8082/route-target/5c49b08b-a708-46d0-a73c-8dea330b70de
2025-08-04 03:58:35,350 - DEBUG - Route Targets: ['target:64512:8000006']
2025-08-04 03:58:35,350 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/2b6cfeea-fcc2-463e-b59a-9ff2a1d5442d
2025-08-04 03:58:35,366 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/30e0c87d-9965-4af0-9284-aaf0f0fe02c3
2025-08-04 03:58:35,386 - INFO - Verified VN network id 14 for VN 2b6cfeea-fcc2-463e-b59a-9ff2a1d5442d
2025-08-04 03:58:35,386 - INFO - Verifications in API Server for VN ctest-vn2-60124453 passed
2025-08-04 03:58:35,386 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/2b6cfeea-fcc2-463e-b59a-9ff2a1d5442d
2025-08-04 03:58:35,404 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/30e0c87d-9965-4af0-9284-aaf0f0fe02c3
2025-08-04 03:58:35,415 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/30e0c87d-9965-4af0-9284-aaf0f0fe02c3
2025-08-04 03:58:35,426 - DEBUG - Requesting: http://10.0.0.27:8082/route-target/5c49b08b-a708-46d0-a73c-8dea330b70de
2025-08-04 03:58:35,444 - DEBUG - Control-node 10.0.0.27 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453', 'interests': '0', 'advertised': '0', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-33294378', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '3129156126332044862', 'uuid-lslong': '13085947531724407853'}, 'enable': 'true', 'created': '2025-08-04T03:56:53', 'last-modified': '2025-08-04T03:58:33', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:02.325116'}
2025-08-04 03:58:35,455 - DEBUG - Route Targets: ['target:64512:8000006']
2025-08-04 03:58:35,462 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-33294378', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '3129156126332044862', 'uuid-lslong': '13085947531724407853'}, 'enable': 'true', 'created': '2025-08-04T03:56:53', 'last-modified': '2025-08-04T03:58:33', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:02.344779'}
2025-08-04 03:58:35,470 - DEBUG - Route Targets: ['target:64512:8000006']
2025-08-04 03:58:35,478 - DEBUG - Control-node 10.0.0.61 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453', 'interests': '0', 'advertised': '0', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-33294378', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '3129156126332044862', 'uuid-lslong': '13085947531724407853'}, 'enable': 'true', 'created': '2025-08-04T03:56:53', 'last-modified': '2025-08-04T03:58:33', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:02.357245'}
2025-08-04 03:58:35,489 - DEBUG - Route Targets: ['target:64512:8000006']
2025-08-04 03:58:35,489 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn2-60124453 passed
2025-08-04 03:58:35,490 - DEBUG - ====Verifying policy data for ctest-vn2-60124453 in API_Server ======
2025-08-04 03:58:35,490 - DEBUG - Requesting: http://10.0.0.27:8082/domains
2025-08-04 03:58:35,498 - DEBUG - Requesting: http://10.0.0.27:8082/domain/8d00e278-1e00-4abc-a644-cc23d080d5d1
2025-08-04 03:58:35,515 - DEBUG - Requesting: http://10.0.0.27:8082/project/a48ab530-4ea0-4d35-9e73-0dcdaaab3e63
2025-08-04 03:58:35,635 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/2b6cfeea-fcc2-463e-b59a-9ff2a1d5442d
2025-08-04 03:58:35,651 - DEBUG - ==>Verifying data for policy with id: 72d67ac5-2721-4bbe-9543-d92285436821, fqn: ['default-domain', 'ctest-AnalyticsTestSanityWithResource-33294378', 'policy1']
2025-08-04 03:58:35,651 - INFO - VN ctest-vn2-60124453 Policy verification: verify_vn_policy_in_api_server, status: True
2025-08-04 03:58:35,651 - DEBUG - Verifying the vn in opserver
2025-08-04 03:58:35,652 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453 virtual network link through opserver 10.0.0.27
2025-08-04 03:58:35,652 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/virtual-networks
2025-08-04 03:58:35,668 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453', 'href': 'http://10.0.0.27:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453?flat'}
2025-08-04 03:58:35,668 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453 virtual network link through opserver 10.0.0.38
2025-08-04 03:58:35,668 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks
2025-08-04 03:58:35,684 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453?flat'}
2025-08-04 03:58:35,685 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453 virtual network link through opserver 10.0.0.61
2025-08-04 03:58:35,685 - DEBUG - Requesting: http://10.0.0.61:8081/analytics/uves/virtual-networks
2025-08-04 03:58:35,703 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453', 'href': 'http://10.0.0.61:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453?flat'}
2025-08-04 03:58:35,703 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453 is found in opserver
2025-08-04 03:58:35,721 - DEBUG - VRF ids for VN ctest-vn2-60124453: {'10.0.0.41': '2', '10.0.0.57': '3'}
2025-08-04 03:58:35,722 - DEBUG - Requesting: http://10.0.0.27:8082/domains
2025-08-04 03:58:35,732 - DEBUG - Requesting: http://10.0.0.27:8082/domain/8d00e278-1e00-4abc-a644-cc23d080d5d1
2025-08-04 03:58:35,753 - DEBUG - Requesting: http://10.0.0.27:8082/project/a48ab530-4ea0-4d35-9e73-0dcdaaab3e63
2025-08-04 03:58:35,880 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/803b3dbe-c1bd-4fc2-9c92-5759bc307503
2025-08-04 03:58:35,898 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/803b3dbe-c1bd-4fc2-9c92-5759bc307503
2025-08-04 03:58:35,924 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/e2f98916-810f-46ff-b3a0-8d9cb6c4d1d1
2025-08-04 03:58:35,934 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/e2f98916-810f-46ff-b3a0-8d9cb6c4d1d1
2025-08-04 03:58:35,945 - DEBUG - Requesting: http://10.0.0.27:8082/route-target/3d83ade0-6dae-4396-ad20-78a064cc6683
2025-08-04 03:58:35,963 - DEBUG - Route Targets: ['target:64512:8000007']
2025-08-04 03:58:35,963 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/803b3dbe-c1bd-4fc2-9c92-5759bc307503
2025-08-04 03:58:35,979 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/e2f98916-810f-46ff-b3a0-8d9cb6c4d1d1
2025-08-04 03:58:36,001 - INFO - Verified VN network id 15 for VN 803b3dbe-c1bd-4fc2-9c92-5759bc307503
2025-08-04 03:58:36,002 - INFO - Verifications in API Server for VN ctest-fip_vn-14850913 passed
2025-08-04 03:58:36,002 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/803b3dbe-c1bd-4fc2-9c92-5759bc307503
2025-08-04 03:58:36,019 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/e2f98916-810f-46ff-b3a0-8d9cb6c4d1d1
2025-08-04 03:58:36,029 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/e2f98916-810f-46ff-b3a0-8d9cb6c4d1d1
2025-08-04 03:58:36,040 - DEBUG - Requesting: http://10.0.0.27:8082/route-target/3d83ade0-6dae-4396-ad20-78a064cc6683
2025-08-04 03:58:36,066 - DEBUG - Control-node 10.0.0.27 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-fip_vn-14850913', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-33294378', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '9240046949984194498', 'uuid-lslong': '11282176059437511939'}, 'enable': 'true', 'created': '2025-08-04T03:56:54', 'last-modified': '2025-08-04T03:56:54', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:01:41.156577'}
2025-08-04 03:58:36,077 - DEBUG - Route Targets: ['target:64512:8000007']
2025-08-04 03:58:36,087 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-fip_vn-14850913', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-33294378', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '9240046949984194498', 'uuid-lslong': '11282176059437511939'}, 'enable': 'true', 'created': '2025-08-04T03:56:54', 'last-modified': '2025-08-04T03:56:54', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:01:41.188381'}
2025-08-04 03:58:36,098 - DEBUG - Route Targets: ['target:64512:8000007']
2025-08-04 03:58:36,108 - DEBUG - Control-node 10.0.0.61 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-fip_vn-14850913', 'interests': '0', 'advertised': '0', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-33294378', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '9240046949984194498', 'uuid-lslong': '11282176059437511939'}, 'enable': 'true', 'created': '2025-08-04T03:56:54', 'last-modified': '2025-08-04T03:56:54', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:01:41.205624'}
2025-08-04 03:58:36,124 - DEBUG - Route Targets: ['target:64512:8000007']
2025-08-04 03:58:36,124 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-fip_vn-14850913 passed
2025-08-04 03:58:36,124 - DEBUG - ====Verifying policy data for ctest-fip_vn-14850913 in API_Server ======
2025-08-04 03:58:36,124 - DEBUG - Requesting: http://10.0.0.27:8082/domains
2025-08-04 03:58:36,134 - DEBUG - Requesting: http://10.0.0.27:8082/domain/8d00e278-1e00-4abc-a644-cc23d080d5d1
2025-08-04 03:58:36,152 - DEBUG - Requesting: http://10.0.0.27:8082/project/a48ab530-4ea0-4d35-9e73-0dcdaaab3e63
2025-08-04 03:58:36,273 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/803b3dbe-c1bd-4fc2-9c92-5759bc307503
2025-08-04 03:58:36,288 - DEBUG - =>VN ctest-fip_vn-14850913 has no policy to be verified
2025-08-04 03:58:36,288 - DEBUG - Verifying the vn in opserver
2025-08-04 03:58:36,288 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-fip_vn-14850913 virtual network link through opserver 10.0.0.27
2025-08-04 03:58:36,288 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/virtual-networks
2025-08-04 03:58:36,306 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-fip_vn-14850913', 'href': 'http://10.0.0.27:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-fip_vn-14850913?flat'}
2025-08-04 03:58:36,306 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-fip_vn-14850913 virtual network link through opserver 10.0.0.38
2025-08-04 03:58:36,306 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks
2025-08-04 03:58:36,324 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-fip_vn-14850913', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-fip_vn-14850913?flat'}
2025-08-04 03:58:36,324 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-fip_vn-14850913 virtual network link through opserver 10.0.0.61
2025-08-04 03:58:36,324 - DEBUG - Requesting: http://10.0.0.61:8081/analytics/uves/virtual-networks
2025-08-04 03:58:36,351 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-fip_vn-14850913', 'href': 'http://10.0.0.61:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-fip_vn-14850913?flat'}
2025-08-04 03:58:36,351 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-fip_vn-14850913 is found in opserver
2025-08-04 03:58:36,369 - DEBUG - VRF ids for VN ctest-fip_vn-14850913: {'10.0.0.41': '3'}
2025-08-04 03:58:36,369 - INFO - start time= 1754279916369510
2025-08-04 03:58:36,369 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 3 -W 1 141.70.171.3, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.57, gateway password: c0ntrail123
2025-08-04 03:58:36,370 - DEBUG - ping -s 56 -c 3 -W 1 141.70.171.3
2025-08-04 03:58:38,615 - DEBUG - PING 141.70.171.3 (141.70.171.3): 56 data bytes
64 bytes from 141.70.171.3: seq=0 ttl=63 time=4.495 ms
64 bytes from 141.70.171.3: seq=1 ttl=63 time=1.235 ms
64 bytes from 141.70.171.3: seq=2 ttl=63 time=1.365 ms
--- 141.70.171.3 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 1.235/2.365/4.495 ms
2025-08-04 03:58:38,615 - INFO - Ping to IP 141.70.171.3 from VM ctest-vn1_vm1-33302525 passed
2025-08-04 03:58:48,616 - INFO - Verify session samples and teardown pkts
2025-08-04 03:58:48,616 - INFO - Verifying SessionSeriesTable through opserver 10.0.0.27
2025-08-04 03:58:48,620 - DEBUG - Posting: http://10.0.0.27:8081/analytics/query, payload {'table': 'SessionSeriesTable', 'start_time': 1754279916369510, 'end_time': 1754279928620834, 'select_fields': ['T'], 'where': [[{'name': 'vn', 'value': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962', 'op': 1, 'value2': None}, {'name': 'remote_vn', 'value': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453', 'op': 1, 'value2': None}, {'name': 'protocol', 'value': '1', 'op': 1, 'value2': None}]], 'session_type': 'client'}
2025-08-04 03:58:49,642 - DEBUG - [{'T': 1754279916909067}, {'T': 1754279917909171}, {'T': 1754279918909218}]
2025-08-04 03:58:49,643 - DEBUG - Posting: http://10.0.0.27:8081/analytics/query, payload {'table': 'SessionSeriesTable', 'start_time': 1754279916369510, 'end_time': 1754279929643008, 'select_fields': ['T'], 'where': [[{'name': 'vn', 'value': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453', 'op': 1, 'value2': None}, {'name': 'remote_vn', 'value': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962', 'op': 1, 'value2': None}, {'name': 'protocol', 'value': '1', 'op': 1, 'value2': None}]], 'session_type': 'server'}
2025-08-04 03:58:50,664 - DEBUG - [{'T': 1754279917533069}, {'T': 1754279918533115}, {'T': 1754279919533170}]
2025-08-04 03:58:50,664 - DEBUG - Posting: http://10.0.0.27:8081/analytics/query, payload {'table': 'SessionSeriesTable', 'start_time': 1754279916369510, 'end_time': 1754279930664704, 'select_fields': ['vn', 'remote_vn', 'sample_count'], 'where': [[{'name': 'vn', 'value': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962', 'op': 1, 'value2': None}, {'name': 'remote_vn', 'value': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453', 'op': 1, 'value2': None}, {'name': 'protocol', 'value': '1', 'op': 1, 'value2': None}]], 'session_type': 'client'}
2025-08-04 03:58:51,688 - DEBUG - [{'remote_vn': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453', 'vn': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962', 'sample_count': 3}]
2025-08-04 03:58:51,688 - INFO - wait for the flows to get expire
2025-08-04 04:02:11,689 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-3750-2:Compute:contrail-vrouter-agent:0?flat
2025-08-04 04:02:11,799 - DEBUG - Requesting: http://10.0.0.61:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-3750-2?flat
2025-08-04 04:02:11,896 - DEBUG - Posting: http://10.0.0.27:8081/analytics/query, payload {'table': 'SessionRecordTable', 'start_time': 1754279916369510, 'end_time': 1754280131896595, 'select_fields': ['vn', 'remote_vn', 'forward_teardown_pkts', 'reverse_teardown_pkts'], 'where': [[{'name': 'vn', 'value': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962', 'op': 1, 'value2': None}, {'name': 'remote_vn', 'value': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453', 'op': 1, 'value2': None}, {'name': 'protocol', 'value': '1', 'op': 1, 'value2': None}]], 'session_type': 'client'}
2025-08-04 04:02:12,971 - DEBUG - [{'forward_flow_uuid': '60fd0725-67eb-4fb2-8c01-69dc5896c994', 'remote_vn': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453', 'reverse_flow_uuid': '759a27fe-3856-41a4-8d56-9bd95daee444', 'vn': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962', 'forward_teardown_pkts': 3, 'reverse_teardown_pkts': 3}]
2025-08-04 04:02:12,971 - DEBUG - Posting: http://10.0.0.27:8081/analytics/query, payload {'table': 'SessionSeriesTable', 'start_time': 1754279916369510, 'end_time': 1754280132971532, 'select_fields': ['vn', 'remote_vn', 'sample_count'], 'where': [[{'name': 'vn', 'value': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962', 'op': 1, 'value2': None}, {'name': 'remote_vn', 'value': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453', 'op': 1, 'value2': None}, {'name': 'protocol', 'value': '1', 'op': 1, 'value2': None}]], 'session_type': 'client'}
2025-08-04 04:02:14,054 - DEBUG - Posting: http://10.0.0.27:8081/analytics/query, payload {'table': 'SessionSeriesTable', 'start_time': 1754279916369510, 'end_time': 1754280134054059, 'select_fields': ['vn', 'remote_vn', 'sample_count'], 'where': [[{'name': 'vn', 'value': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453', 'op': 1, 'value2': None}, {'name': 'remote_vn', 'value': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962', 'op': 1, 'value2': None}, {'name': 'protocol', 'value': '1', 'op': 1, 'value2': None}]], 'session_type': 'server'}
2025-08-04 04:02:15,086 - DEBUG - [{'remote_vn': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962', 'vn': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453', 'sample_count': 4}]
2025-08-04 04:02:15,702 - INFO - Deleted policy policy1
2025-08-04 04:02:15,860 - INFO - Setting flow export rate: None
2025-08-04 04:02:17,144 - DEBUG - Skipping xmpp flap check
2025-08-04 04:02:17,144 - INFO - END TEST : test_verify_session_sampling_teardown : PASSED[0:03:47]
2025-08-04 04:02:17,144 - INFO - --------------------------------------------------------------------------------
2025-08-04 04:02:17,148 - INFO - ================================================================================
2025-08-04 04:02:17,148 - INFO - STARTING TEST : test_verify_session_series_table_inter_vn
2025-08-04 04:02:17,148 - INFO - TEST DESCRIPTION : Verify session series table ,generated stats between different vns
1.query client session samples
2.query for server ports
3.sort results by server_port column
4.verify granularity with T=10
5.verify sampled bytes
6.verify logged bytes
7.query and verify filter by action
2025-08-04 04:02:18,442 - DEBUG - Skipping xmpp flap check
2025-08-04 04:02:18,442 - INFO - Initial checks done. Running the testcase now
2025-08-04 04:02:18,442 - INFO -
2025-08-04 04:02:18,660 - INFO - Setting flow export rate: 100
2025-08-04 04:02:18,708 - DEBUG - Policy np_rules : [rule_sequence = None, rule_uuid = None, direction = <>, protocol = udp, src_addresses = [subnet = None, virtual_network = any, security_group = None, network_policy = None, subnet_list = []], src_ports = [start_port = -1, end_port = -1], application = [], dst_addresses = [subnet = None, virtual_network = any, security_group = None, network_policy = None, subnet_list = []], dst_ports = [start_port = -1, end_port = -1], action_list = simple_action = pass, gateway_name = None, apply_service = [], service_properties = None, mirror_to = None, assign_routing_instance = None, log = False, alert = False, qos_action = None, host_based_service = False, ethertype = None, created = None, last_modified = None, description = None]
2025-08-04 04:02:19,115 - DEBUG - Response for mapping policy(s) with vn {'network': {'id': 'f4f2b026-bd63-4f2f-b7d1-2cc78ff80148', 'name': 'ctest-vn1-08779962', 'tenant_id': 'a48ab5304ea04d359e730dcdaaab3e63', 'project_id': 'a48ab5304ea04d359e730dcdaaab3e63', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': ['62dad8db-c361-4be0-ba79-889d78e2cc4f'], 'fq_name': ['default-domain', 'ctest-AnalyticsTestSanityWithResource-33294378', 'ctest-vn1-08779962'], 'policys': [['default-domain', 'ctest-AnalyticsTestSanityWithResource-33294378', 'policy1']], 'subnet_ipam': [{'subnet_cidr': '214.167.83.192/26', 'ipam_fq_name': ['default-domain', 'default-project', 'default-network-ipam']}], 'port_security_enabled': True, 'description': ''}}
2025-08-04 04:02:19,356 - DEBUG - Response for mapping policy(s) with vn {'network': {'id': '2b6cfeea-fcc2-463e-b59a-9ff2a1d5442d', 'name': 'ctest-vn2-60124453', 'tenant_id': 'a48ab5304ea04d359e730dcdaaab3e63', 'project_id': 'a48ab5304ea04d359e730dcdaaab3e63', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': ['9b4e8e41-f3bf-4647-9138-eee7df38405e'], 'fq_name': ['default-domain', 'ctest-AnalyticsTestSanityWithResource-33294378', 'ctest-vn2-60124453'], 'policys': [['default-domain', 'ctest-AnalyticsTestSanityWithResource-33294378', 'policy1']], 'subnet_ipam': [{'subnet_cidr': '141.70.171.0/26', 'ipam_fq_name': ['default-domain', 'default-project', 'default-network-ipam']}], 'port_security_enabled': True, 'description': ''}}
2025-08-04 04:02:19,460 - DEBUG - Requesting: http://10.0.0.27:8082/domains
2025-08-04 04:02:19,522 - DEBUG - Requesting: http://10.0.0.27:8082/domain/8d00e278-1e00-4abc-a644-cc23d080d5d1
2025-08-04 04:02:19,583 - DEBUG - Requesting: http://10.0.0.27:8082/project/a48ab530-4ea0-4d35-9e73-0dcdaaab3e63
2025-08-04 04:02:19,695 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/f4f2b026-bd63-4f2f-b7d1-2cc78ff80148
2025-08-04 04:02:19,717 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/f4f2b026-bd63-4f2f-b7d1-2cc78ff80148
2025-08-04 04:02:19,736 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/e99be9e0-5910-4043-aa77-dbeaa2b7ac52
2025-08-04 04:02:19,749 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/e99be9e0-5910-4043-aa77-dbeaa2b7ac52
2025-08-04 04:02:19,765 - DEBUG - Requesting: http://10.0.0.27:8082/route-target/ef14356b-eec4-430c-a633-8e3efe436700
2025-08-04 04:02:19,776 - DEBUG - Route Targets: ['target:64512:8000005']
2025-08-04 04:02:19,776 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/f4f2b026-bd63-4f2f-b7d1-2cc78ff80148
2025-08-04 04:02:19,791 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/e99be9e0-5910-4043-aa77-dbeaa2b7ac52
2025-08-04 04:02:19,816 - INFO - Verified VN network id 13 for VN f4f2b026-bd63-4f2f-b7d1-2cc78ff80148
2025-08-04 04:02:19,816 - INFO - Verifications in API Server for VN ctest-vn1-08779962 passed
2025-08-04 04:02:19,816 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/f4f2b026-bd63-4f2f-b7d1-2cc78ff80148
2025-08-04 04:02:19,831 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/e99be9e0-5910-4043-aa77-dbeaa2b7ac52
2025-08-04 04:02:19,845 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/e99be9e0-5910-4043-aa77-dbeaa2b7ac52
2025-08-04 04:02:19,857 - DEBUG - Requesting: http://10.0.0.27:8082/route-target/ef14356b-eec4-430c-a633-8e3efe436700
2025-08-04 04:02:19,873 - DEBUG - Control-node 10.0.0.27 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962', 'interests': '0', 'advertised': '0', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-33294378', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '17650363570051043119', 'uuid-lslong': '13245417214698848584'}, 'enable': 'true', 'created': '2025-08-04T03:56:17', 'last-modified': '2025-08-04T04:02:19', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.810447'}
2025-08-04 04:02:19,881 - DEBUG - Route Targets: ['target:64512:8000005']
2025-08-04 04:02:19,889 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-33294378', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '17650363570051043119', 'uuid-lslong': '13245417214698848584'}, 'enable': 'true', 'created': '2025-08-04T03:56:17', 'last-modified': '2025-08-04T04:02:19', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.826287'}
2025-08-04 04:02:19,899 - DEBUG - Route Targets: ['target:64512:8000005']
2025-08-04 04:02:19,907 - DEBUG - Control-node 10.0.0.61 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-33294378', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '17650363570051043119', 'uuid-lslong': '13245417214698848584'}, 'enable': 'true', 'created': '2025-08-04T03:56:17', 'last-modified': '2025-08-04T04:02:19', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.843232'}
2025-08-04 04:02:19,917 - DEBUG - Route Targets: ['target:64512:8000005']
2025-08-04 04:02:19,917 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn1-08779962 passed
2025-08-04 04:02:19,917 - DEBUG - ====Verifying policy data for ctest-vn1-08779962 in API_Server ======
2025-08-04 04:02:19,917 - DEBUG - Requesting: http://10.0.0.27:8082/domains
2025-08-04 04:02:19,925 - DEBUG - Requesting: http://10.0.0.27:8082/domain/8d00e278-1e00-4abc-a644-cc23d080d5d1
2025-08-04 04:02:19,941 - DEBUG - Requesting: http://10.0.0.27:8082/project/a48ab530-4ea0-4d35-9e73-0dcdaaab3e63
2025-08-04 04:02:20,052 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/f4f2b026-bd63-4f2f-b7d1-2cc78ff80148
2025-08-04 04:02:20,065 - DEBUG - ==>Verifying data for policy with id: 34d84b98-dac0-43ea-b8f6-fac071ae686d, fqn: ['default-domain', 'ctest-AnalyticsTestSanityWithResource-33294378', 'policy1']
2025-08-04 04:02:20,065 - INFO - VN ctest-vn1-08779962 Policy verification: verify_vn_policy_in_api_server, status: True
2025-08-04 04:02:20,065 - DEBUG - Verifying the vn in opserver
2025-08-04 04:02:20,065 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962 virtual network link through opserver 10.0.0.27
2025-08-04 04:02:20,065 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/virtual-networks
2025-08-04 04:02:20,083 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962', 'href': 'http://10.0.0.27:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962?flat'}
2025-08-04 04:02:20,083 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962 virtual network link through opserver 10.0.0.38
2025-08-04 04:02:20,083 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks
2025-08-04 04:02:20,161 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962?flat'}
2025-08-04 04:02:20,161 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962 virtual network link through opserver 10.0.0.61
2025-08-04 04:02:20,161 - DEBUG - Requesting: http://10.0.0.61:8081/analytics/uves/virtual-networks
2025-08-04 04:02:20,226 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962', 'href': 'http://10.0.0.61:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962?flat'}
2025-08-04 04:02:20,226 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962 is found in opserver
2025-08-04 04:02:20,240 - DEBUG - VRF ids for VN ctest-vn1-08779962: {'10.0.0.57': '2'}
2025-08-04 04:02:20,240 - DEBUG - Requesting: http://10.0.0.27:8082/domains
2025-08-04 04:02:20,249 - DEBUG - Requesting: http://10.0.0.27:8082/domain/8d00e278-1e00-4abc-a644-cc23d080d5d1
2025-08-04 04:02:20,264 - DEBUG - Requesting: http://10.0.0.27:8082/project/a48ab530-4ea0-4d35-9e73-0dcdaaab3e63
2025-08-04 04:02:20,369 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/2b6cfeea-fcc2-463e-b59a-9ff2a1d5442d
2025-08-04 04:02:20,389 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/2b6cfeea-fcc2-463e-b59a-9ff2a1d5442d
2025-08-04 04:02:20,403 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/30e0c87d-9965-4af0-9284-aaf0f0fe02c3
2025-08-04 04:02:20,417 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/30e0c87d-9965-4af0-9284-aaf0f0fe02c3
2025-08-04 04:02:20,429 - DEBUG - Requesting: http://10.0.0.27:8082/route-target/5c49b08b-a708-46d0-a73c-8dea330b70de
2025-08-04 04:02:20,441 - DEBUG - Route Targets: ['target:64512:8000006']
2025-08-04 04:02:20,441 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/2b6cfeea-fcc2-463e-b59a-9ff2a1d5442d
2025-08-04 04:02:20,456 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/30e0c87d-9965-4af0-9284-aaf0f0fe02c3
2025-08-04 04:02:20,480 - INFO - Verified VN network id 14 for VN 2b6cfeea-fcc2-463e-b59a-9ff2a1d5442d
2025-08-04 04:02:20,480 - INFO - Verifications in API Server for VN ctest-vn2-60124453 passed
2025-08-04 04:02:20,480 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/2b6cfeea-fcc2-463e-b59a-9ff2a1d5442d
2025-08-04 04:02:20,499 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/30e0c87d-9965-4af0-9284-aaf0f0fe02c3
2025-08-04 04:02:20,512 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/30e0c87d-9965-4af0-9284-aaf0f0fe02c3
2025-08-04 04:02:20,524 - DEBUG - Requesting: http://10.0.0.27:8082/route-target/5c49b08b-a708-46d0-a73c-8dea330b70de
2025-08-04 04:02:20,552 - DEBUG - Control-node 10.0.0.27 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453', 'interests': '0', 'advertised': '0', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-33294378', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '3129156126332044862', 'uuid-lslong': '13085947531724407853'}, 'enable': 'true', 'created': '2025-08-04T03:56:53', 'last-modified': '2025-08-04T04:02:19', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:01.238169'}
2025-08-04 04:02:20,565 - DEBUG - Route Targets: ['target:64512:8000006']
2025-08-04 04:02:20,575 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-33294378', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '3129156126332044862', 'uuid-lslong': '13085947531724407853'}, 'enable': 'true', 'created': '2025-08-04T03:56:53', 'last-modified': '2025-08-04T04:02:19', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:01.263801'}
2025-08-04 04:02:20,593 - DEBUG - Route Targets: ['target:64512:8000006']
2025-08-04 04:02:20,604 - DEBUG - Control-node 10.0.0.61 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453', 'interests': '0', 'advertised': '0', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-33294378', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '3129156126332044862', 'uuid-lslong': '13085947531724407853'}, 'enable': 'true', 'created': '2025-08-04T03:56:53', 'last-modified': '2025-08-04T04:02:19', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:01.292281'}
2025-08-04 04:02:20,617 - DEBUG - Route Targets: ['target:64512:8000006']
2025-08-04 04:02:20,617 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn2-60124453 passed
2025-08-04 04:02:20,617 - DEBUG - ====Verifying policy data for ctest-vn2-60124453 in API_Server ======
2025-08-04 04:02:20,617 - DEBUG - Requesting: http://10.0.0.27:8082/domains
2025-08-04 04:02:20,626 - DEBUG - Requesting: http://10.0.0.27:8082/domain/8d00e278-1e00-4abc-a644-cc23d080d5d1
2025-08-04 04:02:20,647 - DEBUG - Requesting: http://10.0.0.27:8082/project/a48ab530-4ea0-4d35-9e73-0dcdaaab3e63
2025-08-04 04:02:20,756 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/2b6cfeea-fcc2-463e-b59a-9ff2a1d5442d
2025-08-04 04:02:20,772 - DEBUG - ==>Verifying data for policy with id: 34d84b98-dac0-43ea-b8f6-fac071ae686d, fqn: ['default-domain', 'ctest-AnalyticsTestSanityWithResource-33294378', 'policy1']
2025-08-04 04:02:20,772 - INFO - VN ctest-vn2-60124453 Policy verification: verify_vn_policy_in_api_server, status: True
2025-08-04 04:02:20,772 - DEBUG - Verifying the vn in opserver
2025-08-04 04:02:20,772 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453 virtual network link through opserver 10.0.0.27
2025-08-04 04:02:20,772 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/virtual-networks
2025-08-04 04:02:20,788 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453', 'href': 'http://10.0.0.27:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453?flat'}
2025-08-04 04:02:20,789 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453 virtual network link through opserver 10.0.0.38
2025-08-04 04:02:20,789 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks
2025-08-04 04:02:20,850 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453?flat'}
2025-08-04 04:02:20,850 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453 virtual network link through opserver 10.0.0.61
2025-08-04 04:02:20,850 - DEBUG - Requesting: http://10.0.0.61:8081/analytics/uves/virtual-networks
2025-08-04 04:02:20,916 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453', 'href': 'http://10.0.0.61:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453?flat'}
2025-08-04 04:02:20,916 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453 is found in opserver
2025-08-04 04:02:20,934 - DEBUG - VRF ids for VN ctest-vn2-60124453: {'10.0.0.41': '2', '10.0.0.57': '3'}
2025-08-04 04:02:20,934 - DEBUG - Requesting: http://10.0.0.27:8082/domains
2025-08-04 04:02:20,943 - DEBUG - Requesting: http://10.0.0.27:8082/domain/8d00e278-1e00-4abc-a644-cc23d080d5d1
2025-08-04 04:02:20,960 - DEBUG - Requesting: http://10.0.0.27:8082/project/a48ab530-4ea0-4d35-9e73-0dcdaaab3e63
2025-08-04 04:02:21,065 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/f4f2b026-bd63-4f2f-b7d1-2cc78ff80148
2025-08-04 04:02:21,078 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/f4f2b026-bd63-4f2f-b7d1-2cc78ff80148
2025-08-04 04:02:21,093 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/e99be9e0-5910-4043-aa77-dbeaa2b7ac52
2025-08-04 04:02:21,104 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/e99be9e0-5910-4043-aa77-dbeaa2b7ac52
2025-08-04 04:02:21,116 - DEBUG - Requesting: http://10.0.0.27:8082/route-target/ef14356b-eec4-430c-a633-8e3efe436700
2025-08-04 04:02:21,125 - DEBUG - Route Targets: ['target:64512:8000005']
2025-08-04 04:02:21,125 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/f4f2b026-bd63-4f2f-b7d1-2cc78ff80148
2025-08-04 04:02:21,145 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/e99be9e0-5910-4043-aa77-dbeaa2b7ac52
2025-08-04 04:02:21,170 - INFO - Verified VN network id 13 for VN f4f2b026-bd63-4f2f-b7d1-2cc78ff80148
2025-08-04 04:02:21,170 - INFO - Verifications in API Server for VN ctest-vn1-08779962 passed
2025-08-04 04:02:21,170 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/f4f2b026-bd63-4f2f-b7d1-2cc78ff80148
2025-08-04 04:02:21,184 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/e99be9e0-5910-4043-aa77-dbeaa2b7ac52
2025-08-04 04:02:21,196 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/e99be9e0-5910-4043-aa77-dbeaa2b7ac52
2025-08-04 04:02:21,209 - DEBUG - Requesting: http://10.0.0.27:8082/route-target/ef14356b-eec4-430c-a633-8e3efe436700
2025-08-04 04:02:21,230 - DEBUG - Control-node 10.0.0.27 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962', 'interests': '0', 'advertised': '0', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-33294378', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '17650363570051043119', 'uuid-lslong': '13245417214698848584'}, 'enable': 'true', 'created': '2025-08-04T03:56:17', 'last-modified': '2025-08-04T04:02:19', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:02.166748'}
2025-08-04 04:02:21,239 - DEBUG - Route Targets: ['target:64512:8000005']
2025-08-04 04:02:21,247 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-33294378', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '17650363570051043119', 'uuid-lslong': '13245417214698848584'}, 'enable': 'true', 'created': '2025-08-04T03:56:17', 'last-modified': '2025-08-04T04:02:19', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:02.182926'}
2025-08-04 04:02:21,255 - DEBUG - Route Targets: ['target:64512:8000005']
2025-08-04 04:02:21,263 - DEBUG - Control-node 10.0.0.61 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-33294378', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '17650363570051043119', 'uuid-lslong': '13245417214698848584'}, 'enable': 'true', 'created': '2025-08-04T03:56:17', 'last-modified': '2025-08-04T04:02:19', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:02.199012'}
2025-08-04 04:02:21,271 - DEBUG - Route Targets: ['target:64512:8000005']
2025-08-04 04:02:21,271 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn1-08779962 passed
2025-08-04 04:02:21,272 - DEBUG - ====Verifying policy data for ctest-vn1-08779962 in API_Server ======
2025-08-04 04:02:21,272 - DEBUG - Requesting: http://10.0.0.27:8082/domains
2025-08-04 04:02:21,281 - DEBUG - Requesting: http://10.0.0.27:8082/domain/8d00e278-1e00-4abc-a644-cc23d080d5d1
2025-08-04 04:02:21,296 - DEBUG - Requesting: http://10.0.0.27:8082/project/a48ab530-4ea0-4d35-9e73-0dcdaaab3e63
2025-08-04 04:02:21,400 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/f4f2b026-bd63-4f2f-b7d1-2cc78ff80148
2025-08-04 04:02:21,413 - DEBUG - ==>Verifying data for policy with id: 34d84b98-dac0-43ea-b8f6-fac071ae686d, fqn: ['default-domain', 'ctest-AnalyticsTestSanityWithResource-33294378', 'policy1']
2025-08-04 04:02:21,414 - INFO - VN ctest-vn1-08779962 Policy verification: verify_vn_policy_in_api_server, status: True
2025-08-04 04:02:21,414 - DEBUG - Verifying the vn in opserver
2025-08-04 04:02:21,414 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962 virtual network link through opserver 10.0.0.27
2025-08-04 04:02:21,414 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/virtual-networks
2025-08-04 04:02:21,431 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962', 'href': 'http://10.0.0.27:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962?flat'}
2025-08-04 04:02:21,431 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962 virtual network link through opserver 10.0.0.38
2025-08-04 04:02:21,431 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks
2025-08-04 04:02:21,490 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962?flat'}
2025-08-04 04:02:21,490 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962 virtual network link through opserver 10.0.0.61
2025-08-04 04:02:21,490 - DEBUG - Requesting: http://10.0.0.61:8081/analytics/uves/virtual-networks
2025-08-04 04:02:21,507 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962', 'href': 'http://10.0.0.61:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962?flat'}
2025-08-04 04:02:21,507 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962 is found in opserver
2025-08-04 04:02:21,524 - DEBUG - VRF ids for VN ctest-vn1-08779962: {'10.0.0.57': '2'}
2025-08-04 04:02:21,525 - DEBUG - Requesting: http://10.0.0.27:8082/domains
2025-08-04 04:02:21,532 - DEBUG - Requesting: http://10.0.0.27:8082/domain/8d00e278-1e00-4abc-a644-cc23d080d5d1
2025-08-04 04:02:21,548 - DEBUG - Requesting: http://10.0.0.27:8082/project/a48ab530-4ea0-4d35-9e73-0dcdaaab3e63
2025-08-04 04:02:21,650 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/2b6cfeea-fcc2-463e-b59a-9ff2a1d5442d
2025-08-04 04:02:21,663 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/2b6cfeea-fcc2-463e-b59a-9ff2a1d5442d
2025-08-04 04:02:21,677 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/30e0c87d-9965-4af0-9284-aaf0f0fe02c3
2025-08-04 04:02:21,688 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/30e0c87d-9965-4af0-9284-aaf0f0fe02c3
2025-08-04 04:02:21,698 - DEBUG - Requesting: http://10.0.0.27:8082/route-target/5c49b08b-a708-46d0-a73c-8dea330b70de
2025-08-04 04:02:21,706 - DEBUG - Route Targets: ['target:64512:8000006']
2025-08-04 04:02:21,706 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/2b6cfeea-fcc2-463e-b59a-9ff2a1d5442d
2025-08-04 04:02:21,719 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/30e0c87d-9965-4af0-9284-aaf0f0fe02c3
2025-08-04 04:02:21,740 - INFO - Verified VN network id 14 for VN 2b6cfeea-fcc2-463e-b59a-9ff2a1d5442d
2025-08-04 04:02:21,741 - INFO - Verifications in API Server for VN ctest-vn2-60124453 passed
2025-08-04 04:02:21,741 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/2b6cfeea-fcc2-463e-b59a-9ff2a1d5442d
2025-08-04 04:02:21,755 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/30e0c87d-9965-4af0-9284-aaf0f0fe02c3
2025-08-04 04:02:21,767 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/30e0c87d-9965-4af0-9284-aaf0f0fe02c3
2025-08-04 04:02:21,779 - DEBUG - Requesting: http://10.0.0.27:8082/route-target/5c49b08b-a708-46d0-a73c-8dea330b70de
2025-08-04 04:02:21,801 - DEBUG - Control-node 10.0.0.27 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453', 'interests': '0', 'advertised': '0', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-33294378', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '3129156126332044862', 'uuid-lslong': '13085947531724407853'}, 'enable': 'true', 'created': '2025-08-04T03:56:53', 'last-modified': '2025-08-04T04:02:19', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:02.489648'}
2025-08-04 04:02:21,811 - DEBUG - Route Targets: ['target:64512:8000006']
2025-08-04 04:02:21,818 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-33294378', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '3129156126332044862', 'uuid-lslong': '13085947531724407853'}, 'enable': 'true', 'created': '2025-08-04T03:56:53', 'last-modified': '2025-08-04T04:02:19', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:02.508856'}
2025-08-04 04:02:21,825 - DEBUG - Route Targets: ['target:64512:8000006']
2025-08-04 04:02:21,833 - DEBUG - Control-node 10.0.0.61 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453', 'interests': '0', 'advertised': '0', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-33294378', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '3129156126332044862', 'uuid-lslong': '13085947531724407853'}, 'enable': 'true', 'created': '2025-08-04T03:56:53', 'last-modified': '2025-08-04T04:02:19', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:02.523621'}
2025-08-04 04:02:21,842 - DEBUG - Route Targets: ['target:64512:8000006']
2025-08-04 04:02:21,842 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn2-60124453 passed
2025-08-04 04:02:21,842 - DEBUG - ====Verifying policy data for ctest-vn2-60124453 in API_Server ======
2025-08-04 04:02:21,842 - DEBUG - Requesting: http://10.0.0.27:8082/domains
2025-08-04 04:02:21,851 - DEBUG - Requesting: http://10.0.0.27:8082/domain/8d00e278-1e00-4abc-a644-cc23d080d5d1
2025-08-04 04:02:21,867 - DEBUG - Requesting: http://10.0.0.27:8082/project/a48ab530-4ea0-4d35-9e73-0dcdaaab3e63
2025-08-04 04:02:21,976 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/2b6cfeea-fcc2-463e-b59a-9ff2a1d5442d
2025-08-04 04:02:21,990 - DEBUG - ==>Verifying data for policy with id: 34d84b98-dac0-43ea-b8f6-fac071ae686d, fqn: ['default-domain', 'ctest-AnalyticsTestSanityWithResource-33294378', 'policy1']
2025-08-04 04:02:21,990 - INFO - VN ctest-vn2-60124453 Policy verification: verify_vn_policy_in_api_server, status: True
2025-08-04 04:02:21,990 - DEBUG - Verifying the vn in opserver
2025-08-04 04:02:21,990 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453 virtual network link through opserver 10.0.0.27
2025-08-04 04:02:21,991 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/virtual-networks
2025-08-04 04:02:22,007 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453', 'href': 'http://10.0.0.27:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453?flat'}
2025-08-04 04:02:22,007 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453 virtual network link through opserver 10.0.0.38
2025-08-04 04:02:22,007 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks
2025-08-04 04:02:22,025 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453?flat'}
2025-08-04 04:02:22,025 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453 virtual network link through opserver 10.0.0.61
2025-08-04 04:02:22,025 - DEBUG - Requesting: http://10.0.0.61:8081/analytics/uves/virtual-networks
2025-08-04 04:02:22,044 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453', 'href': 'http://10.0.0.61:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453?flat'}
2025-08-04 04:02:22,044 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453 is found in opserver
2025-08-04 04:02:22,057 - DEBUG - VRF ids for VN ctest-vn2-60124453: {'10.0.0.41': '2', '10.0.0.57': '3'}
2025-08-04 04:02:22,057 - DEBUG - Requesting: http://10.0.0.27:8082/domains
2025-08-04 04:02:22,065 - DEBUG - Requesting: http://10.0.0.27:8082/domain/8d00e278-1e00-4abc-a644-cc23d080d5d1
2025-08-04 04:02:22,081 - DEBUG - Requesting: http://10.0.0.27:8082/project/a48ab530-4ea0-4d35-9e73-0dcdaaab3e63
2025-08-04 04:02:22,188 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/803b3dbe-c1bd-4fc2-9c92-5759bc307503
2025-08-04 04:02:22,202 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/803b3dbe-c1bd-4fc2-9c92-5759bc307503
2025-08-04 04:02:22,216 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/e2f98916-810f-46ff-b3a0-8d9cb6c4d1d1
2025-08-04 04:02:22,227 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/e2f98916-810f-46ff-b3a0-8d9cb6c4d1d1
2025-08-04 04:02:22,237 - DEBUG - Requesting: http://10.0.0.27:8082/route-target/3d83ade0-6dae-4396-ad20-78a064cc6683
2025-08-04 04:02:22,246 - DEBUG - Route Targets: ['target:64512:8000007']
2025-08-04 04:02:22,246 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/803b3dbe-c1bd-4fc2-9c92-5759bc307503
2025-08-04 04:02:22,257 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/e2f98916-810f-46ff-b3a0-8d9cb6c4d1d1
2025-08-04 04:02:22,281 - INFO - Verified VN network id 15 for VN 803b3dbe-c1bd-4fc2-9c92-5759bc307503
2025-08-04 04:02:22,281 - INFO - Verifications in API Server for VN ctest-fip_vn-14850913 passed
2025-08-04 04:02:22,281 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/803b3dbe-c1bd-4fc2-9c92-5759bc307503
2025-08-04 04:02:22,300 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/e2f98916-810f-46ff-b3a0-8d9cb6c4d1d1
2025-08-04 04:02:22,314 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/e2f98916-810f-46ff-b3a0-8d9cb6c4d1d1
2025-08-04 04:02:22,324 - DEBUG - Requesting: http://10.0.0.27:8082/route-target/3d83ade0-6dae-4396-ad20-78a064cc6683
2025-08-04 04:02:22,342 - DEBUG - Control-node 10.0.0.27 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-fip_vn-14850913', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-33294378', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '9240046949984194498', 'uuid-lslong': '11282176059437511939'}, 'enable': 'true', 'created': '2025-08-04T03:56:54', 'last-modified': '2025-08-04T03:56:54', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:05:27.435096'}
2025-08-04 04:02:22,352 - DEBUG - Route Targets: ['target:64512:8000007']
2025-08-04 04:02:22,358 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-fip_vn-14850913', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-33294378', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '9240046949984194498', 'uuid-lslong': '11282176059437511939'}, 'enable': 'true', 'created': '2025-08-04T03:56:54', 'last-modified': '2025-08-04T03:56:54', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:05:27.460211'}
2025-08-04 04:02:22,366 - DEBUG - Route Targets: ['target:64512:8000007']
2025-08-04 04:02:22,374 - DEBUG - Control-node 10.0.0.61 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-fip_vn-14850913', 'interests': '0', 'advertised': '0', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-AnalyticsTestSanityWithResource-33294378', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '9240046949984194498', 'uuid-lslong': '11282176059437511939'}, 'enable': 'true', 'created': '2025-08-04T03:56:54', 'last-modified': '2025-08-04T03:56:54', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:05:27.471204'}
2025-08-04 04:02:22,385 - DEBUG - Route Targets: ['target:64512:8000007']
2025-08-04 04:02:22,385 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-fip_vn-14850913 passed
2025-08-04 04:02:22,385 - DEBUG - ====Verifying policy data for ctest-fip_vn-14850913 in API_Server ======
2025-08-04 04:02:22,385 - DEBUG - Requesting: http://10.0.0.27:8082/domains
2025-08-04 04:02:22,393 - DEBUG - Requesting: http://10.0.0.27:8082/domain/8d00e278-1e00-4abc-a644-cc23d080d5d1
2025-08-04 04:02:22,411 - DEBUG - Requesting: http://10.0.0.27:8082/project/a48ab530-4ea0-4d35-9e73-0dcdaaab3e63
2025-08-04 04:02:22,528 - DEBUG - Requesting: http://10.0.0.27:8082/virtual-network/803b3dbe-c1bd-4fc2-9c92-5759bc307503
2025-08-04 04:02:22,548 - DEBUG - =>VN ctest-fip_vn-14850913 has no policy to be verified
2025-08-04 04:02:22,548 - DEBUG - Verifying the vn in opserver
2025-08-04 04:02:22,548 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-fip_vn-14850913 virtual network link through opserver 10.0.0.27
2025-08-04 04:02:22,548 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/virtual-networks
2025-08-04 04:02:22,567 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-fip_vn-14850913', 'href': 'http://10.0.0.27:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-fip_vn-14850913?flat'}
2025-08-04 04:02:22,567 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-fip_vn-14850913 virtual network link through opserver 10.0.0.38
2025-08-04 04:02:22,567 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks
2025-08-04 04:02:22,582 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-fip_vn-14850913', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-fip_vn-14850913?flat'}
2025-08-04 04:02:22,582 - DEBUG - Verifying the default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-fip_vn-14850913 virtual network link through opserver 10.0.0.61
2025-08-04 04:02:22,583 - DEBUG - Requesting: http://10.0.0.61:8081/analytics/uves/virtual-networks
2025-08-04 04:02:22,601 - DEBUG - vn link and name as {'name': 'default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-fip_vn-14850913', 'href': 'http://10.0.0.61:8081/analytics/uves/virtual-network/default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-fip_vn-14850913?flat'}
2025-08-04 04:02:22,601 - INFO - Validated that VN default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-fip_vn-14850913 is found in opserver
2025-08-04 04:02:22,620 - DEBUG - VRF ids for VN ctest-fip_vn-14850913: {'10.0.0.41': '3'}
2025-08-04 04:02:22,620 - INFO - start time= 1754280142620279
2025-08-04 04:02:22,620 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-08-04 04:02:22,620 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-08-04 04:02:22,620 - DEBUG - File /contrail-test/tcutils/traffic_utils/../udpechoserver.py already copied to 141.70.171.3
2025-08-04 04:02:22,620 - DEBUG - Running remote_cmd, Cmd : python /tmp/udpechoserver.py --start_port 9000 --end_port 9000 --pid_file /tmp/server-ctest-random-60301236.pid --stats_file /tmp/server-ctest-random-60301236.stats 0<&- &> /tmp/server-ctest-random-60301236.log, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.41, gateway password: c0ntrail123
2025-08-04 04:02:22,620 - DEBUG - nohup python /tmp/udpechoserver.py --start_port 9000 --end_port 9000 --pid_file /tmp/server-ctest-random-60301236.pid --stats_file /tmp/server-ctest-random-60301236.stats 0<&- &> /tmp/server-ctest-random-60301236.log &
2025-08-04 04:02:22,875 - DEBUG - None
2025-08-04 04:02:22,876 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-08-04 04:02:22,876 - DEBUG - File /contrail-test/tcutils/traffic_utils/../udpechoclient.py already copied to 214.167.83.195
2025-08-04 04:02:22,876 - DEBUG - Running remote_cmd, Cmd : python /tmp/udpechoclient.py --servers 141.70.171.3 --sport 8000 --dports 9000 --retry --count 100 --pid_file /tmp/client-ctest-random-60301236.pid --stats_file /tmp/client-ctest-random-60301236.stats 0<&- &> /tmp/client-ctest-random-60301236.log, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.57, gateway password: c0ntrail123
2025-08-04 04:02:22,876 - DEBUG - nohup python /tmp/udpechoclient.py --servers 141.70.171.3 --sport 8000 --dports 9000 --retry --count 100 --pid_file /tmp/client-ctest-random-60301236.pid --stats_file /tmp/client-ctest-random-60301236.stats 0<&- &> /tmp/client-ctest-random-60301236.log &
2025-08-04 04:02:23,095 - DEBUG - None
2025-08-04 04:02:23,095 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-08-04 04:02:23,096 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-08-04 04:02:23,096 - DEBUG - File /contrail-test/tcutils/traffic_utils/../udpechoserver.py already copied to 141.70.171.3
2025-08-04 04:02:23,096 - DEBUG - Running remote_cmd, Cmd : python /tmp/udpechoserver.py --start_port 9001 --end_port 9001 --pid_file /tmp/server-ctest-random-97693672.pid --stats_file /tmp/server-ctest-random-97693672.stats 0<&- &> /tmp/server-ctest-random-97693672.log, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.41, gateway password: c0ntrail123
2025-08-04 04:02:23,096 - DEBUG - nohup python /tmp/udpechoserver.py --start_port 9001 --end_port 9001 --pid_file /tmp/server-ctest-random-97693672.pid --stats_file /tmp/server-ctest-random-97693672.stats 0<&- &> /tmp/server-ctest-random-97693672.log &
2025-08-04 04:02:23,357 - DEBUG - None
2025-08-04 04:02:23,357 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-08-04 04:02:23,357 - DEBUG - File /contrail-test/tcutils/traffic_utils/../udpechoclient.py already copied to 214.167.83.195
2025-08-04 04:02:23,357 - DEBUG - Running remote_cmd, Cmd : python /tmp/udpechoclient.py --servers 141.70.171.3 --sport 8000 --dports 9001 --retry --count 100 --pid_file /tmp/client-ctest-random-97693672.pid --stats_file /tmp/client-ctest-random-97693672.stats 0<&- &> /tmp/client-ctest-random-97693672.log, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.57, gateway password: c0ntrail123
2025-08-04 04:02:23,358 - DEBUG - nohup python /tmp/udpechoclient.py --servers 141.70.171.3 --sport 8000 --dports 9001 --retry --count 100 --pid_file /tmp/client-ctest-random-97693672.pid --stats_file /tmp/client-ctest-random-97693672.stats 0<&- &> /tmp/client-ctest-random-97693672.log &
2025-08-04 04:02:23,609 - DEBUG - None
2025-08-04 04:02:23,609 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-08-04 04:02:23,610 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-08-04 04:02:23,610 - DEBUG - File /contrail-test/tcutils/traffic_utils/../udpechoserver.py already copied to 141.70.171.3
2025-08-04 04:02:23,610 - DEBUG - Running remote_cmd, Cmd : python /tmp/udpechoserver.py --start_port 9003 --end_port 9003 --pid_file /tmp/server-ctest-random-26811757.pid --stats_file /tmp/server-ctest-random-26811757.stats 0<&- &> /tmp/server-ctest-random-26811757.log, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.41, gateway password: c0ntrail123
2025-08-04 04:02:23,610 - DEBUG - nohup python /tmp/udpechoserver.py --start_port 9003 --end_port 9003 --pid_file /tmp/server-ctest-random-26811757.pid --stats_file /tmp/server-ctest-random-26811757.stats 0<&- &> /tmp/server-ctest-random-26811757.log &
2025-08-04 04:02:23,840 - DEBUG - None
2025-08-04 04:02:23,840 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set
2025-08-04 04:02:23,840 - DEBUG - File /contrail-test/tcutils/traffic_utils/../udpechoclient.py already copied to 214.167.83.195
2025-08-04 04:02:23,840 - DEBUG - Running remote_cmd, Cmd : python /tmp/udpechoclient.py --servers 141.70.171.3 --sport 8000 --dports 9003 --retry --count 100 --pid_file /tmp/client-ctest-random-26811757.pid --stats_file /tmp/client-ctest-random-26811757.stats 0<&- &> /tmp/client-ctest-random-26811757.log, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.57, gateway password: c0ntrail123
2025-08-04 04:02:23,840 - DEBUG - nohup python /tmp/udpechoclient.py --servers 141.70.171.3 --sport 8000 --dports 9003 --retry --count 100 --pid_file /tmp/client-ctest-random-26811757.pid --stats_file /tmp/client-ctest-random-26811757.stats 0<&- &> /tmp/client-ctest-random-26811757.log &
2025-08-04 04:02:24,116 - DEBUG - None
2025-08-04 04:02:34,117 - DEBUG - Running remote_cmd, Cmd : kill $(cat /tmp/client-ctest-random-60301236.pid); sync; cat /tmp/client-ctest-random-60301236.stats, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.57, gateway password: c0ntrail123
2025-08-04 04:02:34,117 - DEBUG - kill $(cat /tmp/client-ctest-random-60301236.pid); sync; cat /tmp/client-ctest-random-60301236.stats
2025-08-04 04:02:34,425 - DEBUG - cat: can't open '/tmp/client-ctest-random-60301236.pid': No such file or directory
sh: you need to specify whom to kill
cat: can't open '/tmp/client-ctest-random-60301236.stats': No such file or directory
2025-08-04 04:02:34,426 - DEBUG - Running remote_cmd, Cmd : kill $(cat /tmp/server-ctest-random-60301236.pid); sync; cat /tmp/server-ctest-random-60301236.stats, host_string: cirros@169.254.0.3, password: gocubsgogateway: ubuntu@10.0.0.41, gateway password: c0ntrail123
2025-08-04 04:02:34,426 - DEBUG - kill $(cat /tmp/server-ctest-random-60301236.pid); sync; cat /tmp/server-ctest-random-60301236.stats
2025-08-04 04:02:34,750 - DEBUG - cat: can't open '/tmp/server-ctest-random-60301236.pid': No such file or directory
sh: you need to specify whom to kill
cat: can't open '/tmp/server-ctest-random-60301236.stats': No such file or directory
2025-08-04 04:02:34,750 - INFO - Client - Sent: 0, Received: 0
2025-08-04 04:02:34,750 - INFO - Server - Sent: 0, Received: 0
2025-08-04 04:02:35,401 - INFO - Deleted policy policy1
2025-08-04 04:02:35,580 - INFO - Setting flow export rate: None
2025-08-04 04:02:36,864 - ERROR - AssertionError
Python 3.6.8: /usr/bin/python3
Mon Aug 4 04:02:34 2025
A problem occurred in a Python script. Here is the sequence of
function calls leading up to the error, in the order they occurred.
/contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={})
78 log.info('Initial checks done. Running the testcase now')
79 log.info('')
80 result = function(self, *args, **kwargs)
81 if self.inputs.upgrade:
82 pid = os.getpid()
result = None
function =
self =
args = ()
kwargs = {}
/contrail-test/serial_scripts/analytics/test_analytics_resource.py in test_verify_session_series_table_inter_vn(self=)
1336 start_time = self.analytics_obj.getstarttime(vm_node_ip)
1337 self.logger.info("start time= %s" % (start_time))
1338 self.setup_and_create_streams(self.res.vn1_vm1_fixture, self.res.vn2_vm2_fixture)
1339 src_vn = self.res.vn1_fixture.vn_fq_name
1340 dst_vn = self.res.vn2_fixture.vn_fq_name
self =
self.setup_and_create_streams = >
self.res =
self.res.vn1_vm1_fixture =
self.res.vn2_vm2_fixture =
/contrail-test/serial_scripts/analytics/base.py in setup_and_create_streams(self=, src_vm=, dst_vm=, sport=8000, dport=9003, count=100)
168 time.sleep(10)
169 for traffic_obj in traffic_objs:
170 self.stop_traffic(traffic_obj)
171
172 #end setup_create_streams
self =
self.stop_traffic = >
traffic_obj =
/contrail-test/common/base.py in stop_traffic(self=, traffic_obj=, expectation=True, unidirection=False, partial=False)
796 assert False, "No drops observed in packets %s"%msg
797 elif expectation:
798 assert sent and recv, "Packets not %s"%msg
799 if recv*100/float(sent) < 90:
800 assert False, "Packets not %s"%msg
sent = 0
recv = 0
msg = 'transferred between 214.167.83.195 and 141.70.171.3, proto udp sport 8000 and dport 9000'
AssertionError: Packets not transferred between 214.167.83.195 and 141.70.171.3, proto udp sport 8000 and dport 9000
__cause__ = None
__class__ =
__context__ = None
__delattr__ =
__dict__ = {}
__dir__ =
__doc__ = 'Assertion failed.'
__eq__ =
__format__ =
__ge__ =
__getattribute__ =
__gt__ =
__hash__ =
__init__ =
__init_subclass__ =
__le__ =
__lt__ =
__ne__ =
__new__ =
__reduce__ =
__reduce_ex__ =
__repr__ =
__setattr__ =
__setstate__ =
__sizeof__ =
__str__ =
__subclasshook__ =
__suppress_context__ = False
__traceback__ =
args = ('Packets not transferred between 214.167.83.195 and 141.70.171.3, proto udp sport 8000 and dport 9000',)
with_traceback =
The above is a description of an error in a Python program. Here is
the original traceback:
Traceback (most recent call last):
File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper
result = function(self, *args, **kwargs)
File "/contrail-test/serial_scripts/analytics/test_analytics_resource.py", line 1338, in test_verify_session_series_table_inter_vn
self.setup_and_create_streams(self.res.vn1_vm1_fixture, self.res.vn2_vm2_fixture)
File "/contrail-test/serial_scripts/analytics/base.py", line 170, in setup_and_create_streams
self.stop_traffic(traffic_obj)
File "/contrail-test/common/base.py", line 798, in stop_traffic
assert sent and recv, "Packets not %s"%msg
AssertionError: Packets not transferred between 214.167.83.195 and 141.70.171.3, proto udp sport 8000 and dport 9000
2025-08-04 04:02:36,865 - DEBUG - Skipping xmpp flap check
2025-08-04 04:02:36,865 - INFO -
2025-08-04 04:02:36,865 - INFO - END TEST : test_verify_session_series_table_inter_vn : FAILED[0:00:19]
2025-08-04 04:02:36,866 - INFO - --------------------------------------------------------------------------------
2025-08-04 04:02:36,869 - INFO - Deleting VM mltf_vm
2025-08-04 04:02:36,977 - INFO - Deleting VM ctest-fvn_vm1-68714219
2025-08-04 04:02:37,074 - INFO - Deleting VM ctest-vn2_vm2-40656987
2025-08-04 04:02:37,185 - INFO - Deleting VN ctest-fip_vn-14850913
2025-08-04 04:02:37,255 - DEBUG - VN 803b3dbe-c1bd-4fc2-9c92-5759bc307503 still in use: Unable to complete operation on network 803b3dbe-c1bd-4fc2-9c92-5759bc307503. There are one or more ports still in use on the network.
Neutron server returns request_ids: ['req-ce1e0fae-f07c-4dad-914b-aa58dea96b26']
2025-08-04 04:02:37,255 - WARNING - Deleting VN ctest-fip_vn-14850913 failed..Will retry
2025-08-04 04:02:39,486 - DEBUG - Response for deleting network ()
2025-08-04 04:02:39,486 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/e2f98916-810f-46ff-b3a0-8d9cb6c4d1d1
2025-08-04 04:02:39,509 - DEBUG - Response Code: 404
2025-08-04 04:02:39,509 - DEBUG - Requesting: http://10.0.0.27:8082/domains
2025-08-04 04:02:39,523 - DEBUG - Requesting: http://10.0.0.27:8082/domain/8d00e278-1e00-4abc-a644-cc23d080d5d1
2025-08-04 04:02:39,539 - DEBUG - Requesting: http://10.0.0.27:8082/project/a48ab530-4ea0-4d35-9e73-0dcdaaab3e63
2025-08-04 04:02:39,633 - INFO - Validated that VN ctest-fip_vn-14850913 is not found in API Server
2025-08-04 04:02:39,648 - DEBUG - VN ctest-fip_vn-14850913 is not present in Agent 10.0.0.41
2025-08-04 04:02:39,660 - DEBUG - VN ctest-fip_vn-14850913 is not present in Agent 10.0.0.57
2025-08-04 04:02:39,660 - INFO - Validated that VN ctest-fip_vn-14850913 is not in any agent
2025-08-04 04:02:39,668 - DEBUG - VRF 3 is not seen in agent 10.0.0.41
2025-08-04 04:02:39,677 - DEBUG - Vrouter 10.0.0.41 does not have vrf 3 for VN ctest-fip_vn-14850913
2025-08-04 04:02:39,677 - INFO - Validated that all vrouters do not have the route table for VN default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-fip_vn-14850913
2025-08-04 04:02:39,706 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-fip_vn-14850913 info
2025-08-04 04:02:39,706 - INFO - Deleting VN ctest-vn2-60124453
2025-08-04 04:02:39,892 - DEBUG - Response for deleting network ()
2025-08-04 04:02:39,892 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/30e0c87d-9965-4af0-9284-aaf0f0fe02c3
2025-08-04 04:02:39,912 - DEBUG - Response Code: 404
2025-08-04 04:02:39,912 - DEBUG - Requesting: http://10.0.0.27:8082/domains
2025-08-04 04:02:39,923 - DEBUG - Requesting: http://10.0.0.27:8082/domain/8d00e278-1e00-4abc-a644-cc23d080d5d1
2025-08-04 04:02:39,944 - DEBUG - Requesting: http://10.0.0.27:8082/project/a48ab530-4ea0-4d35-9e73-0dcdaaab3e63
2025-08-04 04:02:40,047 - INFO - Validated that VN ctest-vn2-60124453 is not found in API Server
2025-08-04 04:02:40,061 - DEBUG - VN ctest-vn2-60124453 is not present in Agent 10.0.0.41
2025-08-04 04:02:40,074 - DEBUG - VN ctest-vn2-60124453 is not present in Agent 10.0.0.57
2025-08-04 04:02:40,074 - INFO - Validated that VN ctest-vn2-60124453 is not in any agent
2025-08-04 04:02:40,082 - DEBUG - VRF 2 is not seen in agent 10.0.0.41
2025-08-04 04:02:40,090 - DEBUG - Vrouter 10.0.0.41 does not have vrf 2 for VN ctest-vn2-60124453
2025-08-04 04:02:40,097 - DEBUG - VRF 3 is not seen in agent 10.0.0.57
2025-08-04 04:02:40,111 - DEBUG - Vrouter 10.0.0.57 does not have vrf 3 for VN ctest-vn2-60124453
2025-08-04 04:02:40,111 - INFO - Validated that all vrouters do not have the route table for VN default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn2-60124453
2025-08-04 04:02:40,145 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-vn2-60124453 info
2025-08-04 04:02:40,145 - INFO - Deleting VM ctest-vn1_vm2-95359845
2025-08-04 04:02:40,234 - INFO - Deleting VM ctest-vn1_vm1-33302525
2025-08-04 04:02:40,316 - INFO - Deleting VN ctest-vn1-08779962
2025-08-04 04:02:40,369 - DEBUG - VN f4f2b026-bd63-4f2f-b7d1-2cc78ff80148 still in use: Unable to complete operation on network f4f2b026-bd63-4f2f-b7d1-2cc78ff80148. There are one or more ports still in use on the network.
Neutron server returns request_ids: ['req-6a8444e3-1bea-4b69-a4db-367b71bf90d6']
2025-08-04 04:02:40,369 - WARNING - Deleting VN ctest-vn1-08779962 failed..Will retry
2025-08-04 04:02:42,597 - DEBUG - Response for deleting network ()
2025-08-04 04:02:42,597 - DEBUG - Requesting: http://10.0.0.27:8082/routing-instance/e99be9e0-5910-4043-aa77-dbeaa2b7ac52
2025-08-04 04:02:42,621 - DEBUG - Response Code: 404
2025-08-04 04:02:42,621 - DEBUG - Requesting: http://10.0.0.27:8082/domains
2025-08-04 04:02:42,629 - DEBUG - Requesting: http://10.0.0.27:8082/domain/8d00e278-1e00-4abc-a644-cc23d080d5d1
2025-08-04 04:02:42,647 - DEBUG - Requesting: http://10.0.0.27:8082/project/a48ab530-4ea0-4d35-9e73-0dcdaaab3e63
2025-08-04 04:02:42,752 - INFO - Validated that VN ctest-vn1-08779962 is not found in API Server
2025-08-04 04:02:42,769 - DEBUG - VN ctest-vn1-08779962 is not present in Agent 10.0.0.41
2025-08-04 04:02:42,783 - DEBUG - VN ctest-vn1-08779962 is not present in Agent 10.0.0.57
2025-08-04 04:02:42,784 - INFO - Validated that VN ctest-vn1-08779962 is not in any agent
2025-08-04 04:02:42,792 - DEBUG - VRF 2 is not seen in agent 10.0.0.57
2025-08-04 04:02:42,800 - DEBUG - Vrouter 10.0.0.57 does not have vrf 2 for VN ctest-vn1-08779962
2025-08-04 04:02:42,800 - INFO - Validated that all vrouters do not have the route table for VN default-domain:ctest-AnalyticsTestSanityWithResource-33294378:ctest-vn1-08779962
2025-08-04 04:02:42,837 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-vn1-08779962 info
2025-08-04 04:02:43,674 - INFO - Deleted project: ctest-AnalyticsTestSanityWithResource-33294378, ID : a48ab530-4ea0-4d35-9e73-0dcdaaab3e63