2026-05-25 04:37:04,967 - INFO - Domain Default found not creating 2026-05-25 04:37:05,192 - INFO - Project ctest-TestDropStats-44123704 not found, creating it 2026-05-25 04:37:05,759 - INFO - Created Project:ctest-TestDropStats-44123704, ID : 6428eabb-35f4-4ec6-9119-26db8da706db 2026-05-25 04:37:07,466 - INFO - ================================================================================ 2026-05-25 04:37:07,466 - INFO - STARTING TEST : test_flow_action_drop_stats 2026-05-25 04:37:07,467 - INFO - TEST DESCRIPTION : Description: Verify flow action drop stats Steps: 1. Create 2 VNs and launch 1 VM in each VN, on different nodes on multi node setup 2. Add policy to deny pkts between VNs. 3. Ping VM2 from VM1 Pass criteria: 1. Verify flow action count gets incremented, get it from introspect 2. Verify flow action count gets incremented for the vmi of VM1, get it from introspect 2026-05-25 04:37:08,759 - DEBUG - Skipping xmpp flap check 2026-05-25 04:37:08,759 - INFO - Initial checks done. Running the testcase now 2026-05-25 04:37:08,759 - INFO - 2026-05-25 04:37:08,792 - DEBUG - Services list from nova: [, , , ] 2026-05-25 04:37:08,792 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-5822-2', 'an-jenkins-deploy-platform-ansible-os-5822-1']} 2026-05-25 04:37:09,742 - DEBUG - Response for create_network : {'network': {'id': 'ab98cad2-0770-4438-9e0d-e203391da22f', 'name': 'ctest-vn-50854477', 'tenant_id': '6428eabb35f44ec6911926db8da706db', 'project_id': '6428eabb35f44ec6911926db8da706db', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestDropStats-44123704', 'ctest-vn-50854477'], 'port_security_enabled': True, 'description': ''}} 2026-05-25 04:37:10,119 - DEBUG - Response for create_subnet : {'subnet': {'id': 'c698c43c-44fd-41ad-9f5f-053c1a334614', 'name': '', 'tenant_id': '6428eabb35f44ec6911926db8da706db', 'network_id': 'ab98cad2-0770-4438-9e0d-e203391da22f', 'ip_version': 4, 'cidr': '132.44.154.128/26', 'allocation_pools': [{'start': '132.44.154.130', 'end': '132.44.154.190'}], 'gateway_ip': '132.44.154.129', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '132.44.154.130', 'tags': [], 'project_id': '6428eabb35f44ec6911926db8da706db'}} 2026-05-25 04:37:10,167 - INFO - Created VN ctest-vn-50854477 2026-05-25 04:37:10,193 - DEBUG - VN ctest-vn-50854477 UUID is ab98cad2-0770-4438-9e0d-e203391da22f 2026-05-25 04:37:10,663 - DEBUG - Response for create_network : {'network': {'id': 'a2e9fca9-0d5c-4348-9022-927fc7a3a978', 'name': 'ctest-vn-29452387', 'tenant_id': '6428eabb35f44ec6911926db8da706db', 'project_id': '6428eabb35f44ec6911926db8da706db', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestDropStats-44123704', 'ctest-vn-29452387'], 'port_security_enabled': True, 'description': ''}} 2026-05-25 04:37:10,922 - DEBUG - Response for create_subnet : {'subnet': {'id': '1ef985b8-c85f-4413-9067-e79783bcd179', 'name': '', 'tenant_id': '6428eabb35f44ec6911926db8da706db', 'network_id': 'a2e9fca9-0d5c-4348-9022-927fc7a3a978', 'ip_version': 4, 'cidr': '110.245.112.0/26', 'allocation_pools': [{'start': '110.245.112.2', 'end': '110.245.112.62'}], 'gateway_ip': '110.245.112.1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '110.245.112.2', 'tags': [], 'project_id': '6428eabb35f44ec6911926db8da706db'}} 2026-05-25 04:37:10,947 - INFO - Created VN ctest-vn-29452387 2026-05-25 04:37:10,962 - DEBUG - VN ctest-vn-29452387 UUID is a2e9fca9-0d5c-4348-9022-927fc7a3a978 2026-05-25 04:37:12,646 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-5822-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-5822-1) 2026-05-25 04:37:14,597 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-5822-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-5822-2) 2026-05-25 04:37:14,598 - INFO - Waiting for VM ctest-vm1-07876244 to be up.. 2026-05-25 04:37:14,708 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2026-05-25 04:37:19,814 - DEBUG - VM is in ACTIVE state now 2026-05-25 04:37:19,815 - INFO - VM name : ctest-vm1-07876244 2026-05-25 04:37:19,926 - DEBUG - VM ctest-vm1-07876244 ID is 4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa 2026-05-25 04:37:19,952 - DEBUG - VM ctest-vm1-07876244 launched on Node an-jenkins-deploy-platform-ansible-os-5822-1 2026-05-25 04:37:20,049 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine/4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa 2026-05-25 04:37:20,376 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine/4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa 2026-05-25 04:37:20,420 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine-interface/ea78889e-8114-457a-b7d4-fdbe94d0ecd1 2026-05-25 04:37:23,624 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1029ms') 2026-05-25 04:37:23,624 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-vm1-07876244 failed! 2026-05-25 04:37:23,647 - DEBUG - Gateway for vn default-domain:ctest-TestDropStats-44123704:ctest-vn-50854477 is 132.44.154.129 and allocation pool is NOT set 2026-05-25 04:37:25,715 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=4.88 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=3.07 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 3.069/3.974/4.880/0.905 ms') 2026-05-25 04:37:25,715 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-vm1-07876244 passed 2026-05-25 04:37:25,790 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2026-05-25 04:37:25,790 - DEBUG - Waiting to SSH to VM ctest-vm1-07876244, IP 132.44.154.131, Port 22 2026-05-25 04:37:25,973 - DEBUG - VM ctest-vm1-07876244 is ready for SSH connections 2026-05-25 04:37:25,973 - INFO - Waiting for VM ctest-vm2-71417537 to be up.. 2026-05-25 04:37:26,102 - DEBUG - VM is in ACTIVE state now 2026-05-25 04:37:26,102 - INFO - VM name : ctest-vm2-71417537 2026-05-25 04:37:26,217 - DEBUG - VM ctest-vm2-71417537 ID is 2c7f212d-5279-4a60-bf61-d32430eb6a95 2026-05-25 04:37:26,217 - DEBUG - VM ctest-vm2-71417537 launched on Node an-jenkins-deploy-platform-ansible-os-5822-2 2026-05-25 04:37:26,317 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine/2c7f212d-5279-4a60-bf61-d32430eb6a95 2026-05-25 04:37:26,330 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine-interface/a7d08547-c982-424c-8d47-a03bf0fbd367 2026-05-25 04:37:27,518 - DEBUG - (True, 'PING 169.254.0.5 (169.254.0.5) 56(84) bytes of data.\r\n64 bytes from 169.254.0.5: icmp_seq=1 ttl=63 time=5.35 ms\r\n64 bytes from 169.254.0.5: icmp_seq=2 ttl=63 time=2.27 ms\r\n\r\n--- 169.254.0.5 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 2.265/3.809/5.354/1.544 ms') 2026-05-25 04:37:27,518 - INFO - Ping to Metadata IP 169.254.0.5 of VM ctest-vm2-71417537 passed 2026-05-25 04:37:27,591 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2026-05-25 04:37:27,591 - DEBUG - Waiting to SSH to VM ctest-vm2-71417537, IP 110.245.112.3, Port 22 2026-05-25 04:37:27,672 - DEBUG - Error on ssh to cirros@169.254.0.5:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.5/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.5/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.5/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2026-05-25 04:37:27,784 - DEBUG - VM ctest-vm2-71417537 is NOT ready for SSH connections, VM status: ACTIVE 2026-05-25 04:37:32,785 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2026-05-25 04:37:32,785 - DEBUG - Waiting to SSH to VM ctest-vm2-71417537, IP 110.245.112.3, Port 22 2026-05-25 04:37:32,958 - DEBUG - VM ctest-vm2-71417537 is ready for SSH connections 2026-05-25 04:37:33,080 - DEBUG - VM is in ACTIVE state now 2026-05-25 04:37:33,080 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2026-05-25 04:37:33,081 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine/4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa 2026-05-25 04:37:33,093 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine-interface/ea78889e-8114-457a-b7d4-fdbe94d0ecd1 2026-05-25 04:37:33,111 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine/4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa 2026-05-25 04:37:33,126 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine-interface/ea78889e-8114-457a-b7d4-fdbe94d0ecd1 2026-05-25 04:37:33,142 - DEBUG - Requesting: http://10.0.0.131:8082/instance-ip/a8212848-097e-49f0-893b-dd8c8627da1b 2026-05-25 04:37:33,167 - DEBUG - Verifying in api server 10.0.0.131 2026-05-25 04:37:33,167 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine/4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa 2026-05-25 04:37:33,184 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine-interface/ea78889e-8114-457a-b7d4-fdbe94d0ecd1 2026-05-25 04:37:33,198 - DEBUG - Requesting: http://10.0.0.131:8082/instance-ip/a8212848-097e-49f0-893b-dd8c8627da1b 2026-05-25 04:37:33,209 - INFO - VM ctest-vm1-07876244 verfication in all API Servers passed 2026-05-25 04:37:33,235 - DEBUG - VM ctest-vm1-07876244 Tap interface: {'index': '3', 'name': 'tapea78889e-81', 'uuid': 'ea78889e-8114-457a-b7d4-fdbe94d0ecd1', 'vrf_name': 'default-domain:ctest-TestDropStats-44123704:ctest-vn-50854477:ctest-vn-50854477', 'active': 'Active', 'ipv4_active': 'Active', 'l2_active': 'L2 Active', 'ip6_active': 'Ipv6 Inactive < no-ipv6-addr >', 'health_check_active': 'Active', 'dhcp_service': 'Enable', 'dhcp_service_v6': 'Disable', 'dns_service': 'Enable', 'type': 'vport', 'label': '30', 'l2_label': '42', 'vxlan_id': '5', 'vn_name': 'default-domain:ctest-TestDropStats-44123704:ctest-vn-50854477', 'vm_uuid': '4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa', 'vm_name': 'ctest-vm1-07876244', 'ip_addr': '132.44.154.131', 'mac_addr': '02:ea:78:88:9e:81', 'policy': 'Enable', 'fip_list': [], 'mdata_ip_addr': '169.254.0.3', 'service_vlan_list': None, 'os_ifindex': '132', 'fabric_port': 'NotFabricPort', 'alloc_linklocal_ip': 'LL-Enable', 'analyzer_name': None, 'config_name': 'default-domain:ctest-TestDropStats-44123704:ea78889e-8114-457a-b7d4-fdbe94d0ecd1', 'sg_uuid_list': ['b95e0bbe-06ff-4b7c-a2cb-8d5a71c01686'], 'static_route_list': None, 'vm_project_uuid': '6428eabb-35f4-4ec6-9119-26db8da706db', 'admin_state': 'Enabled', 'flow_key_idx': '28', 'allowed_address_pair_list': None, 'ip6_addr': '::', 'local_preference': '0', 'tx_vlan_id': '-1', 'rx_vlan_id': '-1', 'parent_interface': None, 'subnet': '--NA--', 'sub_type': 'Tap', 'vrf_assign_acl_uuid': '--NA--', 'vmi_type': 'Virtual Machine', 'transport': 'Ethernet', 'logical_interface_uuid': '00000000-0000-0000-0000-000000000000', 'flood_unknown_unicast': 'false', 'physical_device': None, 'physical_interface': None, 'fixed_ip4_list': None, 'fixed_ip6_list': None, 'fat_flow_list': None, 'metadata_ip_active': 'Active', 'service_health_check_ip': '0.0.0.0', 'alias_ip_list': None, 'drop_new_flows': 'false', 'bridge_domain_list': [], 'vmi_tag_list': None, 'policy_set_acl_list': None, 'slo_list': None, 'vhostuser_mode': '0', 'si_other_end_vmi': '00000000-0000-0000-0000-000000000000', 'cfg_igmp_enable': 'false', 'igmp_enabled': 'false', 'max_flows': '0', 'policy_set_fwaas_list': None, 'bond_interface_list': None, 'mac_ip_learning_enable': 'false', 'mac_ip_list': []} 2026-05-25 04:37:33,235 - DEBUG - Agent 10.0.0.133 vrf name: default-domain:ctest-TestDropStats-44123704:ctest-vn-50854477:ctest-vn-50854477 2026-05-25 04:37:33,241 - DEBUG - Agent VRF Object : {'vrf_list': [{'name': 'default-domain:ctest-TestDropStats-44123704:ctest-vn-50854477:ctest-vn-50854477', 'ucindex': '3', 'mcindex': '3', 'l2index': '3', 'source': 'Config; ', 'uc6index': '3', 'vn': 'default-domain:ctest-TestDropStats-44123704:ctest-vn-50854477', 'table_label': '-1', 'vxlan_id': '5', 'evpnindex': '3', 'brindex': '3', 'mplsindex': '0', 'RD': '10.20.0.242:3', 'mac_aging_time': '0', 'layer2_control_word': 'false', 'forwarding_vrf': None, 'hbf_rintf': '-1', 'hbf_lintf': '-1'}]} 2026-05-25 04:37:33,327 - DEBUG - VM ctest-vm1-07876244 : Tap interface tapea78889e-81 is set to Active 2026-05-25 04:37:33,328 - DEBUG - Tap interface tapea78889e-81 detail : {'index': '3', 'name': 'tapea78889e-81', 'uuid': 'ea78889e-8114-457a-b7d4-fdbe94d0ecd1', 'vrf_name': 'default-domain:ctest-TestDropStats-44123704:ctest-vn-50854477:ctest-vn-50854477', 'active': 'Active', 'ipv4_active': 'Active', 'l2_active': 'L2 Active', 'ip6_active': 'Ipv6 Inactive < no-ipv6-addr >', 'health_check_active': 'Active', 'dhcp_service': 'Enable', 'dhcp_service_v6': 'Disable', 'dns_service': 'Enable', 'type': 'vport', 'label': '30', 'l2_label': '42', 'vxlan_id': '5', 'vn_name': 'default-domain:ctest-TestDropStats-44123704:ctest-vn-50854477', 'vm_uuid': '4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa', 'vm_name': 'ctest-vm1-07876244', 'ip_addr': '132.44.154.131', 'mac_addr': '02:ea:78:88:9e:81', 'policy': 'Enable', 'fip_list': [], 'mdata_ip_addr': '169.254.0.3', 'service_vlan_list': None, 'os_ifindex': '132', 'fabric_port': 'NotFabricPort', 'alloc_linklocal_ip': 'LL-Enable', 'analyzer_name': None, 'config_name': 'default-domain:ctest-TestDropStats-44123704:ea78889e-8114-457a-b7d4-fdbe94d0ecd1', 'sg_uuid_list': ['b95e0bbe-06ff-4b7c-a2cb-8d5a71c01686'], 'static_route_list': None, 'vm_project_uuid': '6428eabb-35f4-4ec6-9119-26db8da706db', 'admin_state': 'Enabled', 'flow_key_idx': '28', 'allowed_address_pair_list': None, 'ip6_addr': '::', 'local_preference': '0', 'tx_vlan_id': '-1', 'rx_vlan_id': '-1', 'parent_interface': None, 'subnet': '--NA--', 'sub_type': 'Tap', 'vrf_assign_acl_uuid': '--NA--', 'vmi_type': 'Virtual Machine', 'transport': 'Ethernet', 'logical_interface_uuid': '00000000-0000-0000-0000-000000000000', 'flood_unknown_unicast': 'false', 'physical_device': None, 'physical_interface': None, 'fixed_ip4_list': None, 'fixed_ip6_list': None, 'fat_flow_list': None, 'metadata_ip_active': 'Active', 'service_health_check_ip': '0.0.0.0', 'alias_ip_list': None, 'drop_new_flows': 'false', 'bridge_domain_list': [], 'vmi_tag_list': None, 'policy_set_acl_list': None, 'slo_list': None, 'vhostuser_mode': '0', 'si_other_end_vmi': '00000000-0000-0000-0000-000000000000', 'cfg_igmp_enable': 'false', 'igmp_enabled': 'false', 'max_flows': '0', 'policy_set_fwaas_list': None, 'bond_interface_list': None, 'mac_ip_learning_enable': 'false', 'mac_ip_list': []} 2026-05-25 04:37:33,450 - DEBUG - Starting Layer 2 verification in Agent 2026-05-25 04:37:33,468 - DEBUG - Layer 2 path is seen for VM MAC 02:ea:78:88:9e:81 in agent 10.0.0.133 2026-05-25 04:37:33,468 - DEBUG - Active layer 2 route in agent is present for VMI tapea78889e-81 2026-05-25 04:37:33,468 - DEBUG - L2 label(42) matches bw route table and interface table 2026-05-25 04:37:33,477 - DEBUG - VN default-domain:ctest-TestDropStats-44123704:ctest-vn-50854477 is found in Agent of node 10.0.0.133 2026-05-25 04:37:33,611 - DEBUG - VRF IDs of VN default-domain:ctest-TestDropStats-44123704:ctest-vn-50854477 is consistent in agent 10.0.0.133 2026-05-25 04:37:33,611 - DEBUG - Route for VM IP ['132.44.154.131'] is consistent in agent 10.0.0.133 2026-05-25 04:37:33,611 - DEBUG - VN default-domain:ctest-TestDropStats-44123704:ctest-vn-50854477 verification for VM ctest-vm1-07876244 in Agent 10.0.0.133 passed 2026-05-25 04:37:33,730 - DEBUG - Starting all layer 2 verification in agent 10.0.0.133 2026-05-25 04:37:33,746 - DEBUG - Route for VM MAC 02:ea:78:88:9e:81 is consistent in agent 10.0.0.133 2026-05-25 04:37:33,754 - INFO - VM ctest-vm1-07876244 verifications in Compute nodes passed 2026-05-25 04:37:34,016 - DEBUG - Validated VM route 132.44.154.131 in vrouter of 10.0.0.133 2026-05-25 04:37:34,024 - DEBUG - Remote Nh validation passed. Expected Tunnel, label: None, 0; Got None,0 2026-05-25 04:37:34,024 - INFO - Validated routes of VM ctest-vm1-07876244 in all vrouters 2026-05-25 04:37:34,586 - DEBUG - Starting all layer2 verification in 10.0.0.19 Control Node 2026-05-25 04:37:34,725 - DEBUG - Layer2 route found for VM MAC 02:ea:78:88:9e:81 in Control-node 10.0.0.19 2026-05-25 04:37:34,725 - DEBUG - L2 Label for VM ctest-vm1-07876244 same between Control-node 10.0.0.19 and Agent, Expected: 42, Seen: 42 2026-05-25 04:37:34,725 - DEBUG - Starting all layer2 verification in 10.0.0.73 Control Node 2026-05-25 04:37:34,860 - DEBUG - Layer2 route found for VM MAC 02:ea:78:88:9e:81 in Control-node 10.0.0.73 2026-05-25 04:37:34,860 - DEBUG - L2 Label for VM ctest-vm1-07876244 same between Control-node 10.0.0.73 and Agent, Expected: 42, Seen: 42 2026-05-25 04:37:34,860 - INFO - Verification in Control-nodes for VM ctest-vm1-07876244 passed 2026-05-25 04:37:34,860 - DEBUG - Verifying the vm in opserver 2026-05-25 04:37:34,861 - DEBUG - Verifying in collector 10.0.0.131 ... 2026-05-25 04:37:34,861 - DEBUG - Requesting: http://10.0.0.131:8081/analytics/uves/virtual-machine/4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa?flat 2026-05-25 04:37:35,185 - DEBUG - Requesting: http://10.0.0.131:8081/analytics/uves/virtual-machine/4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa?flat 2026-05-25 04:37:35,283 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/virtual-machine-interface/default-domain:ctest-TestDropStats-44123704:ea78889e-8114-457a-b7d4-fdbe94d0ecd1?flat 2026-05-25 04:37:35,622 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/virtual-machine-interface/default-domain:ctest-TestDropStats-44123704:ea78889e-8114-457a-b7d4-fdbe94d0ecd1?flat 2026-05-25 04:37:35,721 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/virtual-machine-interface/default-domain:ctest-TestDropStats-44123704:ea78889e-8114-457a-b7d4-fdbe94d0ecd1?flat 2026-05-25 04:37:35,802 - DEBUG - Requesting: http://10.0.0.131:8081/analytics/uves/virtual-machine/4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa?flat 2026-05-25 04:37:35,881 - DEBUG - Verifying vm in vn uve 2026-05-25 04:37:35,881 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/virtual-machine-interface/default-domain:ctest-TestDropStats-44123704:ea78889e-8114-457a-b7d4-fdbe94d0ecd1?flat 2026-05-25 04:37:35,979 - DEBUG - VM uve shows interface as {'__T': 1779683843081368, 'ip6_active': False, 'port_mirror_enabled': False, 'label': 30, 'rx_vlan': 65535, 'l2_active': True, 'ip4_active': True, 'active': True, 'ip_address': '132.44.154.131', 'vm_uuid': '4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa', 'tx_vlan': 65535, 'mac_address': '02:ea:78:88:9e:81', 'fixed_ip4_list': ['132.44.154.131'], 'uuid': 'ea78889e-8114-457a-b7d4-fdbe94d0ecd1', 'virtual_network': 'default-domain:ctest-TestDropStats-44123704:ctest-vn-50854477', 'gateway': '132.44.154.129', 'vm_name': 'ctest-vm1-07876244', 'ip6_address': '::', 'is_health_check_active': True, 'vhostuser_mode': 0, 'vn_uuid': 'ab98cad2-0770-4438-9e0d-e203391da22f', 'admin_state': True} 2026-05-25 04:37:35,979 - DEBUG - VM uve shows ip address as ['132.44.154.131', '::'] 2026-05-25 04:37:35,979 - DEBUG - VM uve shows virtual network as default-domain:ctest-TestDropStats-44123704:ctest-vn-50854477 2026-05-25 04:37:35,979 - INFO - Verifying through opserver in 10.0.0.73 2026-05-25 04:37:35,979 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/virtual-network/default-domain:ctest-TestDropStats-44123704:ctest-vn-50854477?flat 2026-05-25 04:37:36,009 - DEBUG - expected vm list 4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa 2026-05-25 04:37:36,009 - DEBUG - Extracted vm list ['4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa'] 2026-05-25 04:37:36,009 - INFO - VM 4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa is present in default-domain:ctest-TestDropStats-44123704:ctest-vn-50854477 2026-05-25 04:37:36,009 - DEBUG - Verifying vm in vrouter uve 2026-05-25 04:37:36,009 - DEBUG - Getting info from collector 10.0.0.131.. 2026-05-25 04:37:36,009 - DEBUG - Requesting: http://10.0.0.131:8081/analytics/uves/virtual-machine/4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa?flat 2026-05-25 04:37:36,110 - DEBUG - VM uve shows vrouter as ['an-jenkins-deploy-platform-ansible-os-5822-1'] 2026-05-25 04:37:36,110 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-5822-1:Compute:contrail-vrouter-agent:0?flat 2026-05-25 04:37:36,173 - DEBUG - Requesting: http://10.0.0.131:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-5822-1?flat 2026-05-25 04:37:36,212 - DEBUG - VM 4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa is present in vrouter an-jenkins-deploy-platform-ansible-os-5822-1 uve 2026-05-25 04:37:36,212 - DEBUG - Validated that VM ctest-vm1-07876244 is in Vrouter an-jenkins-deploy-platform-ansible-os-5822-1 UVE 2026-05-25 04:37:36,212 - DEBUG - Verifying vm tap interface/vn in vrouter uve 2026-05-25 04:37:36,223 - DEBUG - Expected tap interface of VM uuid 4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa is default-domain:ctest-TestDropStats-44123704:ea78889e-8114-457a-b7d4-fdbe94d0ecd1 2026-05-25 04:37:36,223 - DEBUG - Expected VN of VM uuid 4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa is default-domain:ctest-TestDropStats-44123704:ctest-vn-50854477 2026-05-25 04:37:36,223 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-5822-1:Compute:contrail-vrouter-agent:0?flat 2026-05-25 04:37:36,273 - DEBUG - Requesting: http://10.0.0.131:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-5822-1?flat 2026-05-25 04:37:36,311 - DEBUG - VM 4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa is present in vrouter an-jenkins-deploy-platform-ansible-os-5822-1 uve 2026-05-25 04:37:36,311 - INFO - tap interface default-domain:ctest-TestDropStats-44123704:ea78889e-8114-457a-b7d4-fdbe94d0ecd1 of vm 4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa is present in vrouter an-jenkins-deploy-platform-ansible-os-5822-1 uve 2026-05-25 04:37:36,311 - DEBUG - Connected network default-domain:ctest-TestDropStats-44123704:ctest-vn-50854477 of vm 4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa is present in vrouter an-jenkins-deploy-platform-ansible-os-5822-1 uve 2026-05-25 04:37:36,311 - DEBUG - Verifying the 4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa virtual network link through opserver 10.0.0.73 2026-05-25 04:37:36,311 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/virtual-machines 2026-05-25 04:37:36,327 - DEBUG - VM link and name as {'name': '4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa', 'href': 'http://10.0.0.73:8081/analytics/uves/virtual-machine/4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa?flat'} 2026-05-25 04:37:36,327 - DEBUG - Verifying the 4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa virtual network link through opserver 10.0.0.19 2026-05-25 04:37:36,327 - DEBUG - Requesting: http://10.0.0.19:8081/analytics/uves/virtual-machines 2026-05-25 04:37:36,656 - DEBUG - Requesting: http://10.0.0.19:8081/analytics/uves/virtual-machines 2026-05-25 04:37:36,792 - DEBUG - VM link and name as {'name': '4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa', 'href': 'http://10.0.0.19:8081/analytics/uves/virtual-machine/4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa?flat'} 2026-05-25 04:37:36,792 - DEBUG - Verifying the 4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa virtual network link through opserver 10.0.0.131 2026-05-25 04:37:36,792 - DEBUG - Requesting: http://10.0.0.131:8081/analytics/uves/virtual-machines 2026-05-25 04:37:36,808 - DEBUG - VM link and name as {'name': '4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa', 'href': 'http://10.0.0.131:8081/analytics/uves/virtual-machine/4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa?flat'} 2026-05-25 04:37:36,808 - INFO - VM ctest-vm1-07876244 validations in Opserver passed 2026-05-25 04:37:36,921 - DEBUG - VM is in ACTIVE state now 2026-05-25 04:37:36,921 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2026-05-25 04:37:36,921 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine/2c7f212d-5279-4a60-bf61-d32430eb6a95 2026-05-25 04:37:36,931 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine-interface/a7d08547-c982-424c-8d47-a03bf0fbd367 2026-05-25 04:37:36,943 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine/2c7f212d-5279-4a60-bf61-d32430eb6a95 2026-05-25 04:37:36,960 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine-interface/a7d08547-c982-424c-8d47-a03bf0fbd367 2026-05-25 04:37:36,976 - DEBUG - Requesting: http://10.0.0.131:8082/instance-ip/18471597-af0d-471b-a1e9-faef7256b51f 2026-05-25 04:37:36,990 - DEBUG - Verifying in api server 10.0.0.131 2026-05-25 04:37:36,990 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine/2c7f212d-5279-4a60-bf61-d32430eb6a95 2026-05-25 04:37:36,999 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine-interface/a7d08547-c982-424c-8d47-a03bf0fbd367 2026-05-25 04:37:37,011 - DEBUG - Requesting: http://10.0.0.131:8082/instance-ip/18471597-af0d-471b-a1e9-faef7256b51f 2026-05-25 04:37:37,027 - INFO - VM ctest-vm2-71417537 verfication in all API Servers passed 2026-05-25 04:37:37,049 - DEBUG - VM ctest-vm2-71417537 Tap interface: {'index': '5', 'name': 'tapa7d08547-c9', 'uuid': 'a7d08547-c982-424c-8d47-a03bf0fbd367', 'vrf_name': 'default-domain:ctest-TestDropStats-44123704:ctest-vn-29452387:ctest-vn-29452387', 'active': 'Active', 'ipv4_active': 'Active', 'l2_active': 'L2 Active', 'ip6_active': 'Ipv6 Inactive < no-ipv6-addr >', 'health_check_active': 'Active', 'dhcp_service': 'Enable', 'dhcp_service_v6': 'Disable', 'dns_service': 'Enable', 'type': 'vport', 'label': '24', 'l2_label': '29', 'vxlan_id': '6', 'vn_name': 'default-domain:ctest-TestDropStats-44123704:ctest-vn-29452387', 'vm_uuid': '2c7f212d-5279-4a60-bf61-d32430eb6a95', 'vm_name': 'ctest-vm2-71417537', 'ip_addr': '110.245.112.3', 'mac_addr': '02:a7:d0:85:47:c9', 'policy': 'Enable', 'fip_list': [], 'mdata_ip_addr': '169.254.0.5', 'service_vlan_list': None, 'os_ifindex': '166', 'fabric_port': 'NotFabricPort', 'alloc_linklocal_ip': 'LL-Enable', 'analyzer_name': None, 'config_name': 'default-domain:ctest-TestDropStats-44123704:a7d08547-c982-424c-8d47-a03bf0fbd367', 'sg_uuid_list': ['b95e0bbe-06ff-4b7c-a2cb-8d5a71c01686'], 'static_route_list': None, 'vm_project_uuid': '6428eabb-35f4-4ec6-9119-26db8da706db', 'admin_state': 'Enabled', 'flow_key_idx': '27', 'allowed_address_pair_list': None, 'ip6_addr': '::', 'local_preference': '0', 'tx_vlan_id': '-1', 'rx_vlan_id': '-1', 'parent_interface': None, 'subnet': '--NA--', 'sub_type': 'Tap', 'vrf_assign_acl_uuid': '--NA--', 'vmi_type': 'Virtual Machine', 'transport': 'Ethernet', 'logical_interface_uuid': '00000000-0000-0000-0000-000000000000', 'flood_unknown_unicast': 'false', 'physical_device': None, 'physical_interface': None, 'fixed_ip4_list': None, 'fixed_ip6_list': None, 'fat_flow_list': None, 'metadata_ip_active': 'Active', 'service_health_check_ip': '0.0.0.0', 'alias_ip_list': None, 'drop_new_flows': 'false', 'bridge_domain_list': [], 'vmi_tag_list': None, 'policy_set_acl_list': None, 'slo_list': None, 'vhostuser_mode': '0', 'si_other_end_vmi': '00000000-0000-0000-0000-000000000000', 'cfg_igmp_enable': 'false', 'igmp_enabled': 'false', 'max_flows': '0', 'policy_set_fwaas_list': None, 'bond_interface_list': None, 'mac_ip_learning_enable': 'false', 'mac_ip_list': []} 2026-05-25 04:37:37,050 - DEBUG - Agent 10.0.0.240 vrf name: default-domain:ctest-TestDropStats-44123704:ctest-vn-29452387:ctest-vn-29452387 2026-05-25 04:37:37,057 - DEBUG - Agent VRF Object : {'vrf_list': [{'name': 'default-domain:ctest-TestDropStats-44123704:ctest-vn-29452387:ctest-vn-29452387', 'ucindex': '2', 'mcindex': '2', 'l2index': '2', 'source': 'Config; ', 'uc6index': '2', 'vn': 'default-domain:ctest-TestDropStats-44123704:ctest-vn-29452387', 'table_label': '-1', 'vxlan_id': '6', 'evpnindex': '2', 'brindex': '2', 'mplsindex': '0', 'RD': '10.20.0.19:2', 'mac_aging_time': '0', 'layer2_control_word': 'false', 'forwarding_vrf': None, 'hbf_rintf': '-1', 'hbf_lintf': '-1'}]} 2026-05-25 04:37:37,147 - DEBUG - VM ctest-vm2-71417537 : Tap interface tapa7d08547-c9 is set to Active 2026-05-25 04:37:37,147 - DEBUG - Tap interface tapa7d08547-c9 detail : {'index': '5', 'name': 'tapa7d08547-c9', 'uuid': 'a7d08547-c982-424c-8d47-a03bf0fbd367', 'vrf_name': 'default-domain:ctest-TestDropStats-44123704:ctest-vn-29452387:ctest-vn-29452387', 'active': 'Active', 'ipv4_active': 'Active', 'l2_active': 'L2 Active', 'ip6_active': 'Ipv6 Inactive < no-ipv6-addr >', 'health_check_active': 'Active', 'dhcp_service': 'Enable', 'dhcp_service_v6': 'Disable', 'dns_service': 'Enable', 'type': 'vport', 'label': '24', 'l2_label': '29', 'vxlan_id': '6', 'vn_name': 'default-domain:ctest-TestDropStats-44123704:ctest-vn-29452387', 'vm_uuid': '2c7f212d-5279-4a60-bf61-d32430eb6a95', 'vm_name': 'ctest-vm2-71417537', 'ip_addr': '110.245.112.3', 'mac_addr': '02:a7:d0:85:47:c9', 'policy': 'Enable', 'fip_list': [], 'mdata_ip_addr': '169.254.0.5', 'service_vlan_list': None, 'os_ifindex': '166', 'fabric_port': 'NotFabricPort', 'alloc_linklocal_ip': 'LL-Enable', 'analyzer_name': None, 'config_name': 'default-domain:ctest-TestDropStats-44123704:a7d08547-c982-424c-8d47-a03bf0fbd367', 'sg_uuid_list': ['b95e0bbe-06ff-4b7c-a2cb-8d5a71c01686'], 'static_route_list': None, 'vm_project_uuid': '6428eabb-35f4-4ec6-9119-26db8da706db', 'admin_state': 'Enabled', 'flow_key_idx': '27', 'allowed_address_pair_list': None, 'ip6_addr': '::', 'local_preference': '0', 'tx_vlan_id': '-1', 'rx_vlan_id': '-1', 'parent_interface': None, 'subnet': '--NA--', 'sub_type': 'Tap', 'vrf_assign_acl_uuid': '--NA--', 'vmi_type': 'Virtual Machine', 'transport': 'Ethernet', 'logical_interface_uuid': '00000000-0000-0000-0000-000000000000', 'flood_unknown_unicast': 'false', 'physical_device': None, 'physical_interface': None, 'fixed_ip4_list': None, 'fixed_ip6_list': None, 'fat_flow_list': None, 'metadata_ip_active': 'Active', 'service_health_check_ip': '0.0.0.0', 'alias_ip_list': None, 'drop_new_flows': 'false', 'bridge_domain_list': [], 'vmi_tag_list': None, 'policy_set_acl_list': None, 'slo_list': None, 'vhostuser_mode': '0', 'si_other_end_vmi': '00000000-0000-0000-0000-000000000000', 'cfg_igmp_enable': 'false', 'igmp_enabled': 'false', 'max_flows': '0', 'policy_set_fwaas_list': None, 'bond_interface_list': None, 'mac_ip_learning_enable': 'false', 'mac_ip_list': []} 2026-05-25 04:37:37,266 - DEBUG - Starting Layer 2 verification in Agent 2026-05-25 04:37:37,278 - DEBUG - Layer 2 path is seen for VM MAC 02:a7:d0:85:47:c9 in agent 10.0.0.240 2026-05-25 04:37:37,278 - DEBUG - Active layer 2 route in agent is present for VMI tapa7d08547-c9 2026-05-25 04:37:37,278 - DEBUG - L2 label(29) matches bw route table and interface table 2026-05-25 04:37:37,291 - DEBUG - VN default-domain:ctest-TestDropStats-44123704:ctest-vn-29452387 is found in Agent of node 10.0.0.240 2026-05-25 04:37:37,431 - DEBUG - VRF IDs of VN default-domain:ctest-TestDropStats-44123704:ctest-vn-29452387 is consistent in agent 10.0.0.240 2026-05-25 04:37:37,431 - DEBUG - Route for VM IP ['110.245.112.3'] is consistent in agent 10.0.0.240 2026-05-25 04:37:37,431 - DEBUG - VN default-domain:ctest-TestDropStats-44123704:ctest-vn-29452387 verification for VM ctest-vm2-71417537 in Agent 10.0.0.240 passed 2026-05-25 04:37:37,556 - DEBUG - Starting all layer 2 verification in agent 10.0.0.240 2026-05-25 04:37:37,569 - DEBUG - Route for VM MAC 02:a7:d0:85:47:c9 is consistent in agent 10.0.0.240 2026-05-25 04:37:37,569 - INFO - VM ctest-vm2-71417537 verifications in Compute nodes passed 2026-05-25 04:37:37,822 - DEBUG - Validated VM route 110.245.112.3 in vrouter of 10.0.0.240 2026-05-25 04:37:37,832 - DEBUG - Remote Nh validation passed. Expected Tunnel, label: None, 0; Got None,0 2026-05-25 04:37:37,832 - INFO - Validated routes of VM ctest-vm2-71417537 in all vrouters 2026-05-25 04:37:38,378 - DEBUG - Starting all layer2 verification in 10.0.0.131 Control Node 2026-05-25 04:37:38,515 - DEBUG - Layer2 route found for VM MAC 02:a7:d0:85:47:c9 in Control-node 10.0.0.131 2026-05-25 04:37:38,515 - DEBUG - L2 Label for VM ctest-vm2-71417537 same between Control-node 10.0.0.131 and Agent, Expected: 29, Seen: 29 2026-05-25 04:37:38,515 - DEBUG - Starting all layer2 verification in 10.0.0.73 Control Node 2026-05-25 04:37:38,645 - DEBUG - Layer2 route found for VM MAC 02:a7:d0:85:47:c9 in Control-node 10.0.0.73 2026-05-25 04:37:38,645 - DEBUG - L2 Label for VM ctest-vm2-71417537 same between Control-node 10.0.0.73 and Agent, Expected: 29, Seen: 29 2026-05-25 04:37:38,645 - INFO - Verification in Control-nodes for VM ctest-vm2-71417537 passed 2026-05-25 04:37:38,646 - DEBUG - Verifying the vm in opserver 2026-05-25 04:37:38,646 - DEBUG - Verifying in collector 10.0.0.131 ... 2026-05-25 04:37:38,646 - DEBUG - Requesting: http://10.0.0.131:8081/analytics/uves/virtual-machine/2c7f212d-5279-4a60-bf61-d32430eb6a95?flat 2026-05-25 04:37:38,667 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/virtual-machine-interface/default-domain:ctest-TestDropStats-44123704:a7d08547-c982-424c-8d47-a03bf0fbd367?flat 2026-05-25 04:37:38,715 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/virtual-machine-interface/default-domain:ctest-TestDropStats-44123704:a7d08547-c982-424c-8d47-a03bf0fbd367?flat 2026-05-25 04:37:38,744 - DEBUG - Requesting: http://10.0.0.131:8081/analytics/uves/virtual-machine/2c7f212d-5279-4a60-bf61-d32430eb6a95?flat 2026-05-25 04:37:38,763 - DEBUG - Verifying vm in vn uve 2026-05-25 04:37:38,763 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/virtual-machine-interface/default-domain:ctest-TestDropStats-44123704:a7d08547-c982-424c-8d47-a03bf0fbd367?flat 2026-05-25 04:37:38,794 - DEBUG - VM uve shows interface as {'__T': 1779683851567749, 'ip6_active': False, 'port_mirror_enabled': False, 'label': 24, 'rx_vlan': 65535, 'l2_active': True, 'ip4_active': True, 'active': True, 'ip_address': '110.245.112.3', 'vm_uuid': '2c7f212d-5279-4a60-bf61-d32430eb6a95', 'tx_vlan': 65535, 'mac_address': '02:a7:d0:85:47:c9', 'fixed_ip4_list': ['110.245.112.3'], 'uuid': 'a7d08547-c982-424c-8d47-a03bf0fbd367', 'virtual_network': 'default-domain:ctest-TestDropStats-44123704:ctest-vn-29452387', 'gateway': '110.245.112.1', 'vm_name': 'ctest-vm2-71417537', 'ip6_address': '::', 'is_health_check_active': True, 'vhostuser_mode': 0, 'vn_uuid': 'a2e9fca9-0d5c-4348-9022-927fc7a3a978', 'admin_state': True} 2026-05-25 04:37:38,794 - DEBUG - VM uve shows ip address as ['110.245.112.3', '::'] 2026-05-25 04:37:38,794 - DEBUG - VM uve shows virtual network as default-domain:ctest-TestDropStats-44123704:ctest-vn-29452387 2026-05-25 04:37:38,794 - INFO - Verifying through opserver in 10.0.0.73 2026-05-25 04:37:38,794 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/virtual-network/default-domain:ctest-TestDropStats-44123704:ctest-vn-29452387?flat 2026-05-25 04:37:38,828 - DEBUG - expected vm list 2c7f212d-5279-4a60-bf61-d32430eb6a95 2026-05-25 04:37:38,829 - DEBUG - Extracted vm list ['2c7f212d-5279-4a60-bf61-d32430eb6a95'] 2026-05-25 04:37:38,829 - INFO - VM 2c7f212d-5279-4a60-bf61-d32430eb6a95 is present in default-domain:ctest-TestDropStats-44123704:ctest-vn-29452387 2026-05-25 04:37:38,829 - DEBUG - Verifying vm in vrouter uve 2026-05-25 04:37:38,829 - DEBUG - Getting info from collector 10.0.0.131.. 2026-05-25 04:37:38,829 - DEBUG - Requesting: http://10.0.0.131:8081/analytics/uves/virtual-machine/2c7f212d-5279-4a60-bf61-d32430eb6a95?flat 2026-05-25 04:37:38,855 - DEBUG - VM uve shows vrouter as ['an-jenkins-deploy-platform-ansible-os-5822-2'] 2026-05-25 04:37:38,855 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-5822-2:Compute:contrail-vrouter-agent:0?flat 2026-05-25 04:37:38,910 - DEBUG - Requesting: http://10.0.0.131:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-5822-2?flat 2026-05-25 04:37:38,944 - DEBUG - VM 2c7f212d-5279-4a60-bf61-d32430eb6a95 is present in vrouter an-jenkins-deploy-platform-ansible-os-5822-2 uve 2026-05-25 04:37:38,945 - DEBUG - Validated that VM ctest-vm2-71417537 is in Vrouter an-jenkins-deploy-platform-ansible-os-5822-2 UVE 2026-05-25 04:37:38,945 - DEBUG - Verifying vm tap interface/vn in vrouter uve 2026-05-25 04:37:38,953 - DEBUG - Expected tap interface of VM uuid 2c7f212d-5279-4a60-bf61-d32430eb6a95 is default-domain:ctest-TestDropStats-44123704:a7d08547-c982-424c-8d47-a03bf0fbd367 2026-05-25 04:37:38,953 - DEBUG - Expected VN of VM uuid 2c7f212d-5279-4a60-bf61-d32430eb6a95 is default-domain:ctest-TestDropStats-44123704:ctest-vn-29452387 2026-05-25 04:37:38,953 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-5822-2:Compute:contrail-vrouter-agent:0?flat 2026-05-25 04:37:39,006 - DEBUG - Requesting: http://10.0.0.131:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-5822-2?flat 2026-05-25 04:37:39,039 - DEBUG - VM 2c7f212d-5279-4a60-bf61-d32430eb6a95 is present in vrouter an-jenkins-deploy-platform-ansible-os-5822-2 uve 2026-05-25 04:37:39,039 - INFO - tap interface default-domain:ctest-TestDropStats-44123704:a7d08547-c982-424c-8d47-a03bf0fbd367 of vm 2c7f212d-5279-4a60-bf61-d32430eb6a95 is present in vrouter an-jenkins-deploy-platform-ansible-os-5822-2 uve 2026-05-25 04:37:39,039 - DEBUG - Connected network default-domain:ctest-TestDropStats-44123704:ctest-vn-29452387 of vm 2c7f212d-5279-4a60-bf61-d32430eb6a95 is present in vrouter an-jenkins-deploy-platform-ansible-os-5822-2 uve 2026-05-25 04:37:39,039 - DEBUG - Verifying the 2c7f212d-5279-4a60-bf61-d32430eb6a95 virtual network link through opserver 10.0.0.73 2026-05-25 04:37:39,039 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/virtual-machines 2026-05-25 04:37:39,056 - DEBUG - VM link and name as {'name': '2c7f212d-5279-4a60-bf61-d32430eb6a95', 'href': 'http://10.0.0.73:8081/analytics/uves/virtual-machine/2c7f212d-5279-4a60-bf61-d32430eb6a95?flat'} 2026-05-25 04:37:39,056 - DEBUG - Verifying the 2c7f212d-5279-4a60-bf61-d32430eb6a95 virtual network link through opserver 10.0.0.19 2026-05-25 04:37:39,056 - DEBUG - Requesting: http://10.0.0.19:8081/analytics/uves/virtual-machines 2026-05-25 04:37:39,169 - DEBUG - VM link and name as {'name': '2c7f212d-5279-4a60-bf61-d32430eb6a95', 'href': 'http://10.0.0.19:8081/analytics/uves/virtual-machine/2c7f212d-5279-4a60-bf61-d32430eb6a95?flat'} 2026-05-25 04:37:39,169 - DEBUG - Verifying the 2c7f212d-5279-4a60-bf61-d32430eb6a95 virtual network link through opserver 10.0.0.131 2026-05-25 04:37:39,169 - DEBUG - Requesting: http://10.0.0.131:8081/analytics/uves/virtual-machines 2026-05-25 04:37:39,185 - DEBUG - VM link and name as {'name': '2c7f212d-5279-4a60-bf61-d32430eb6a95', 'href': 'http://10.0.0.131:8081/analytics/uves/virtual-machine/2c7f212d-5279-4a60-bf61-d32430eb6a95?flat'} 2026-05-25 04:37:39,185 - INFO - VM ctest-vm2-71417537 validations in Opserver passed 2026-05-25 04:37:39,212 - INFO - ---------------------------------------------------------------------------------------------------- 2026-05-25 04:37:39,212 - INFO - ds_discard : 0 2026-05-25 04:37:39,213 - INFO - ds_pull : 0 2026-05-25 04:37:39,213 - INFO - ds_invalid_if : 0 2026-05-25 04:37:39,213 - INFO - ds_invalid_arp : 0 2026-05-25 04:37:39,213 - INFO - ds_trap_no_if : 0 2026-05-25 04:37:39,213 - INFO - ds_nowhere_to_go : 0 2026-05-25 04:37:39,213 - INFO - ds_flow_queue_limit_exceeded : 0 2026-05-25 04:37:39,213 - INFO - ds_flow_no_memory : 0 2026-05-25 04:37:39,213 - INFO - ds_flow_invalid_protocol : 0 2026-05-25 04:37:39,213 - INFO - ds_flow_nat_no_rflow : 0 2026-05-25 04:37:39,213 - INFO - ds_flow_action_drop : 0 2026-05-25 04:37:39,213 - INFO - ds_flow_action_invalid : 0 2026-05-25 04:37:39,213 - INFO - ds_flow_unusable : 0 2026-05-25 04:37:39,214 - INFO - ds_flow_table_full : 0 2026-05-25 04:37:39,214 - INFO - ds_interface_tx_discard : 0 2026-05-25 04:37:39,214 - INFO - ds_interface_drop : 0 2026-05-25 04:37:39,214 - INFO - ds_duplicated : 0 2026-05-25 04:37:39,214 - INFO - ds_push : 0 2026-05-25 04:37:39,214 - INFO - ds_ttl_exceeded : 0 2026-05-25 04:37:39,214 - INFO - ds_invalid_nh : 0 2026-05-25 04:37:39,214 - INFO - ds_invalid_label : 0 2026-05-25 04:37:39,214 - INFO - ds_invalid_protocol : 0 2026-05-25 04:37:39,214 - INFO - ds_interface_rx_discard : 0 2026-05-25 04:37:39,214 - INFO - ds_invalid_mcast_source : 0 2026-05-25 04:37:39,214 - INFO - ds_head_alloc_fail : 0 2026-05-25 04:37:39,214 - INFO - ds_pcow_fail : 0 2026-05-25 04:37:39,214 - INFO - ds_mcast_clone_fail : 0 2026-05-25 04:37:39,214 - INFO - ds_rewrite_fail : 0 2026-05-25 04:37:39,214 - INFO - ds_misc : 0 2026-05-25 04:37:39,214 - INFO - ds_invalid_packet : 0 2026-05-25 04:37:39,215 - INFO - ds_cksum_err : 0 2026-05-25 04:37:39,215 - INFO - ds_no_fmd : 0 2026-05-25 04:37:39,215 - INFO - ds_invalid_vnid : 0 2026-05-25 04:37:39,215 - INFO - ds_frag_err : 0 2026-05-25 04:37:39,215 - INFO - ds_invalid_source : 0 2026-05-25 04:37:39,215 - INFO - ds_mcast_df_bit : 0 2026-05-25 04:37:39,215 - INFO - ds_l2_no_route : 0 2026-05-25 04:37:39,215 - INFO - ds_vlan_fwd_tx : 0 2026-05-25 04:37:39,215 - INFO - ds_vlan_fwd_enq : 0 2026-05-25 04:37:39,215 - INFO - ds_drop_new_flow : 0 2026-05-25 04:37:39,215 - INFO - ds_flow_evict : 0 2026-05-25 04:37:39,215 - INFO - ds_trap_original : 0 2026-05-25 04:37:39,215 - INFO - ds_no_memory : 0 2026-05-25 04:37:39,215 - INFO - ds_fragment_queue_fail : 0 2026-05-25 04:37:39,215 - INFO - ds_no_frag_entry : 0 2026-05-25 04:37:39,215 - INFO - ds_icmp_error : 0 2026-05-25 04:37:39,215 - INFO - ds_clone_fail : 0 2026-05-25 04:37:39,215 - INFO - ds_drop_pkts : 0 2026-05-25 04:37:39,224 - INFO - ---------------------------------------------------------------------------------------------------- 2026-05-25 04:37:39,225 - INFO - ds_discard : 8 2026-05-25 04:37:39,225 - INFO - ds_pull : 0 2026-05-25 04:37:39,225 - INFO - ds_invalid_if : 0 2026-05-25 04:37:39,225 - INFO - ds_invalid_arp : 0 2026-05-25 04:37:39,225 - INFO - ds_trap_no_if : 0 2026-05-25 04:37:39,225 - INFO - ds_nowhere_to_go : 0 2026-05-25 04:37:39,225 - INFO - ds_flow_queue_limit_exceeded : 0 2026-05-25 04:37:39,225 - INFO - ds_flow_no_memory : 0 2026-05-25 04:37:39,225 - INFO - ds_flow_invalid_protocol : 0 2026-05-25 04:37:39,225 - INFO - ds_flow_nat_no_rflow : 0 2026-05-25 04:37:39,225 - INFO - ds_flow_action_drop : 16 2026-05-25 04:37:39,225 - INFO - ds_flow_action_invalid : 0 2026-05-25 04:37:39,225 - INFO - ds_flow_unusable : 0 2026-05-25 04:37:39,225 - INFO - ds_flow_table_full : 0 2026-05-25 04:37:39,225 - INFO - ds_interface_tx_discard : 0 2026-05-25 04:37:39,226 - INFO - ds_interface_drop : 10 2026-05-25 04:37:39,226 - INFO - ds_duplicated : 1 2026-05-25 04:37:39,226 - INFO - ds_push : 0 2026-05-25 04:37:39,226 - INFO - ds_ttl_exceeded : 0 2026-05-25 04:37:39,226 - INFO - ds_invalid_nh : 4 2026-05-25 04:37:39,226 - INFO - ds_invalid_label : 0 2026-05-25 04:37:39,226 - INFO - ds_invalid_protocol : 0 2026-05-25 04:37:39,226 - INFO - ds_interface_rx_discard : 0 2026-05-25 04:37:39,226 - INFO - ds_invalid_mcast_source : 0 2026-05-25 04:37:39,226 - INFO - ds_head_alloc_fail : 0 2026-05-25 04:37:39,226 - INFO - ds_pcow_fail : 0 2026-05-25 04:37:39,226 - INFO - ds_mcast_clone_fail : 0 2026-05-25 04:37:39,226 - INFO - ds_rewrite_fail : 0 2026-05-25 04:37:39,226 - INFO - ds_misc : 0 2026-05-25 04:37:39,226 - INFO - ds_invalid_packet : 0 2026-05-25 04:37:39,226 - INFO - ds_cksum_err : 0 2026-05-25 04:37:39,226 - INFO - ds_no_fmd : 0 2026-05-25 04:37:39,227 - INFO - ds_invalid_vnid : 0 2026-05-25 04:37:39,227 - INFO - ds_frag_err : 0 2026-05-25 04:37:39,227 - INFO - ds_invalid_source : 0 2026-05-25 04:37:39,227 - INFO - ds_mcast_df_bit : 0 2026-05-25 04:37:39,227 - INFO - ds_l2_no_route : 1 2026-05-25 04:37:39,227 - INFO - ds_vlan_fwd_tx : 0 2026-05-25 04:37:39,227 - INFO - ds_vlan_fwd_enq : 0 2026-05-25 04:37:39,227 - INFO - ds_drop_new_flow : 0 2026-05-25 04:37:39,227 - INFO - ds_flow_evict : 0 2026-05-25 04:37:39,227 - INFO - ds_trap_original : 0 2026-05-25 04:37:39,227 - INFO - ds_no_memory : 0 2026-05-25 04:37:39,227 - INFO - ds_fragment_queue_fail : 0 2026-05-25 04:37:39,227 - INFO - ds_no_frag_entry : 0 2026-05-25 04:37:39,227 - INFO - ds_icmp_error : 0 2026-05-25 04:37:39,227 - INFO - ds_clone_fail : 0 2026-05-25 04:37:39,227 - INFO - ds_drop_pkts : 40 2026-05-25 04:37:39,227 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 5 -W 1 110.245.112.3, host_string: cirros@169.254.0.3, password: cubswin:)gateway: ubuntu@10.0.0.133, gateway password: c0ntrail123 2026-05-25 04:37:39,228 - DEBUG - ping -s 56 -c 5 -W 1 110.245.112.3 2026-05-25 04:37:45,262 - DEBUG - PING 110.245.112.3 (110.245.112.3): 56 data bytes --- 110.245.112.3 ping statistics --- 5 packets transmitted, 0 packets received, 100% packet loss 2026-05-25 04:37:45,262 - WARNING - Ping to IP 110.245.112.3 from VM ctest-vm1-07876244 failed 2026-05-25 04:37:53,356 - INFO - Vif and vrouter dropstats of type ds_flow_action_drop verifed 2026-05-25 04:37:53,356 - INFO - ---------------------------------------------------------------------------------------------------- 2026-05-25 04:37:53,357 - INFO - ds_discard : 3 2026-05-25 04:37:53,357 - INFO - ds_pull : 0 2026-05-25 04:37:53,357 - INFO - ds_invalid_if : 0 2026-05-25 04:37:53,357 - INFO - ds_invalid_arp : 0 2026-05-25 04:37:53,357 - INFO - ds_trap_no_if : 0 2026-05-25 04:37:53,357 - INFO - ds_nowhere_to_go : 0 2026-05-25 04:37:53,357 - INFO - ds_flow_queue_limit_exceeded : 0 2026-05-25 04:37:53,357 - INFO - ds_flow_no_memory : 0 2026-05-25 04:37:53,357 - INFO - ds_flow_invalid_protocol : 0 2026-05-25 04:37:53,357 - INFO - ds_flow_nat_no_rflow : 0 2026-05-25 04:37:53,357 - INFO - ds_flow_action_drop : 5 2026-05-25 04:37:53,357 - INFO - ds_flow_action_invalid : 0 2026-05-25 04:37:53,357 - INFO - ds_flow_unusable : 0 2026-05-25 04:37:53,357 - INFO - ds_flow_table_full : 0 2026-05-25 04:37:53,357 - INFO - ds_interface_tx_discard : 0 2026-05-25 04:37:53,357 - INFO - ds_interface_drop : 0 2026-05-25 04:37:53,357 - INFO - ds_duplicated : 0 2026-05-25 04:37:53,357 - INFO - ds_push : 0 2026-05-25 04:37:53,357 - INFO - ds_ttl_exceeded : 0 2026-05-25 04:37:53,357 - INFO - ds_invalid_nh : 0 2026-05-25 04:37:53,358 - INFO - ds_invalid_label : 0 2026-05-25 04:37:53,358 - INFO - ds_invalid_protocol : 0 2026-05-25 04:37:53,358 - INFO - ds_interface_rx_discard : 0 2026-05-25 04:37:53,358 - INFO - ds_invalid_mcast_source : 0 2026-05-25 04:37:53,358 - INFO - ds_head_alloc_fail : 0 2026-05-25 04:37:53,358 - INFO - ds_pcow_fail : 0 2026-05-25 04:37:53,358 - INFO - ds_mcast_clone_fail : 0 2026-05-25 04:37:53,358 - INFO - ds_rewrite_fail : 0 2026-05-25 04:37:53,358 - INFO - ds_misc : 0 2026-05-25 04:37:53,358 - INFO - ds_invalid_packet : 0 2026-05-25 04:37:53,358 - INFO - ds_cksum_err : 0 2026-05-25 04:37:53,358 - INFO - ds_no_fmd : 0 2026-05-25 04:37:53,358 - INFO - ds_invalid_vnid : 0 2026-05-25 04:37:53,358 - INFO - ds_frag_err : 0 2026-05-25 04:37:53,358 - INFO - ds_invalid_source : 0 2026-05-25 04:37:53,358 - INFO - ds_mcast_df_bit : 0 2026-05-25 04:37:53,358 - INFO - ds_l2_no_route : 0 2026-05-25 04:37:53,358 - INFO - ds_vlan_fwd_tx : 0 2026-05-25 04:37:53,358 - INFO - ds_vlan_fwd_enq : 0 2026-05-25 04:37:53,358 - INFO - ds_drop_new_flow : 0 2026-05-25 04:37:53,358 - INFO - ds_flow_evict : 0 2026-05-25 04:37:53,358 - INFO - ds_trap_original : 0 2026-05-25 04:37:53,358 - INFO - ds_no_memory : 0 2026-05-25 04:37:53,358 - INFO - ds_fragment_queue_fail : 0 2026-05-25 04:37:53,358 - INFO - ds_no_frag_entry : 0 2026-05-25 04:37:53,358 - INFO - ds_icmp_error : 0 2026-05-25 04:37:53,359 - INFO - ds_clone_fail : 0 2026-05-25 04:37:53,359 - INFO - ds_drop_pkts : 8 2026-05-25 04:37:53,359 - INFO - ---------------------------------------------------------------------------------------------------- 2026-05-25 04:37:53,359 - INFO - ds_discard : 11 2026-05-25 04:37:53,359 - INFO - ds_pull : 0 2026-05-25 04:37:53,359 - INFO - ds_invalid_if : 0 2026-05-25 04:37:53,359 - INFO - ds_invalid_arp : 0 2026-05-25 04:37:53,359 - INFO - ds_trap_no_if : 0 2026-05-25 04:37:53,359 - INFO - ds_nowhere_to_go : 0 2026-05-25 04:37:53,359 - INFO - ds_flow_queue_limit_exceeded : 0 2026-05-25 04:37:53,359 - INFO - ds_flow_no_memory : 0 2026-05-25 04:37:53,359 - INFO - ds_flow_invalid_protocol : 0 2026-05-25 04:37:53,359 - INFO - ds_flow_nat_no_rflow : 0 2026-05-25 04:37:53,359 - INFO - ds_flow_action_drop : 21 2026-05-25 04:37:53,359 - INFO - ds_flow_action_invalid : 0 2026-05-25 04:37:53,359 - INFO - ds_flow_unusable : 0 2026-05-25 04:37:53,359 - INFO - ds_flow_table_full : 0 2026-05-25 04:37:53,359 - INFO - ds_interface_tx_discard : 0 2026-05-25 04:37:53,359 - INFO - ds_interface_drop : 11 2026-05-25 04:37:53,359 - INFO - ds_duplicated : 1 2026-05-25 04:37:53,359 - INFO - ds_push : 0 2026-05-25 04:37:53,359 - INFO - ds_ttl_exceeded : 0 2026-05-25 04:37:53,359 - INFO - ds_invalid_nh : 4 2026-05-25 04:37:53,359 - INFO - ds_invalid_label : 0 2026-05-25 04:37:53,360 - INFO - ds_invalid_protocol : 0 2026-05-25 04:37:53,360 - INFO - ds_interface_rx_discard : 0 2026-05-25 04:37:53,360 - INFO - ds_invalid_mcast_source : 0 2026-05-25 04:37:53,360 - INFO - ds_head_alloc_fail : 0 2026-05-25 04:37:53,360 - INFO - ds_pcow_fail : 0 2026-05-25 04:37:53,360 - INFO - ds_mcast_clone_fail : 0 2026-05-25 04:37:53,360 - INFO - ds_rewrite_fail : 0 2026-05-25 04:37:53,360 - INFO - ds_misc : 0 2026-05-25 04:37:53,360 - INFO - ds_invalid_packet : 0 2026-05-25 04:37:53,360 - INFO - ds_cksum_err : 0 2026-05-25 04:37:53,360 - INFO - ds_no_fmd : 0 2026-05-25 04:37:53,360 - INFO - ds_invalid_vnid : 0 2026-05-25 04:37:53,360 - INFO - ds_frag_err : 0 2026-05-25 04:37:53,360 - INFO - ds_invalid_source : 0 2026-05-25 04:37:53,360 - INFO - ds_mcast_df_bit : 0 2026-05-25 04:37:53,360 - INFO - ds_l2_no_route : 1 2026-05-25 04:37:53,360 - INFO - ds_vlan_fwd_tx : 0 2026-05-25 04:37:53,360 - INFO - ds_vlan_fwd_enq : 0 2026-05-25 04:37:53,360 - INFO - ds_drop_new_flow : 0 2026-05-25 04:37:53,360 - INFO - ds_flow_evict : 0 2026-05-25 04:37:53,360 - INFO - ds_trap_original : 0 2026-05-25 04:37:53,360 - INFO - ds_no_memory : 0 2026-05-25 04:37:53,360 - INFO - ds_fragment_queue_fail : 0 2026-05-25 04:37:53,360 - INFO - ds_no_frag_entry : 0 2026-05-25 04:37:53,361 - INFO - ds_icmp_error : 0 2026-05-25 04:37:53,361 - INFO - ds_clone_fail : 0 2026-05-25 04:37:53,361 - INFO - ds_drop_pkts : 49 2026-05-25 04:37:53,361 - INFO - Deleting VM ctest-vm2-71417537 2026-05-25 04:37:53,449 - DEBUG - Verifying in api server 10.0.0.131 2026-05-25 04:37:53,450 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine/2c7f212d-5279-4a60-bf61-d32430eb6a95 2026-05-25 04:37:53,462 - DEBUG - VM ID 2c7f212d-5279-4a60-bf61-d32430eb6a95 of VM ctest-vm2-71417537 is still found in API Server 2026-05-25 04:37:55,463 - DEBUG - Verifying in api server 10.0.0.131 2026-05-25 04:37:55,463 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine/2c7f212d-5279-4a60-bf61-d32430eb6a95 2026-05-25 04:37:55,472 - DEBUG - Response Code: 404 2026-05-25 04:37:55,472 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine/2c7f212d-5279-4a60-bf61-d32430eb6a95 2026-05-25 04:37:55,479 - DEBUG - Response Code: 404 2026-05-25 04:37:55,480 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine/2c7f212d-5279-4a60-bf61-d32430eb6a95 2026-05-25 04:37:55,489 - DEBUG - Response Code: 404 2026-05-25 04:37:55,489 - INFO - VM ctest-vm2-71417537 is fully removed in API-Server 2026-05-25 04:37:55,650 - DEBUG - Skipping VM-deletion-check in nova db since mysql_token is not found 2026-05-25 04:37:55,705 - DEBUG - Validated that vrouter 10.0.0.133 does not have VMs route for VN default-domain:ctest-TestDropStats-44123704:ctest-vn-29452387 2026-05-25 04:37:55,719 - DEBUG - Validated that vrouter 10.0.0.240 does not have VMs route for VN default-domain:ctest-TestDropStats-44123704:ctest-vn-29452387 2026-05-25 04:37:55,719 - INFO - Validated that all vrouters do not have VMs route for VN default-domain:ctest-TestDropStats-44123704:ctest-vn-29452387 2026-05-25 04:37:55,719 - INFO - VM ctest-vm2-71417537 is removed in Compute, and routes are removed in all compute nodes 2026-05-25 04:37:55,974 - INFO - Routes for VM ctest-vm2-71417537 is removed in all control-nodes 2026-05-25 04:37:55,974 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-5822-2:Compute:contrail-vrouter-agent:0?flat 2026-05-25 04:37:56,028 - DEBUG - Requesting: http://10.0.0.131:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-5822-2?flat 2026-05-25 04:37:56,062 - DEBUG - VM 2c7f212d-5279-4a60-bf61-d32430eb6a95 is still present in vrouter an-jenkins-deploy-platform-ansible-os-5822-2 uve 2026-05-25 04:37:56,062 - DEBUG - Verifying if the vm interface deleted from vroter uve... 2026-05-25 04:37:56,062 - DEBUG - interface for vm 2c7f212d-5279-4a60-bf61-d32430eb6a95 deleted from vrouter uve ... 2026-05-25 04:37:56,062 - DEBUG - Deleted interface not in error interface list ... 2026-05-25 04:37:59,063 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-5822-2:Compute:contrail-vrouter-agent:0?flat 2026-05-25 04:37:59,126 - DEBUG - Requesting: http://10.0.0.131:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-5822-2?flat 2026-05-25 04:37:59,164 - DEBUG - VM 2c7f212d-5279-4a60-bf61-d32430eb6a95 is still present in vrouter an-jenkins-deploy-platform-ansible-os-5822-2 uve 2026-05-25 04:37:59,165 - DEBUG - Verifying if the vm interface deleted from vroter uve... 2026-05-25 04:37:59,165 - DEBUG - interface for vm 2c7f212d-5279-4a60-bf61-d32430eb6a95 deleted from vrouter uve ... 2026-05-25 04:37:59,165 - DEBUG - Deleted interface not in error interface list ... 2026-05-25 04:38:02,165 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-5822-2:Compute:contrail-vrouter-agent:0?flat 2026-05-25 04:38:02,217 - DEBUG - Requesting: http://10.0.0.131:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-5822-2?flat 2026-05-25 04:38:02,258 - DEBUG - VM 2c7f212d-5279-4a60-bf61-d32430eb6a95 is not present in vrouter an-jenkins-deploy-platform-ansible-os-5822-2 uve 2026-05-25 04:38:02,258 - DEBUG - Verifying if the vm interface deleted from vroter uve... 2026-05-25 04:38:02,258 - DEBUG - interface for vm 2c7f212d-5279-4a60-bf61-d32430eb6a95 deleted from vrouter uve ... 2026-05-25 04:38:02,258 - DEBUG - Deleted interface not in error interface list ... 2026-05-25 04:38:02,258 - DEBUG - Validated that VM 2c7f212d-5279-4a60-bf61-d32430eb6a95 is removed in Vrouter UVE 2026-05-25 04:38:02,258 - DEBUG - Verifying through opserver in 10.0.0.73 2026-05-25 04:38:02,259 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/virtual-network/default-domain:ctest-TestDropStats-44123704:ctest-vn-29452387?flat 2026-05-25 04:38:02,282 - DEBUG - 2c7f212d-5279-4a60-bf61-d32430eb6a95 BM not in default-domain:ctest-TestDropStats-44123704:ctest-vn-29452387 uve 2026-05-25 04:38:02,282 - DEBUG - Validated that VM 2c7f212d-5279-4a60-bf61-d32430eb6a95 is not present in the VN UVE 2026-05-25 04:38:02,282 - DEBUG - Verifying the 2c7f212d-5279-4a60-bf61-d32430eb6a95 virtual network link through opserver 10.0.0.73 2026-05-25 04:38:02,282 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/virtual-machines 2026-05-25 04:38:02,299 - INFO - 2c7f212d-5279-4a60-bf61-d32430eb6a95 vm uve deleted from opserver 2026-05-25 04:38:02,300 - DEBUG - Verifying the 2c7f212d-5279-4a60-bf61-d32430eb6a95 virtual network link through opserver 10.0.0.19 2026-05-25 04:38:02,300 - DEBUG - Requesting: http://10.0.0.19:8081/analytics/uves/virtual-machines 2026-05-25 04:38:02,382 - INFO - 2c7f212d-5279-4a60-bf61-d32430eb6a95 vm uve deleted from opserver 2026-05-25 04:38:02,382 - DEBUG - Verifying the 2c7f212d-5279-4a60-bf61-d32430eb6a95 virtual network link through opserver 10.0.0.131 2026-05-25 04:38:02,382 - DEBUG - Requesting: http://10.0.0.131:8081/analytics/uves/virtual-machines 2026-05-25 04:38:02,399 - INFO - 2c7f212d-5279-4a60-bf61-d32430eb6a95 vm uve deleted from opserver 2026-05-25 04:38:02,400 - INFO - Validated that VM 2c7f212d-5279-4a60-bf61-d32430eb6a95 is removed from Opserver 2026-05-25 04:38:02,400 - INFO - Deleting VM ctest-vm1-07876244 2026-05-25 04:38:02,496 - DEBUG - Verifying in api server 10.0.0.131 2026-05-25 04:38:02,497 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine/4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa 2026-05-25 04:38:02,508 - DEBUG - VM ID 4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa of VM ctest-vm1-07876244 is still found in API Server 2026-05-25 04:38:04,509 - DEBUG - Verifying in api server 10.0.0.131 2026-05-25 04:38:04,509 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine/4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa 2026-05-25 04:38:04,517 - DEBUG - Response Code: 404 2026-05-25 04:38:04,518 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine/4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa 2026-05-25 04:38:04,524 - DEBUG - Response Code: 404 2026-05-25 04:38:04,525 - DEBUG - Requesting: http://10.0.0.131:8082/virtual-machine/4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa 2026-05-25 04:38:04,533 - DEBUG - Response Code: 404 2026-05-25 04:38:04,533 - INFO - VM ctest-vm1-07876244 is fully removed in API-Server 2026-05-25 04:38:04,709 - DEBUG - Skipping VM-deletion-check in nova db since mysql_token is not found 2026-05-25 04:38:04,763 - DEBUG - Validated that vrouter 10.0.0.133 does not have VMs route for VN default-domain:ctest-TestDropStats-44123704:ctest-vn-50854477 2026-05-25 04:38:04,775 - DEBUG - Validated that vrouter 10.0.0.240 does not have VMs route for VN default-domain:ctest-TestDropStats-44123704:ctest-vn-50854477 2026-05-25 04:38:04,776 - INFO - Validated that all vrouters do not have VMs route for VN default-domain:ctest-TestDropStats-44123704:ctest-vn-50854477 2026-05-25 04:38:04,776 - INFO - VM ctest-vm1-07876244 is removed in Compute, and routes are removed in all compute nodes 2026-05-25 04:38:05,022 - INFO - Routes for VM ctest-vm1-07876244 is removed in all control-nodes 2026-05-25 04:38:05,022 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-5822-1:Compute:contrail-vrouter-agent:0?flat 2026-05-25 04:38:05,073 - DEBUG - Requesting: http://10.0.0.131:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-5822-1?flat 2026-05-25 04:38:05,107 - DEBUG - VM 4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa is still present in vrouter an-jenkins-deploy-platform-ansible-os-5822-1 uve 2026-05-25 04:38:05,107 - DEBUG - Verifying if the vm interface deleted from vroter uve... 2026-05-25 04:38:05,107 - DEBUG - interface for vm 4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa deleted from vrouter uve ... 2026-05-25 04:38:05,108 - DEBUG - Deleted interface not in error interface list ... 2026-05-25 04:38:08,108 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-5822-1:Compute:contrail-vrouter-agent:0?flat 2026-05-25 04:38:08,173 - DEBUG - Requesting: http://10.0.0.131:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-5822-1?flat 2026-05-25 04:38:08,206 - DEBUG - VM 4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa is still present in vrouter an-jenkins-deploy-platform-ansible-os-5822-1 uve 2026-05-25 04:38:08,206 - DEBUG - Verifying if the vm interface deleted from vroter uve... 2026-05-25 04:38:08,206 - DEBUG - interface for vm 4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa deleted from vrouter uve ... 2026-05-25 04:38:08,206 - DEBUG - Deleted interface not in error interface list ... 2026-05-25 04:38:11,207 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-5822-1:Compute:contrail-vrouter-agent:0?flat 2026-05-25 04:38:11,265 - DEBUG - Requesting: http://10.0.0.131:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-5822-1?flat 2026-05-25 04:38:11,306 - DEBUG - VM 4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa is still present in vrouter an-jenkins-deploy-platform-ansible-os-5822-1 uve 2026-05-25 04:38:11,306 - DEBUG - Verifying if the vm interface deleted from vroter uve... 2026-05-25 04:38:11,307 - DEBUG - interface for vm 4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa deleted from vrouter uve ... 2026-05-25 04:38:11,307 - DEBUG - Deleted interface not in error interface list ... 2026-05-25 04:38:14,308 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-5822-1:Compute:contrail-vrouter-agent:0?flat 2026-05-25 04:38:14,387 - DEBUG - Requesting: http://10.0.0.131:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-5822-1?flat 2026-05-25 04:38:14,439 - DEBUG - VM 4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa is still present in vrouter an-jenkins-deploy-platform-ansible-os-5822-1 uve 2026-05-25 04:38:14,439 - DEBUG - Verifying if the vm interface deleted from vroter uve... 2026-05-25 04:38:14,439 - DEBUG - interface for vm 4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa deleted from vrouter uve ... 2026-05-25 04:38:14,439 - DEBUG - Deleted interface not in error interface list ... 2026-05-25 04:38:17,439 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-5822-1:Compute:contrail-vrouter-agent:0?flat 2026-05-25 04:38:17,486 - DEBUG - Requesting: http://10.0.0.131:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-5822-1?flat 2026-05-25 04:38:17,524 - DEBUG - VM 4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa is still present in vrouter an-jenkins-deploy-platform-ansible-os-5822-1 uve 2026-05-25 04:38:17,524 - DEBUG - Verifying if the vm interface deleted from vroter uve... 2026-05-25 04:38:17,524 - DEBUG - interface for vm 4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa deleted from vrouter uve ... 2026-05-25 04:38:17,524 - DEBUG - Deleted interface not in error interface list ... 2026-05-25 04:38:20,525 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-5822-1:Compute:contrail-vrouter-agent:0?flat 2026-05-25 04:38:20,585 - DEBUG - Requesting: http://10.0.0.131:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-5822-1?flat 2026-05-25 04:38:20,620 - DEBUG - VM 4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa is still present in vrouter an-jenkins-deploy-platform-ansible-os-5822-1 uve 2026-05-25 04:38:20,620 - DEBUG - Verifying if the vm interface deleted from vroter uve... 2026-05-25 04:38:20,620 - DEBUG - interface for vm 4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa deleted from vrouter uve ... 2026-05-25 04:38:20,620 - DEBUG - Deleted interface not in error interface list ... 2026-05-25 04:38:23,620 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-5822-1:Compute:contrail-vrouter-agent:0?flat 2026-05-25 04:38:23,680 - DEBUG - Requesting: http://10.0.0.131:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-5822-1?flat 2026-05-25 04:38:23,715 - DEBUG - VM 4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa is not present in vrouter an-jenkins-deploy-platform-ansible-os-5822-1 uve 2026-05-25 04:38:23,716 - DEBUG - Verifying if the vm interface deleted from vroter uve... 2026-05-25 04:38:23,716 - DEBUG - interface for vm 4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa deleted from vrouter uve ... 2026-05-25 04:38:23,716 - DEBUG - Deleted interface not in error interface list ... 2026-05-25 04:38:23,716 - DEBUG - Validated that VM 4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa is removed in Vrouter UVE 2026-05-25 04:38:23,716 - DEBUG - Verifying through opserver in 10.0.0.73 2026-05-25 04:38:23,716 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/virtual-network/default-domain:ctest-TestDropStats-44123704:ctest-vn-50854477?flat 2026-05-25 04:38:23,744 - DEBUG - 4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa BM not in default-domain:ctest-TestDropStats-44123704:ctest-vn-50854477 uve 2026-05-25 04:38:23,744 - DEBUG - Validated that VM 4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa is not present in the VN UVE 2026-05-25 04:38:23,744 - DEBUG - Verifying the 4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa virtual network link through opserver 10.0.0.73 2026-05-25 04:38:23,744 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/virtual-machines 2026-05-25 04:38:23,758 - INFO - 4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa vm uve deleted from opserver 2026-05-25 04:38:23,758 - DEBUG - Verifying the 4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa virtual network link through opserver 10.0.0.19 2026-05-25 04:38:23,761 - DEBUG - Requesting: http://10.0.0.19:8081/analytics/uves/virtual-machines 2026-05-25 04:38:23,775 - INFO - 4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa vm uve deleted from opserver 2026-05-25 04:38:23,775 - DEBUG - Verifying the 4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa virtual network link through opserver 10.0.0.131 2026-05-25 04:38:23,775 - DEBUG - Requesting: http://10.0.0.131:8081/analytics/uves/virtual-machines 2026-05-25 04:38:23,790 - INFO - 4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa vm uve deleted from opserver 2026-05-25 04:38:23,790 - INFO - Validated that VM 4b0bf5c0-ca33-4c65-b184-ec7d48eb9dfa is removed from Opserver 2026-05-25 04:38:23,790 - INFO - Deleting VN ctest-vn-29452387 2026-05-25 04:38:24,051 - DEBUG - Response for deleting network () 2026-05-25 04:38:24,051 - INFO - Deleting VN ctest-vn-50854477 2026-05-25 04:38:24,336 - DEBUG - Response for deleting network () 2026-05-25 04:38:25,631 - DEBUG - Skipping xmpp flap check 2026-05-25 04:38:25,631 - INFO - END TEST : test_flow_action_drop_stats : PASSED[0:01:18] 2026-05-25 04:38:25,632 - INFO - -------------------------------------------------------------------------------- 2026-05-25 04:38:26,492 - INFO - Deleted project: ctest-TestDropStats-44123704, ID : 6428eabb-35f4-4ec6-9119-26db8da706db