2026-06-15 03:53:11,186 - INFO - Domain Default found not creating 2026-06-15 03:53:11,336 - INFO - Project ctest-TestDropStats-57016829 not found, creating it 2026-06-15 03:53:11,890 - INFO - Created Project:ctest-TestDropStats-57016829, ID : c8280ec8-3916-4d47-844f-212e1318f1db 2026-06-15 03:53:13,387 - INFO - ================================================================================ 2026-06-15 03:53:13,388 - INFO - STARTING TEST : test_flow_action_drop_stats 2026-06-15 03:53:13,388 - INFO - TEST DESCRIPTION : Description: Verify flow action drop stats Steps: 1. Create 2 VNs and launch 1 VM in each VN, on different nodes on multi node setup 2. Add policy to deny pkts between VNs. 3. Ping VM2 from VM1 Pass criteria: 1. Verify flow action count gets incremented, get it from introspect 2. Verify flow action count gets incremented for the vmi of VM1, get it from introspect 2026-06-15 03:53:14,650 - DEBUG - Skipping xmpp flap check 2026-06-15 03:53:14,650 - INFO - Initial checks done. Running the testcase now 2026-06-15 03:53:14,650 - INFO - 2026-06-15 03:53:14,692 - DEBUG - Services list from nova: [, , , ] 2026-06-15 03:53:14,692 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-5940-1', 'an-jenkins-deploy-platform-ansible-os-5940-2']} 2026-06-15 03:53:15,363 - DEBUG - Response for create_network : {'network': {'id': 'c5785afb-89c5-4a9c-8418-d5902b4bba61', 'name': 'ctest-vn-01937378', 'tenant_id': 'c8280ec839164d47844f212e1318f1db', 'project_id': 'c8280ec839164d47844f212e1318f1db', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestDropStats-57016829', 'ctest-vn-01937378'], 'port_security_enabled': True, 'description': ''}} 2026-06-15 03:53:15,674 - DEBUG - Response for create_subnet : {'subnet': {'id': '7eacdeeb-33e5-4c00-8536-e552637b85ca', 'name': '', 'tenant_id': 'c8280ec839164d47844f212e1318f1db', 'network_id': 'c5785afb-89c5-4a9c-8418-d5902b4bba61', 'ip_version': 4, 'cidr': '115.244.231.0/26', 'allocation_pools': [{'start': '115.244.231.2', 'end': '115.244.231.62'}], 'gateway_ip': '115.244.231.1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '115.244.231.2', 'tags': [], 'project_id': 'c8280ec839164d47844f212e1318f1db'}} 2026-06-15 03:53:15,691 - INFO - Created VN ctest-vn-01937378 2026-06-15 03:53:15,703 - DEBUG - VN ctest-vn-01937378 UUID is c5785afb-89c5-4a9c-8418-d5902b4bba61 2026-06-15 03:53:16,103 - DEBUG - Response for create_network : {'network': {'id': '70953075-7218-47ff-8c7f-33f417cad04a', 'name': 'ctest-vn-72875685', 'tenant_id': 'c8280ec839164d47844f212e1318f1db', 'project_id': 'c8280ec839164d47844f212e1318f1db', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestDropStats-57016829', 'ctest-vn-72875685'], 'port_security_enabled': True, 'description': ''}} 2026-06-15 03:53:16,346 - DEBUG - Response for create_subnet : {'subnet': {'id': 'df115c8e-c180-44a8-902c-30ad9a9ddbc2', 'name': '', 'tenant_id': 'c8280ec839164d47844f212e1318f1db', 'network_id': '70953075-7218-47ff-8c7f-33f417cad04a', 'ip_version': 4, 'cidr': '93.96.109.192/26', 'allocation_pools': [{'start': '93.96.109.194', 'end': '93.96.109.254'}], 'gateway_ip': '93.96.109.193', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '93.96.109.194', 'tags': [], 'project_id': 'c8280ec839164d47844f212e1318f1db'}} 2026-06-15 03:53:16,368 - INFO - Created VN ctest-vn-72875685 2026-06-15 03:53:16,379 - DEBUG - VN ctest-vn-72875685 UUID is 70953075-7218-47ff-8c7f-33f417cad04a 2026-06-15 03:53:17,837 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-5940-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-5940-1) 2026-06-15 03:53:19,088 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-5940-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-5940-2) 2026-06-15 03:53:19,088 - INFO - Waiting for VM ctest-vm1-47907004 to be up.. 2026-06-15 03:53:19,151 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2026-06-15 03:53:24,242 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2026-06-15 03:53:29,343 - DEBUG - VM is in ACTIVE state now 2026-06-15 03:53:29,343 - INFO - VM name : ctest-vm1-47907004 2026-06-15 03:53:29,439 - DEBUG - VM ctest-vm1-47907004 ID is b987a1cd-ef58-4565-9f6b-90d220408483 2026-06-15 03:53:29,480 - DEBUG - VM ctest-vm1-47907004 launched on Node an-jenkins-deploy-platform-ansible-os-5940-1 2026-06-15 03:53:29,577 - DEBUG - Requesting: http://10.0.0.65:8082/virtual-machine/b987a1cd-ef58-4565-9f6b-90d220408483 2026-06-15 03:53:29,871 - DEBUG - Requesting: http://10.0.0.65:8082/virtual-machine/b987a1cd-ef58-4565-9f6b-90d220408483 2026-06-15 03:53:29,901 - DEBUG - Requesting: http://10.0.0.65:8082/virtual-machine-interface/10f54797-abbf-42a9-9098-1d83a106cf83 2026-06-15 03:53:33,096 - DEBUG - (True, 'PING 169.254.0.5 (169.254.0.5) 56(84) bytes of data.\r\n\r\n--- 169.254.0.5 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1023ms') 2026-06-15 03:53:33,096 - DEBUG - Ping to Metadata IP 169.254.0.5 of VM ctest-vm1-47907004 failed! 2026-06-15 03:53:33,111 - DEBUG - Gateway for vn default-domain:ctest-TestDropStats-57016829:ctest-vn-01937378 is 115.244.231.1 and allocation pool is NOT set 2026-06-15 03:53:35,177 - DEBUG - (True, 'PING 169.254.0.5 (169.254.0.5) 56(84) bytes of data.\r\n64 bytes from 169.254.0.5: icmp_seq=1 ttl=63 time=4.53 ms\r\n64 bytes from 169.254.0.5: icmp_seq=2 ttl=63 time=0.444 ms\r\n\r\n--- 169.254.0.5 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1002ms\r\nrtt min/avg/max/mdev = 0.444/2.487/4.531/2.043 ms') 2026-06-15 03:53:35,178 - INFO - Ping to Metadata IP 169.254.0.5 of VM ctest-vm1-47907004 passed 2026-06-15 03:53:35,248 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2026-06-15 03:53:35,248 - DEBUG - Waiting to SSH to VM ctest-vm1-47907004, IP 115.244.231.3, Port 22 2026-06-15 03:53:35,405 - DEBUG - VM ctest-vm1-47907004 is ready for SSH connections 2026-06-15 03:53:35,405 - INFO - Waiting for VM ctest-vm2-99697257 to be up.. 2026-06-15 03:53:35,487 - DEBUG - VM is in ACTIVE state now 2026-06-15 03:53:35,487 - INFO - VM name : ctest-vm2-99697257 2026-06-15 03:53:35,563 - DEBUG - VM ctest-vm2-99697257 ID is 34afe8d3-5707-403c-93b9-e4fc43a1e3bb 2026-06-15 03:53:35,563 - DEBUG - VM ctest-vm2-99697257 launched on Node an-jenkins-deploy-platform-ansible-os-5940-2 2026-06-15 03:53:35,647 - DEBUG - Requesting: http://10.0.0.65:8082/virtual-machine/34afe8d3-5707-403c-93b9-e4fc43a1e3bb 2026-06-15 03:53:35,657 - DEBUG - Requesting: http://10.0.0.65:8082/virtual-machine-interface/96b62f3e-9641-460b-8e9c-5045672b230d 2026-06-15 03:53:36,835 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=6.48 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=0.576 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1002ms\r\nrtt min/avg/max/mdev = 0.576/3.528/6.480/2.952 ms') 2026-06-15 03:53:36,835 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-vm2-99697257 passed 2026-06-15 03:53:36,909 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2026-06-15 03:53:36,909 - DEBUG - Waiting to SSH to VM ctest-vm2-99697257, IP 93.96.109.195, Port 22 2026-06-15 03:53:37,079 - DEBUG - VM ctest-vm2-99697257 is ready for SSH connections 2026-06-15 03:53:37,157 - DEBUG - VM is in ACTIVE state now 2026-06-15 03:53:37,157 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2026-06-15 03:53:37,157 - DEBUG - Requesting: http://10.0.0.65:8082/virtual-machine/b987a1cd-ef58-4565-9f6b-90d220408483 2026-06-15 03:53:37,166 - DEBUG - Requesting: http://10.0.0.65:8082/virtual-machine-interface/10f54797-abbf-42a9-9098-1d83a106cf83 2026-06-15 03:53:37,178 - DEBUG - Requesting: http://10.0.0.65:8082/virtual-machine/b987a1cd-ef58-4565-9f6b-90d220408483 2026-06-15 03:53:37,187 - DEBUG - Requesting: http://10.0.0.65:8082/virtual-machine-interface/10f54797-abbf-42a9-9098-1d83a106cf83 2026-06-15 03:53:37,198 - DEBUG - Requesting: http://10.0.0.65:8082/instance-ip/8eaa868b-6589-4877-846a-3c25cb1e8325 2026-06-15 03:53:37,208 - DEBUG - Verifying in api server 10.0.0.65 2026-06-15 03:53:37,208 - DEBUG - Requesting: http://10.0.0.65:8082/virtual-machine/b987a1cd-ef58-4565-9f6b-90d220408483 2026-06-15 03:53:37,218 - DEBUG - Requesting: http://10.0.0.65:8082/virtual-machine-interface/10f54797-abbf-42a9-9098-1d83a106cf83 2026-06-15 03:53:37,230 - DEBUG - Requesting: http://10.0.0.65:8082/instance-ip/8eaa868b-6589-4877-846a-3c25cb1e8325 2026-06-15 03:53:37,240 - INFO - VM ctest-vm1-47907004 verfication in all API Servers passed 2026-06-15 03:53:37,269 - DEBUG - VM ctest-vm1-47907004 Tap interface: {'index': '5', 'name': 'tap10f54797-ab', 'uuid': '10f54797-abbf-42a9-9098-1d83a106cf83', 'vrf_name': 'default-domain:ctest-TestDropStats-57016829:ctest-vn-01937378:ctest-vn-01937378', 'active': 'Active', 'ipv4_active': 'Active', 'l2_active': 'L2 Active', 'ip6_active': 'Ipv6 Inactive < no-ipv6-addr >', 'health_check_active': 'Active', 'dhcp_service': 'Enable', 'dhcp_service_v6': 'Disable', 'dns_service': 'Enable', 'type': 'vport', 'label': '29', 'l2_label': '42', 'vxlan_id': '5', 'vn_name': 'default-domain:ctest-TestDropStats-57016829:ctest-vn-01937378', 'vm_uuid': 'b987a1cd-ef58-4565-9f6b-90d220408483', 'vm_name': 'ctest-vm1-47907004', 'ip_addr': '115.244.231.3', 'mac_addr': '02:10:f5:47:97:ab', 'policy': 'Enable', 'fip_list': [], 'mdata_ip_addr': '169.254.0.5', 'service_vlan_list': None, 'os_ifindex': '126', 'fabric_port': 'NotFabricPort', 'alloc_linklocal_ip': 'LL-Enable', 'analyzer_name': None, 'config_name': 'default-domain:ctest-TestDropStats-57016829:10f54797-abbf-42a9-9098-1d83a106cf83', 'sg_uuid_list': ['e5f47c7d-ef33-46e6-a770-a71a17cc56c1'], 'static_route_list': None, 'vm_project_uuid': 'c8280ec8-3916-4d47-844f-212e1318f1db', 'admin_state': 'Enabled', 'flow_key_idx': '30', 'allowed_address_pair_list': None, 'ip6_addr': '::', 'local_preference': '0', 'tx_vlan_id': '-1', 'rx_vlan_id': '-1', 'parent_interface': None, 'subnet': '--NA--', 'sub_type': 'Tap', 'vrf_assign_acl_uuid': '--NA--', 'vmi_type': 'Virtual Machine', 'transport': 'Ethernet', 'logical_interface_uuid': '00000000-0000-0000-0000-000000000000', 'flood_unknown_unicast': 'false', 'physical_device': None, 'physical_interface': None, 'fixed_ip4_list': None, 'fixed_ip6_list': None, 'fat_flow_list': None, 'metadata_ip_active': 'Active', 'service_health_check_ip': '0.0.0.0', 'alias_ip_list': None, 'drop_new_flows': 'false', 'bridge_domain_list': [], 'vmi_tag_list': None, 'policy_set_acl_list': None, 'slo_list': None, 'vhostuser_mode': '0', 'si_other_end_vmi': '00000000-0000-0000-0000-000000000000', 'cfg_igmp_enable': 'false', 'igmp_enabled': 'false', 'max_flows': '0', 'policy_set_fwaas_list': None, 'bond_interface_list': None, 'mac_ip_learning_enable': 'false', 'mac_ip_list': []} 2026-06-15 03:53:37,269 - DEBUG - Agent 10.0.0.141 vrf name: default-domain:ctest-TestDropStats-57016829:ctest-vn-01937378:ctest-vn-01937378 2026-06-15 03:53:37,279 - DEBUG - Agent VRF Object : {'vrf_list': [{'name': 'default-domain:ctest-TestDropStats-57016829:ctest-vn-01937378:ctest-vn-01937378', 'ucindex': '3', 'mcindex': '3', 'l2index': '3', 'source': 'Config; ', 'uc6index': '3', 'vn': 'default-domain:ctest-TestDropStats-57016829:ctest-vn-01937378', 'table_label': '-1', 'vxlan_id': '5', 'evpnindex': '3', 'brindex': '3', 'mplsindex': '0', 'RD': '10.20.0.14:3', 'mac_aging_time': '0', 'layer2_control_word': 'false', 'forwarding_vrf': None, 'hbf_rintf': '-1', 'hbf_lintf': '-1'}]} 2026-06-15 03:53:37,358 - DEBUG - VM ctest-vm1-47907004 : Tap interface tap10f54797-ab is set to Active 2026-06-15 03:53:37,358 - DEBUG - Tap interface tap10f54797-ab detail : {'index': '5', 'name': 'tap10f54797-ab', 'uuid': '10f54797-abbf-42a9-9098-1d83a106cf83', 'vrf_name': 'default-domain:ctest-TestDropStats-57016829:ctest-vn-01937378:ctest-vn-01937378', 'active': 'Active', 'ipv4_active': 'Active', 'l2_active': 'L2 Active', 'ip6_active': 'Ipv6 Inactive < no-ipv6-addr >', 'health_check_active': 'Active', 'dhcp_service': 'Enable', 'dhcp_service_v6': 'Disable', 'dns_service': 'Enable', 'type': 'vport', 'label': '29', 'l2_label': '42', 'vxlan_id': '5', 'vn_name': 'default-domain:ctest-TestDropStats-57016829:ctest-vn-01937378', 'vm_uuid': 'b987a1cd-ef58-4565-9f6b-90d220408483', 'vm_name': 'ctest-vm1-47907004', 'ip_addr': '115.244.231.3', 'mac_addr': '02:10:f5:47:97:ab', 'policy': 'Enable', 'fip_list': [], 'mdata_ip_addr': '169.254.0.5', 'service_vlan_list': None, 'os_ifindex': '126', 'fabric_port': 'NotFabricPort', 'alloc_linklocal_ip': 'LL-Enable', 'analyzer_name': None, 'config_name': 'default-domain:ctest-TestDropStats-57016829:10f54797-abbf-42a9-9098-1d83a106cf83', 'sg_uuid_list': ['e5f47c7d-ef33-46e6-a770-a71a17cc56c1'], 'static_route_list': None, 'vm_project_uuid': 'c8280ec8-3916-4d47-844f-212e1318f1db', 'admin_state': 'Enabled', 'flow_key_idx': '30', 'allowed_address_pair_list': None, 'ip6_addr': '::', 'local_preference': '0', 'tx_vlan_id': '-1', 'rx_vlan_id': '-1', 'parent_interface': None, 'subnet': '--NA--', 'sub_type': 'Tap', 'vrf_assign_acl_uuid': '--NA--', 'vmi_type': 'Virtual Machine', 'transport': 'Ethernet', 'logical_interface_uuid': '00000000-0000-0000-0000-000000000000', 'flood_unknown_unicast': 'false', 'physical_device': None, 'physical_interface': None, 'fixed_ip4_list': None, 'fixed_ip6_list': None, 'fat_flow_list': None, 'metadata_ip_active': 'Active', 'service_health_check_ip': '0.0.0.0', 'alias_ip_list': None, 'drop_new_flows': 'false', 'bridge_domain_list': [], 'vmi_tag_list': None, 'policy_set_acl_list': None, 'slo_list': None, 'vhostuser_mode': '0', 'si_other_end_vmi': '00000000-0000-0000-0000-000000000000', 'cfg_igmp_enable': 'false', 'igmp_enabled': 'false', 'max_flows': '0', 'policy_set_fwaas_list': None, 'bond_interface_list': None, 'mac_ip_learning_enable': 'false', 'mac_ip_list': []} 2026-06-15 03:53:37,475 - DEBUG - Starting Layer 2 verification in Agent 2026-06-15 03:53:37,486 - DEBUG - Layer 2 path is seen for VM MAC 02:10:f5:47:97:ab in agent 10.0.0.141 2026-06-15 03:53:37,486 - DEBUG - Active layer 2 route in agent is present for VMI tap10f54797-ab 2026-06-15 03:53:37,486 - DEBUG - L2 label(42) matches bw route table and interface table 2026-06-15 03:53:37,492 - DEBUG - VN default-domain:ctest-TestDropStats-57016829:ctest-vn-01937378 is found in Agent of node 10.0.0.141 2026-06-15 03:53:37,618 - DEBUG - VRF IDs of VN default-domain:ctest-TestDropStats-57016829:ctest-vn-01937378 is consistent in agent 10.0.0.141 2026-06-15 03:53:37,618 - DEBUG - Route for VM IP ['115.244.231.3'] is consistent in agent 10.0.0.141 2026-06-15 03:53:37,618 - DEBUG - VN default-domain:ctest-TestDropStats-57016829:ctest-vn-01937378 verification for VM ctest-vm1-47907004 in Agent 10.0.0.141 passed 2026-06-15 03:53:37,729 - DEBUG - Starting all layer 2 verification in agent 10.0.0.141 2026-06-15 03:53:37,743 - DEBUG - Route for VM MAC 02:10:f5:47:97:ab is consistent in agent 10.0.0.141 2026-06-15 03:53:37,750 - INFO - VM ctest-vm1-47907004 verifications in Compute nodes passed 2026-06-15 03:53:38,043 - DEBUG - Validated VM route 115.244.231.3 in vrouter of 10.0.0.141 2026-06-15 03:53:38,049 - DEBUG - Remote Nh validation passed. Expected Tunnel, label: None, 0; Got None,0 2026-06-15 03:53:38,050 - INFO - Validated routes of VM ctest-vm1-47907004 in all vrouters 2026-06-15 03:53:38,553 - DEBUG - Starting all layer2 verification in 10.0.0.65 Control Node 2026-06-15 03:53:38,672 - DEBUG - Layer2 route found for VM MAC 02:10:f5:47:97:ab in Control-node 10.0.0.65 2026-06-15 03:53:38,672 - DEBUG - L2 Label for VM ctest-vm1-47907004 same between Control-node 10.0.0.65 and Agent, Expected: 42, Seen: 42 2026-06-15 03:53:38,672 - DEBUG - Starting all layer2 verification in 10.0.0.23 Control Node 2026-06-15 03:53:38,795 - DEBUG - Layer2 route found for VM MAC 02:10:f5:47:97:ab in Control-node 10.0.0.23 2026-06-15 03:53:38,795 - DEBUG - L2 Label for VM ctest-vm1-47907004 same between Control-node 10.0.0.23 and Agent, Expected: 42, Seen: 42 2026-06-15 03:53:38,795 - INFO - Verification in Control-nodes for VM ctest-vm1-47907004 passed 2026-06-15 03:53:38,795 - DEBUG - Verifying the vm in opserver 2026-06-15 03:53:38,795 - DEBUG - Verifying in collector 10.0.0.65 ... 2026-06-15 03:53:38,795 - DEBUG - Requesting: http://10.0.0.65:8081/analytics/uves/virtual-machine/b987a1cd-ef58-4565-9f6b-90d220408483?flat 2026-06-15 03:53:39,113 - DEBUG - Requesting: http://10.0.0.65:8081/analytics/uves/virtual-machine/b987a1cd-ef58-4565-9f6b-90d220408483?flat 2026-06-15 03:53:39,184 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-machine-interface/default-domain:ctest-TestDropStats-57016829:10f54797-abbf-42a9-9098-1d83a106cf83?flat 2026-06-15 03:53:39,488 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-machine-interface/default-domain:ctest-TestDropStats-57016829:10f54797-abbf-42a9-9098-1d83a106cf83?flat 2026-06-15 03:53:39,563 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-machine-interface/default-domain:ctest-TestDropStats-57016829:10f54797-abbf-42a9-9098-1d83a106cf83?flat 2026-06-15 03:53:39,634 - DEBUG - Requesting: http://10.0.0.65:8081/analytics/uves/virtual-machine/b987a1cd-ef58-4565-9f6b-90d220408483?flat 2026-06-15 03:53:39,684 - DEBUG - Verifying vm in vn uve 2026-06-15 03:53:39,684 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-machine-interface/default-domain:ctest-TestDropStats-57016829:10f54797-abbf-42a9-9098-1d83a106cf83?flat 2026-06-15 03:53:39,755 - DEBUG - VM uve shows interface as {'tx_vlan': 65535, 'ip6_address': '::', 'l2_active': True, 'vm_uuid': 'b987a1cd-ef58-4565-9f6b-90d220408483', 'port_mirror_enabled': False, 'is_health_check_active': True, 'active': True, 'virtual_network': 'default-domain:ctest-TestDropStats-57016829:ctest-vn-01937378', 'admin_state': True, '__T': 1781495617310751, 'ip4_active': True, 'uuid': '10f54797-abbf-42a9-9098-1d83a106cf83', 'ip_address': '115.244.231.3', 'label': 29, 'vn_uuid': 'c5785afb-89c5-4a9c-8418-d5902b4bba61', 'vhostuser_mode': 0, 'fixed_ip4_list': ['115.244.231.3'], 'vm_name': 'ctest-vm1-47907004', 'rx_vlan': 65535, 'gateway': '115.244.231.1', 'ip6_active': False, 'mac_address': '02:10:f5:47:97:ab'} 2026-06-15 03:53:39,755 - DEBUG - VM uve shows ip address as ['115.244.231.3', '::'] 2026-06-15 03:53:39,755 - DEBUG - VM uve shows virtual network as default-domain:ctest-TestDropStats-57016829:ctest-vn-01937378 2026-06-15 03:53:39,755 - INFO - Verifying through opserver in 10.0.0.23 2026-06-15 03:53:39,755 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-TestDropStats-57016829:ctest-vn-01937378?flat 2026-06-15 03:53:39,774 - DEBUG - expected vm list b987a1cd-ef58-4565-9f6b-90d220408483 2026-06-15 03:53:39,775 - DEBUG - Extracted vm list ['b987a1cd-ef58-4565-9f6b-90d220408483'] 2026-06-15 03:53:39,775 - INFO - VM b987a1cd-ef58-4565-9f6b-90d220408483 is present in default-domain:ctest-TestDropStats-57016829:ctest-vn-01937378 2026-06-15 03:53:39,775 - DEBUG - Verifying vm in vrouter uve 2026-06-15 03:53:39,775 - DEBUG - Getting info from collector 10.0.0.65.. 2026-06-15 03:53:39,775 - DEBUG - Requesting: http://10.0.0.65:8081/analytics/uves/virtual-machine/b987a1cd-ef58-4565-9f6b-90d220408483?flat 2026-06-15 03:53:39,840 - DEBUG - VM uve shows vrouter as ['an-jenkins-deploy-platform-ansible-os-5940-1'] 2026-06-15 03:53:39,840 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-5940-1:Compute:contrail-vrouter-agent:0?flat 2026-06-15 03:53:39,872 - DEBUG - Requesting: http://10.0.0.65:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-5940-1?flat 2026-06-15 03:53:39,913 - DEBUG - VM b987a1cd-ef58-4565-9f6b-90d220408483 is present in vrouter an-jenkins-deploy-platform-ansible-os-5940-1 uve 2026-06-15 03:53:39,913 - DEBUG - Validated that VM ctest-vm1-47907004 is in Vrouter an-jenkins-deploy-platform-ansible-os-5940-1 UVE 2026-06-15 03:53:39,913 - DEBUG - Verifying vm tap interface/vn in vrouter uve 2026-06-15 03:53:39,922 - DEBUG - Expected tap interface of VM uuid b987a1cd-ef58-4565-9f6b-90d220408483 is default-domain:ctest-TestDropStats-57016829:10f54797-abbf-42a9-9098-1d83a106cf83 2026-06-15 03:53:39,923 - DEBUG - Expected VN of VM uuid b987a1cd-ef58-4565-9f6b-90d220408483 is default-domain:ctest-TestDropStats-57016829:ctest-vn-01937378 2026-06-15 03:53:39,923 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-5940-1:Compute:contrail-vrouter-agent:0?flat 2026-06-15 03:53:39,959 - DEBUG - Requesting: http://10.0.0.65:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-5940-1?flat 2026-06-15 03:53:39,987 - DEBUG - VM b987a1cd-ef58-4565-9f6b-90d220408483 is present in vrouter an-jenkins-deploy-platform-ansible-os-5940-1 uve 2026-06-15 03:53:39,987 - INFO - tap interface default-domain:ctest-TestDropStats-57016829:10f54797-abbf-42a9-9098-1d83a106cf83 of vm b987a1cd-ef58-4565-9f6b-90d220408483 is present in vrouter an-jenkins-deploy-platform-ansible-os-5940-1 uve 2026-06-15 03:53:39,987 - DEBUG - Connected network default-domain:ctest-TestDropStats-57016829:ctest-vn-01937378 of vm b987a1cd-ef58-4565-9f6b-90d220408483 is present in vrouter an-jenkins-deploy-platform-ansible-os-5940-1 uve 2026-06-15 03:53:39,987 - DEBUG - Verifying the b987a1cd-ef58-4565-9f6b-90d220408483 virtual network link through opserver 10.0.0.23 2026-06-15 03:53:39,987 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-machines 2026-06-15 03:53:40,003 - DEBUG - VM link and name as {'name': 'b987a1cd-ef58-4565-9f6b-90d220408483', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-machine/b987a1cd-ef58-4565-9f6b-90d220408483?flat'} 2026-06-15 03:53:40,003 - DEBUG - Verifying the b987a1cd-ef58-4565-9f6b-90d220408483 virtual network link through opserver 10.0.0.33 2026-06-15 03:53:40,003 - DEBUG - Requesting: http://10.0.0.33:8081/analytics/uves/virtual-machines 2026-06-15 03:53:40,314 - DEBUG - Requesting: http://10.0.0.33:8081/analytics/uves/virtual-machines 2026-06-15 03:53:40,376 - DEBUG - VM link and name as {'name': 'b987a1cd-ef58-4565-9f6b-90d220408483', 'href': 'http://10.0.0.33:8081/analytics/uves/virtual-machine/b987a1cd-ef58-4565-9f6b-90d220408483?flat'} 2026-06-15 03:53:40,376 - DEBUG - Verifying the b987a1cd-ef58-4565-9f6b-90d220408483 virtual network link through opserver 10.0.0.65 2026-06-15 03:53:40,376 - DEBUG - Requesting: http://10.0.0.65:8081/analytics/uves/virtual-machines 2026-06-15 03:53:40,389 - DEBUG - VM link and name as {'name': 'b987a1cd-ef58-4565-9f6b-90d220408483', 'href': 'http://10.0.0.65:8081/analytics/uves/virtual-machine/b987a1cd-ef58-4565-9f6b-90d220408483?flat'} 2026-06-15 03:53:40,389 - INFO - VM ctest-vm1-47907004 validations in Opserver passed 2026-06-15 03:53:40,483 - DEBUG - VM is in ACTIVE state now 2026-06-15 03:53:40,483 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2026-06-15 03:53:40,483 - DEBUG - Requesting: http://10.0.0.65:8082/virtual-machine/34afe8d3-5707-403c-93b9-e4fc43a1e3bb 2026-06-15 03:53:40,491 - DEBUG - Requesting: http://10.0.0.65:8082/virtual-machine-interface/96b62f3e-9641-460b-8e9c-5045672b230d 2026-06-15 03:53:40,503 - DEBUG - Requesting: http://10.0.0.65:8082/virtual-machine/34afe8d3-5707-403c-93b9-e4fc43a1e3bb 2026-06-15 03:53:40,513 - DEBUG - Requesting: http://10.0.0.65:8082/virtual-machine-interface/96b62f3e-9641-460b-8e9c-5045672b230d 2026-06-15 03:53:40,523 - DEBUG - Requesting: http://10.0.0.65:8082/instance-ip/c337f5ce-5fdb-4b2b-bb4c-5ac93fbd13d7 2026-06-15 03:53:40,532 - DEBUG - Verifying in api server 10.0.0.65 2026-06-15 03:53:40,532 - DEBUG - Requesting: http://10.0.0.65:8082/virtual-machine/34afe8d3-5707-403c-93b9-e4fc43a1e3bb 2026-06-15 03:53:40,539 - DEBUG - Requesting: http://10.0.0.65:8082/virtual-machine-interface/96b62f3e-9641-460b-8e9c-5045672b230d 2026-06-15 03:53:40,549 - DEBUG - Requesting: http://10.0.0.65:8082/instance-ip/c337f5ce-5fdb-4b2b-bb4c-5ac93fbd13d7 2026-06-15 03:53:40,558 - INFO - VM ctest-vm2-99697257 verfication in all API Servers passed 2026-06-15 03:53:40,582 - DEBUG - VM ctest-vm2-99697257 Tap interface: {'index': '3', 'name': 'tap96b62f3e-96', 'uuid': '96b62f3e-9641-460b-8e9c-5045672b230d', 'vrf_name': 'default-domain:ctest-TestDropStats-57016829:ctest-vn-72875685:ctest-vn-72875685', 'active': 'Active', 'ipv4_active': 'Active', 'l2_active': 'L2 Active', 'ip6_active': 'Ipv6 Inactive < no-ipv6-addr >', 'health_check_active': 'Active', 'dhcp_service': 'Enable', 'dhcp_service_v6': 'Disable', 'dns_service': 'Enable', 'type': 'vport', 'label': '30', 'l2_label': '42', 'vxlan_id': '6', 'vn_name': 'default-domain:ctest-TestDropStats-57016829:ctest-vn-72875685', 'vm_uuid': '34afe8d3-5707-403c-93b9-e4fc43a1e3bb', 'vm_name': 'ctest-vm2-99697257', 'ip_addr': '93.96.109.195', 'mac_addr': '02:96:b6:2f:3e:96', 'policy': 'Enable', 'fip_list': [], 'mdata_ip_addr': '169.254.0.3', 'service_vlan_list': None, 'os_ifindex': '98', 'fabric_port': 'NotFabricPort', 'alloc_linklocal_ip': 'LL-Enable', 'analyzer_name': None, 'config_name': 'default-domain:ctest-TestDropStats-57016829:96b62f3e-9641-460b-8e9c-5045672b230d', 'sg_uuid_list': ['e5f47c7d-ef33-46e6-a770-a71a17cc56c1'], 'static_route_list': None, 'vm_project_uuid': 'c8280ec8-3916-4d47-844f-212e1318f1db', 'admin_state': 'Enabled', 'flow_key_idx': '35', 'allowed_address_pair_list': None, 'ip6_addr': '::', 'local_preference': '0', 'tx_vlan_id': '-1', 'rx_vlan_id': '-1', 'parent_interface': None, 'subnet': '--NA--', 'sub_type': 'Tap', 'vrf_assign_acl_uuid': '--NA--', 'vmi_type': 'Virtual Machine', 'transport': 'Ethernet', 'logical_interface_uuid': '00000000-0000-0000-0000-000000000000', 'flood_unknown_unicast': 'false', 'physical_device': None, 'physical_interface': None, 'fixed_ip4_list': None, 'fixed_ip6_list': None, 'fat_flow_list': None, 'metadata_ip_active': 'Active', 'service_health_check_ip': '0.0.0.0', 'alias_ip_list': None, 'drop_new_flows': 'false', 'bridge_domain_list': [], 'vmi_tag_list': None, 'policy_set_acl_list': None, 'slo_list': None, 'vhostuser_mode': '0', 'si_other_end_vmi': '00000000-0000-0000-0000-000000000000', 'cfg_igmp_enable': 'false', 'igmp_enabled': 'false', 'max_flows': '0', 'policy_set_fwaas_list': None, 'bond_interface_list': None, 'mac_ip_learning_enable': 'false', 'mac_ip_list': []} 2026-06-15 03:53:40,582 - DEBUG - Agent 10.0.0.130 vrf name: default-domain:ctest-TestDropStats-57016829:ctest-vn-72875685:ctest-vn-72875685 2026-06-15 03:53:40,588 - DEBUG - Agent VRF Object : {'vrf_list': [{'name': 'default-domain:ctest-TestDropStats-57016829:ctest-vn-72875685:ctest-vn-72875685', 'ucindex': '3', 'mcindex': '3', 'l2index': '3', 'source': 'Config; ', 'uc6index': '3', 'vn': 'default-domain:ctest-TestDropStats-57016829:ctest-vn-72875685', 'table_label': '-1', 'vxlan_id': '6', 'evpnindex': '3', 'brindex': '3', 'mplsindex': '0', 'RD': '10.20.0.18:3', 'mac_aging_time': '0', 'layer2_control_word': 'false', 'forwarding_vrf': None, 'hbf_rintf': '-1', 'hbf_lintf': '-1'}]} 2026-06-15 03:53:40,670 - DEBUG - VM ctest-vm2-99697257 : Tap interface tap96b62f3e-96 is set to Active 2026-06-15 03:53:40,670 - DEBUG - Tap interface tap96b62f3e-96 detail : {'index': '3', 'name': 'tap96b62f3e-96', 'uuid': '96b62f3e-9641-460b-8e9c-5045672b230d', 'vrf_name': 'default-domain:ctest-TestDropStats-57016829:ctest-vn-72875685:ctest-vn-72875685', 'active': 'Active', 'ipv4_active': 'Active', 'l2_active': 'L2 Active', 'ip6_active': 'Ipv6 Inactive < no-ipv6-addr >', 'health_check_active': 'Active', 'dhcp_service': 'Enable', 'dhcp_service_v6': 'Disable', 'dns_service': 'Enable', 'type': 'vport', 'label': '30', 'l2_label': '42', 'vxlan_id': '6', 'vn_name': 'default-domain:ctest-TestDropStats-57016829:ctest-vn-72875685', 'vm_uuid': '34afe8d3-5707-403c-93b9-e4fc43a1e3bb', 'vm_name': 'ctest-vm2-99697257', 'ip_addr': '93.96.109.195', 'mac_addr': '02:96:b6:2f:3e:96', 'policy': 'Enable', 'fip_list': [], 'mdata_ip_addr': '169.254.0.3', 'service_vlan_list': None, 'os_ifindex': '98', 'fabric_port': 'NotFabricPort', 'alloc_linklocal_ip': 'LL-Enable', 'analyzer_name': None, 'config_name': 'default-domain:ctest-TestDropStats-57016829:96b62f3e-9641-460b-8e9c-5045672b230d', 'sg_uuid_list': ['e5f47c7d-ef33-46e6-a770-a71a17cc56c1'], 'static_route_list': None, 'vm_project_uuid': 'c8280ec8-3916-4d47-844f-212e1318f1db', 'admin_state': 'Enabled', 'flow_key_idx': '35', 'allowed_address_pair_list': None, 'ip6_addr': '::', 'local_preference': '0', 'tx_vlan_id': '-1', 'rx_vlan_id': '-1', 'parent_interface': None, 'subnet': '--NA--', 'sub_type': 'Tap', 'vrf_assign_acl_uuid': '--NA--', 'vmi_type': 'Virtual Machine', 'transport': 'Ethernet', 'logical_interface_uuid': '00000000-0000-0000-0000-000000000000', 'flood_unknown_unicast': 'false', 'physical_device': None, 'physical_interface': None, 'fixed_ip4_list': None, 'fixed_ip6_list': None, 'fat_flow_list': None, 'metadata_ip_active': 'Active', 'service_health_check_ip': '0.0.0.0', 'alias_ip_list': None, 'drop_new_flows': 'false', 'bridge_domain_list': [], 'vmi_tag_list': None, 'policy_set_acl_list': None, 'slo_list': None, 'vhostuser_mode': '0', 'si_other_end_vmi': '00000000-0000-0000-0000-000000000000', 'cfg_igmp_enable': 'false', 'igmp_enabled': 'false', 'max_flows': '0', 'policy_set_fwaas_list': None, 'bond_interface_list': None, 'mac_ip_learning_enable': 'false', 'mac_ip_list': []} 2026-06-15 03:53:40,787 - DEBUG - Starting Layer 2 verification in Agent 2026-06-15 03:53:40,797 - DEBUG - Layer 2 path is seen for VM MAC 02:96:b6:2f:3e:96 in agent 10.0.0.130 2026-06-15 03:53:40,797 - DEBUG - Active layer 2 route in agent is present for VMI tap96b62f3e-96 2026-06-15 03:53:40,797 - DEBUG - L2 label(42) matches bw route table and interface table 2026-06-15 03:53:40,810 - DEBUG - VN default-domain:ctest-TestDropStats-57016829:ctest-vn-72875685 is found in Agent of node 10.0.0.130 2026-06-15 03:53:40,943 - DEBUG - VRF IDs of VN default-domain:ctest-TestDropStats-57016829:ctest-vn-72875685 is consistent in agent 10.0.0.130 2026-06-15 03:53:40,943 - DEBUG - Route for VM IP ['93.96.109.195'] is consistent in agent 10.0.0.130 2026-06-15 03:53:40,943 - DEBUG - VN default-domain:ctest-TestDropStats-57016829:ctest-vn-72875685 verification for VM ctest-vm2-99697257 in Agent 10.0.0.130 passed 2026-06-15 03:53:41,057 - DEBUG - Starting all layer 2 verification in agent 10.0.0.130 2026-06-15 03:53:41,068 - DEBUG - Route for VM MAC 02:96:b6:2f:3e:96 is consistent in agent 10.0.0.130 2026-06-15 03:53:41,068 - INFO - VM ctest-vm2-99697257 verifications in Compute nodes passed 2026-06-15 03:53:41,325 - DEBUG - Validated VM route 93.96.109.195 in vrouter of 10.0.0.130 2026-06-15 03:53:41,330 - DEBUG - Remote Nh validation passed. Expected Tunnel, label: None, 0; Got None,0 2026-06-15 03:53:41,331 - INFO - Validated routes of VM ctest-vm2-99697257 in all vrouters 2026-06-15 03:53:41,839 - DEBUG - Starting all layer2 verification in 10.0.0.65 Control Node 2026-06-15 03:53:41,972 - DEBUG - Layer2 route found for VM MAC 02:96:b6:2f:3e:96 in Control-node 10.0.0.65 2026-06-15 03:53:41,972 - DEBUG - L2 Label for VM ctest-vm2-99697257 same between Control-node 10.0.0.65 and Agent, Expected: 42, Seen: 42 2026-06-15 03:53:41,972 - DEBUG - Starting all layer2 verification in 10.0.0.33 Control Node 2026-06-15 03:53:42,094 - DEBUG - Layer2 route found for VM MAC 02:96:b6:2f:3e:96 in Control-node 10.0.0.33 2026-06-15 03:53:42,094 - DEBUG - L2 Label for VM ctest-vm2-99697257 same between Control-node 10.0.0.33 and Agent, Expected: 42, Seen: 42 2026-06-15 03:53:42,094 - INFO - Verification in Control-nodes for VM ctest-vm2-99697257 passed 2026-06-15 03:53:42,094 - DEBUG - Verifying the vm in opserver 2026-06-15 03:53:42,094 - DEBUG - Verifying in collector 10.0.0.65 ... 2026-06-15 03:53:42,094 - DEBUG - Requesting: http://10.0.0.65:8081/analytics/uves/virtual-machine/34afe8d3-5707-403c-93b9-e4fc43a1e3bb?flat 2026-06-15 03:53:42,112 - DEBUG - Failed to get VM ctest-vm2-99697257, ID 34afe8d3-5707-403c-93b9-e4fc43a1e3bb info from Opserver 2026-06-15 03:53:44,113 - DEBUG - Verifying the vm in opserver 2026-06-15 03:53:44,113 - DEBUG - Verifying in collector 10.0.0.65 ... 2026-06-15 03:53:44,113 - DEBUG - Requesting: http://10.0.0.65:8081/analytics/uves/virtual-machine/34afe8d3-5707-403c-93b9-e4fc43a1e3bb?flat 2026-06-15 03:53:44,127 - DEBUG - Failed to get VM ctest-vm2-99697257, ID 34afe8d3-5707-403c-93b9-e4fc43a1e3bb info from Opserver 2026-06-15 03:53:46,127 - DEBUG - Verifying the vm in opserver 2026-06-15 03:53:46,127 - DEBUG - Verifying in collector 10.0.0.65 ... 2026-06-15 03:53:46,128 - DEBUG - Requesting: http://10.0.0.65:8081/analytics/uves/virtual-machine/34afe8d3-5707-403c-93b9-e4fc43a1e3bb?flat 2026-06-15 03:53:46,142 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-machine-interface/default-domain:ctest-TestDropStats-57016829:96b62f3e-9641-460b-8e9c-5045672b230d?flat 2026-06-15 03:53:46,164 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-machine-interface/default-domain:ctest-TestDropStats-57016829:96b62f3e-9641-460b-8e9c-5045672b230d?flat 2026-06-15 03:53:46,182 - DEBUG - Requesting: http://10.0.0.65:8081/analytics/uves/virtual-machine/34afe8d3-5707-403c-93b9-e4fc43a1e3bb?flat 2026-06-15 03:53:46,200 - DEBUG - Verifying vm in vn uve 2026-06-15 03:53:46,200 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-machine-interface/default-domain:ctest-TestDropStats-57016829:96b62f3e-9641-460b-8e9c-5045672b230d?flat 2026-06-15 03:53:46,221 - DEBUG - VM uve shows interface as {'tx_vlan': 65535, 'ip6_address': '::', 'l2_active': True, 'vm_uuid': '34afe8d3-5707-403c-93b9-e4fc43a1e3bb', 'port_mirror_enabled': False, 'is_health_check_active': True, 'active': True, 'virtual_network': 'default-domain:ctest-TestDropStats-57016829:ctest-vn-72875685', 'admin_state': True, '__T': 1781495625957933, 'ip4_active': True, 'uuid': '96b62f3e-9641-460b-8e9c-5045672b230d', 'ip_address': '93.96.109.195', 'label': 30, 'vn_uuid': '70953075-7218-47ff-8c7f-33f417cad04a', 'vhostuser_mode': 0, 'fixed_ip4_list': ['93.96.109.195'], 'vm_name': 'ctest-vm2-99697257', 'rx_vlan': 65535, 'gateway': '93.96.109.193', 'ip6_active': False, 'mac_address': '02:96:b6:2f:3e:96'} 2026-06-15 03:53:46,221 - DEBUG - VM uve shows ip address as ['93.96.109.195', '::'] 2026-06-15 03:53:46,221 - DEBUG - VM uve shows virtual network as default-domain:ctest-TestDropStats-57016829:ctest-vn-72875685 2026-06-15 03:53:46,221 - INFO - Verifying through opserver in 10.0.0.23 2026-06-15 03:53:46,221 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-TestDropStats-57016829:ctest-vn-72875685?flat 2026-06-15 03:53:46,244 - DEBUG - expected vm list 34afe8d3-5707-403c-93b9-e4fc43a1e3bb 2026-06-15 03:53:46,244 - DEBUG - Extracted vm list ['34afe8d3-5707-403c-93b9-e4fc43a1e3bb'] 2026-06-15 03:53:46,244 - INFO - VM 34afe8d3-5707-403c-93b9-e4fc43a1e3bb is present in default-domain:ctest-TestDropStats-57016829:ctest-vn-72875685 2026-06-15 03:53:46,244 - DEBUG - Verifying vm in vrouter uve 2026-06-15 03:53:46,244 - DEBUG - Getting info from collector 10.0.0.65.. 2026-06-15 03:53:46,244 - DEBUG - Requesting: http://10.0.0.65:8081/analytics/uves/virtual-machine/34afe8d3-5707-403c-93b9-e4fc43a1e3bb?flat 2026-06-15 03:53:46,260 - DEBUG - VM uve shows vrouter as ['an-jenkins-deploy-platform-ansible-os-5940-2'] 2026-06-15 03:53:46,260 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-5940-2:Compute:contrail-vrouter-agent:0?flat 2026-06-15 03:53:46,313 - DEBUG - Requesting: http://10.0.0.65:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-5940-2?flat 2026-06-15 03:53:46,343 - DEBUG - VM 34afe8d3-5707-403c-93b9-e4fc43a1e3bb is present in vrouter an-jenkins-deploy-platform-ansible-os-5940-2 uve 2026-06-15 03:53:46,343 - DEBUG - Validated that VM ctest-vm2-99697257 is in Vrouter an-jenkins-deploy-platform-ansible-os-5940-2 UVE 2026-06-15 03:53:46,343 - DEBUG - Verifying vm tap interface/vn in vrouter uve 2026-06-15 03:53:46,352 - DEBUG - Expected tap interface of VM uuid 34afe8d3-5707-403c-93b9-e4fc43a1e3bb is default-domain:ctest-TestDropStats-57016829:96b62f3e-9641-460b-8e9c-5045672b230d 2026-06-15 03:53:46,353 - DEBUG - Expected VN of VM uuid 34afe8d3-5707-403c-93b9-e4fc43a1e3bb is default-domain:ctest-TestDropStats-57016829:ctest-vn-72875685 2026-06-15 03:53:46,353 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-5940-2:Compute:contrail-vrouter-agent:0?flat 2026-06-15 03:53:46,415 - DEBUG - Requesting: http://10.0.0.65:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-5940-2?flat 2026-06-15 03:53:46,447 - DEBUG - VM 34afe8d3-5707-403c-93b9-e4fc43a1e3bb is present in vrouter an-jenkins-deploy-platform-ansible-os-5940-2 uve 2026-06-15 03:53:46,447 - INFO - tap interface default-domain:ctest-TestDropStats-57016829:96b62f3e-9641-460b-8e9c-5045672b230d of vm 34afe8d3-5707-403c-93b9-e4fc43a1e3bb is present in vrouter an-jenkins-deploy-platform-ansible-os-5940-2 uve 2026-06-15 03:53:46,447 - DEBUG - Connected network default-domain:ctest-TestDropStats-57016829:ctest-vn-72875685 of vm 34afe8d3-5707-403c-93b9-e4fc43a1e3bb is present in vrouter an-jenkins-deploy-platform-ansible-os-5940-2 uve 2026-06-15 03:53:46,447 - DEBUG - Verifying the 34afe8d3-5707-403c-93b9-e4fc43a1e3bb virtual network link through opserver 10.0.0.23 2026-06-15 03:53:46,447 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-machines 2026-06-15 03:53:46,464 - DEBUG - VM link and name as {'name': '34afe8d3-5707-403c-93b9-e4fc43a1e3bb', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-machine/34afe8d3-5707-403c-93b9-e4fc43a1e3bb?flat'} 2026-06-15 03:53:46,464 - DEBUG - Verifying the 34afe8d3-5707-403c-93b9-e4fc43a1e3bb virtual network link through opserver 10.0.0.33 2026-06-15 03:53:46,465 - DEBUG - Requesting: http://10.0.0.33:8081/analytics/uves/virtual-machines 2026-06-15 03:53:46,531 - DEBUG - VM link and name as {'name': '34afe8d3-5707-403c-93b9-e4fc43a1e3bb', 'href': 'http://10.0.0.33:8081/analytics/uves/virtual-machine/34afe8d3-5707-403c-93b9-e4fc43a1e3bb?flat'} 2026-06-15 03:53:46,531 - DEBUG - Verifying the 34afe8d3-5707-403c-93b9-e4fc43a1e3bb virtual network link through opserver 10.0.0.65 2026-06-15 03:53:46,531 - DEBUG - Requesting: http://10.0.0.65:8081/analytics/uves/virtual-machines 2026-06-15 03:53:46,560 - DEBUG - VM link and name as {'name': '34afe8d3-5707-403c-93b9-e4fc43a1e3bb', 'href': 'http://10.0.0.65:8081/analytics/uves/virtual-machine/34afe8d3-5707-403c-93b9-e4fc43a1e3bb?flat'} 2026-06-15 03:53:46,560 - INFO - VM ctest-vm2-99697257 validations in Opserver passed 2026-06-15 03:53:46,593 - INFO - ---------------------------------------------------------------------------------------------------- 2026-06-15 03:53:46,593 - INFO - ds_discard : 2 2026-06-15 03:53:46,593 - INFO - ds_pull : 0 2026-06-15 03:53:46,594 - INFO - ds_invalid_if : 0 2026-06-15 03:53:46,594 - INFO - ds_invalid_arp : 0 2026-06-15 03:53:46,594 - INFO - ds_trap_no_if : 0 2026-06-15 03:53:46,594 - INFO - ds_nowhere_to_go : 0 2026-06-15 03:53:46,594 - INFO - ds_flow_queue_limit_exceeded : 0 2026-06-15 03:53:46,594 - INFO - ds_flow_no_memory : 0 2026-06-15 03:53:46,595 - INFO - ds_flow_invalid_protocol : 0 2026-06-15 03:53:46,595 - INFO - ds_flow_nat_no_rflow : 0 2026-06-15 03:53:46,595 - INFO - ds_flow_action_drop : 0 2026-06-15 03:53:46,595 - INFO - ds_flow_action_invalid : 0 2026-06-15 03:53:46,595 - INFO - ds_flow_unusable : 0 2026-06-15 03:53:46,595 - INFO - ds_flow_table_full : 0 2026-06-15 03:53:46,596 - INFO - ds_interface_tx_discard : 0 2026-06-15 03:53:46,596 - INFO - ds_interface_drop : 0 2026-06-15 03:53:46,596 - INFO - ds_duplicated : 0 2026-06-15 03:53:46,596 - INFO - ds_push : 0 2026-06-15 03:53:46,596 - INFO - ds_ttl_exceeded : 0 2026-06-15 03:53:46,596 - INFO - ds_invalid_nh : 0 2026-06-15 03:53:46,597 - INFO - ds_invalid_label : 0 2026-06-15 03:53:46,597 - INFO - ds_invalid_protocol : 0 2026-06-15 03:53:46,597 - INFO - ds_interface_rx_discard : 0 2026-06-15 03:53:46,597 - INFO - ds_invalid_mcast_source : 0 2026-06-15 03:53:46,597 - INFO - ds_head_alloc_fail : 0 2026-06-15 03:53:46,597 - INFO - ds_pcow_fail : 0 2026-06-15 03:53:46,597 - INFO - ds_mcast_clone_fail : 0 2026-06-15 03:53:46,598 - INFO - ds_rewrite_fail : 0 2026-06-15 03:53:46,598 - INFO - ds_misc : 0 2026-06-15 03:53:46,598 - INFO - ds_invalid_packet : 0 2026-06-15 03:53:46,598 - INFO - ds_cksum_err : 0 2026-06-15 03:53:46,598 - INFO - ds_no_fmd : 0 2026-06-15 03:53:46,598 - INFO - ds_invalid_vnid : 0 2026-06-15 03:53:46,599 - INFO - ds_frag_err : 0 2026-06-15 03:53:46,599 - INFO - ds_invalid_source : 0 2026-06-15 03:53:46,599 - INFO - ds_mcast_df_bit : 0 2026-06-15 03:53:46,599 - INFO - ds_l2_no_route : 1 2026-06-15 03:53:46,599 - INFO - ds_vlan_fwd_tx : 0 2026-06-15 03:53:46,599 - INFO - ds_vlan_fwd_enq : 0 2026-06-15 03:53:46,600 - INFO - ds_drop_new_flow : 0 2026-06-15 03:53:46,600 - INFO - ds_flow_evict : 0 2026-06-15 03:53:46,600 - INFO - ds_trap_original : 0 2026-06-15 03:53:46,600 - INFO - ds_no_memory : 0 2026-06-15 03:53:46,600 - INFO - ds_fragment_queue_fail : 0 2026-06-15 03:53:46,600 - INFO - ds_no_frag_entry : 0 2026-06-15 03:53:46,600 - INFO - ds_icmp_error : 0 2026-06-15 03:53:46,601 - INFO - ds_clone_fail : 0 2026-06-15 03:53:46,601 - INFO - ds_drop_pkts : 3 2026-06-15 03:53:46,610 - INFO - ---------------------------------------------------------------------------------------------------- 2026-06-15 03:53:46,610 - INFO - ds_discard : 8 2026-06-15 03:53:46,610 - INFO - ds_pull : 0 2026-06-15 03:53:46,610 - INFO - ds_invalid_if : 2 2026-06-15 03:53:46,610 - INFO - ds_invalid_arp : 0 2026-06-15 03:53:46,610 - INFO - ds_trap_no_if : 0 2026-06-15 03:53:46,611 - INFO - ds_nowhere_to_go : 0 2026-06-15 03:53:46,611 - INFO - ds_flow_queue_limit_exceeded : 0 2026-06-15 03:53:46,611 - INFO - ds_flow_no_memory : 0 2026-06-15 03:53:46,611 - INFO - ds_flow_invalid_protocol : 0 2026-06-15 03:53:46,611 - INFO - ds_flow_nat_no_rflow : 0 2026-06-15 03:53:46,611 - INFO - ds_flow_action_drop : 16 2026-06-15 03:53:46,611 - INFO - ds_flow_action_invalid : 0 2026-06-15 03:53:46,611 - INFO - ds_flow_unusable : 0 2026-06-15 03:53:46,611 - INFO - ds_flow_table_full : 0 2026-06-15 03:53:46,612 - INFO - ds_interface_tx_discard : 0 2026-06-15 03:53:46,612 - INFO - ds_interface_drop : 0 2026-06-15 03:53:46,612 - INFO - ds_duplicated : 0 2026-06-15 03:53:46,612 - INFO - ds_push : 0 2026-06-15 03:53:46,612 - INFO - ds_ttl_exceeded : 0 2026-06-15 03:53:46,612 - INFO - ds_invalid_nh : 3 2026-06-15 03:53:46,612 - INFO - ds_invalid_label : 0 2026-06-15 03:53:46,612 - INFO - ds_invalid_protocol : 0 2026-06-15 03:53:46,612 - INFO - ds_interface_rx_discard : 0 2026-06-15 03:53:46,612 - INFO - ds_invalid_mcast_source : 0 2026-06-15 03:53:46,613 - INFO - ds_head_alloc_fail : 0 2026-06-15 03:53:46,613 - INFO - ds_pcow_fail : 0 2026-06-15 03:53:46,613 - INFO - ds_mcast_clone_fail : 0 2026-06-15 03:53:46,613 - INFO - ds_rewrite_fail : 0 2026-06-15 03:53:46,613 - INFO - ds_misc : 0 2026-06-15 03:53:46,613 - INFO - ds_invalid_packet : 0 2026-06-15 03:53:46,613 - INFO - ds_cksum_err : 0 2026-06-15 03:53:46,614 - INFO - ds_no_fmd : 0 2026-06-15 03:53:46,614 - INFO - ds_invalid_vnid : 0 2026-06-15 03:53:46,614 - INFO - ds_frag_err : 0 2026-06-15 03:53:46,614 - INFO - ds_invalid_source : 0 2026-06-15 03:53:46,614 - INFO - ds_mcast_df_bit : 0 2026-06-15 03:53:46,614 - INFO - ds_l2_no_route : 2 2026-06-15 03:53:46,614 - INFO - ds_vlan_fwd_tx : 0 2026-06-15 03:53:46,614 - INFO - ds_vlan_fwd_enq : 0 2026-06-15 03:53:46,614 - INFO - ds_drop_new_flow : 0 2026-06-15 03:53:46,615 - INFO - ds_flow_evict : 0 2026-06-15 03:53:46,615 - INFO - ds_trap_original : 0 2026-06-15 03:53:46,615 - INFO - ds_no_memory : 0 2026-06-15 03:53:46,615 - INFO - ds_fragment_queue_fail : 0 2026-06-15 03:53:46,615 - INFO - ds_no_frag_entry : 0 2026-06-15 03:53:46,615 - INFO - ds_icmp_error : 0 2026-06-15 03:53:46,615 - INFO - ds_clone_fail : 0 2026-06-15 03:53:46,615 - INFO - ds_drop_pkts : 31 2026-06-15 03:53:46,615 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 5 -W 1 93.96.109.195, host_string: cirros@169.254.0.5, password: cubswin:)gateway: ubuntu@10.0.0.141, gateway password: c0ntrail123 2026-06-15 03:53:46,616 - DEBUG - ping -s 56 -c 5 -W 1 93.96.109.195 2026-06-15 03:53:52,581 - DEBUG - PING 93.96.109.195 (93.96.109.195): 56 data bytes --- 93.96.109.195 ping statistics --- 5 packets transmitted, 0 packets received, 100% packet loss 2026-06-15 03:53:52,581 - WARNING - Ping to IP 93.96.109.195 from VM ctest-vm1-47907004 failed 2026-06-15 03:54:08,738 - INFO - Vif and vrouter dropstats of type ds_flow_action_drop verifed 2026-06-15 03:54:08,738 - INFO - ---------------------------------------------------------------------------------------------------- 2026-06-15 03:54:08,738 - INFO - ds_discard : 3 2026-06-15 03:54:08,739 - INFO - ds_pull : 0 2026-06-15 03:54:08,739 - INFO - ds_invalid_if : 0 2026-06-15 03:54:08,739 - INFO - ds_invalid_arp : 0 2026-06-15 03:54:08,739 - INFO - ds_trap_no_if : 0 2026-06-15 03:54:08,739 - INFO - ds_nowhere_to_go : 0 2026-06-15 03:54:08,739 - INFO - ds_flow_queue_limit_exceeded : 0 2026-06-15 03:54:08,739 - INFO - ds_flow_no_memory : 0 2026-06-15 03:54:08,739 - INFO - ds_flow_invalid_protocol : 0 2026-06-15 03:54:08,739 - INFO - ds_flow_nat_no_rflow : 0 2026-06-15 03:54:08,739 - INFO - ds_flow_action_drop : 5 2026-06-15 03:54:08,739 - INFO - ds_flow_action_invalid : 0 2026-06-15 03:54:08,739 - INFO - ds_flow_unusable : 0 2026-06-15 03:54:08,739 - INFO - ds_flow_table_full : 0 2026-06-15 03:54:08,739 - INFO - ds_interface_tx_discard : 0 2026-06-15 03:54:08,739 - INFO - ds_interface_drop : 0 2026-06-15 03:54:08,739 - INFO - ds_duplicated : 0 2026-06-15 03:54:08,739 - INFO - ds_push : 0 2026-06-15 03:54:08,740 - INFO - ds_ttl_exceeded : 0 2026-06-15 03:54:08,740 - INFO - ds_invalid_nh : 0 2026-06-15 03:54:08,740 - INFO - ds_invalid_label : 0 2026-06-15 03:54:08,740 - INFO - ds_invalid_protocol : 0 2026-06-15 03:54:08,740 - INFO - ds_interface_rx_discard : 0 2026-06-15 03:54:08,740 - INFO - ds_invalid_mcast_source : 0 2026-06-15 03:54:08,740 - INFO - ds_head_alloc_fail : 0 2026-06-15 03:54:08,740 - INFO - ds_pcow_fail : 0 2026-06-15 03:54:08,740 - INFO - ds_mcast_clone_fail : 0 2026-06-15 03:54:08,740 - INFO - ds_rewrite_fail : 0 2026-06-15 03:54:08,740 - INFO - ds_misc : 0 2026-06-15 03:54:08,740 - INFO - ds_invalid_packet : 0 2026-06-15 03:54:08,740 - INFO - ds_cksum_err : 0 2026-06-15 03:54:08,740 - INFO - ds_no_fmd : 0 2026-06-15 03:54:08,740 - INFO - ds_invalid_vnid : 0 2026-06-15 03:54:08,740 - INFO - ds_frag_err : 0 2026-06-15 03:54:08,740 - INFO - ds_invalid_source : 0 2026-06-15 03:54:08,740 - INFO - ds_mcast_df_bit : 0 2026-06-15 03:54:08,740 - INFO - ds_l2_no_route : 1 2026-06-15 03:54:08,740 - INFO - ds_vlan_fwd_tx : 0 2026-06-15 03:54:08,740 - INFO - ds_vlan_fwd_enq : 0 2026-06-15 03:54:08,740 - INFO - ds_drop_new_flow : 0 2026-06-15 03:54:08,741 - INFO - ds_flow_evict : 0 2026-06-15 03:54:08,741 - INFO - ds_trap_original : 0 2026-06-15 03:54:08,741 - INFO - ds_no_memory : 0 2026-06-15 03:54:08,741 - INFO - ds_fragment_queue_fail : 0 2026-06-15 03:54:08,741 - INFO - ds_no_frag_entry : 0 2026-06-15 03:54:08,741 - INFO - ds_icmp_error : 0 2026-06-15 03:54:08,741 - INFO - ds_clone_fail : 0 2026-06-15 03:54:08,741 - INFO - ds_drop_pkts : 9 2026-06-15 03:54:08,741 - INFO - ---------------------------------------------------------------------------------------------------- 2026-06-15 03:54:08,741 - INFO - ds_discard : 9 2026-06-15 03:54:08,741 - INFO - ds_pull : 0 2026-06-15 03:54:08,741 - INFO - ds_invalid_if : 2 2026-06-15 03:54:08,741 - INFO - ds_invalid_arp : 0 2026-06-15 03:54:08,741 - INFO - ds_trap_no_if : 0 2026-06-15 03:54:08,741 - INFO - ds_nowhere_to_go : 0 2026-06-15 03:54:08,741 - INFO - ds_flow_queue_limit_exceeded : 0 2026-06-15 03:54:08,741 - INFO - ds_flow_no_memory : 0 2026-06-15 03:54:08,741 - INFO - ds_flow_invalid_protocol : 0 2026-06-15 03:54:08,741 - INFO - ds_flow_nat_no_rflow : 0 2026-06-15 03:54:08,741 - INFO - ds_flow_action_drop : 21 2026-06-15 03:54:08,741 - INFO - ds_flow_action_invalid : 0 2026-06-15 03:54:08,741 - INFO - ds_flow_unusable : 0 2026-06-15 03:54:08,741 - INFO - ds_flow_table_full : 0 2026-06-15 03:54:08,741 - INFO - ds_interface_tx_discard : 0 2026-06-15 03:54:08,741 - INFO - ds_interface_drop : 0 2026-06-15 03:54:08,741 - INFO - ds_duplicated : 0 2026-06-15 03:54:08,741 - INFO - ds_push : 0 2026-06-15 03:54:08,742 - INFO - ds_ttl_exceeded : 0 2026-06-15 03:54:08,742 - INFO - ds_invalid_nh : 3 2026-06-15 03:54:08,742 - INFO - ds_invalid_label : 0 2026-06-15 03:54:08,742 - INFO - ds_invalid_protocol : 0 2026-06-15 03:54:08,742 - INFO - ds_interface_rx_discard : 0 2026-06-15 03:54:08,742 - INFO - ds_invalid_mcast_source : 0 2026-06-15 03:54:08,742 - INFO - ds_head_alloc_fail : 0 2026-06-15 03:54:08,742 - INFO - ds_pcow_fail : 0 2026-06-15 03:54:08,742 - INFO - ds_mcast_clone_fail : 0 2026-06-15 03:54:08,742 - INFO - ds_rewrite_fail : 0 2026-06-15 03:54:08,742 - INFO - ds_misc : 0 2026-06-15 03:54:08,742 - INFO - ds_invalid_packet : 0 2026-06-15 03:54:08,742 - INFO - ds_cksum_err : 0 2026-06-15 03:54:08,742 - INFO - ds_no_fmd : 0 2026-06-15 03:54:08,742 - INFO - ds_invalid_vnid : 0 2026-06-15 03:54:08,742 - INFO - ds_frag_err : 0 2026-06-15 03:54:08,742 - INFO - ds_invalid_source : 0 2026-06-15 03:54:08,742 - INFO - ds_mcast_df_bit : 0 2026-06-15 03:54:08,742 - INFO - ds_l2_no_route : 2 2026-06-15 03:54:08,742 - INFO - ds_vlan_fwd_tx : 0 2026-06-15 03:54:08,742 - INFO - ds_vlan_fwd_enq : 0 2026-06-15 03:54:08,743 - INFO - ds_drop_new_flow : 0 2026-06-15 03:54:08,743 - INFO - ds_flow_evict : 0 2026-06-15 03:54:08,743 - INFO - ds_trap_original : 0 2026-06-15 03:54:08,743 - INFO - ds_no_memory : 0 2026-06-15 03:54:08,743 - INFO - ds_fragment_queue_fail : 0 2026-06-15 03:54:08,743 - INFO - ds_no_frag_entry : 0 2026-06-15 03:54:08,743 - INFO - ds_icmp_error : 0 2026-06-15 03:54:08,743 - INFO - ds_clone_fail : 0 2026-06-15 03:54:08,743 - INFO - ds_drop_pkts : 37 2026-06-15 03:54:08,743 - INFO - Deleting VM ctest-vm2-99697257 2026-06-15 03:54:08,822 - DEBUG - Verifying in api server 10.0.0.65 2026-06-15 03:54:08,822 - DEBUG - Requesting: http://10.0.0.65:8082/virtual-machine/34afe8d3-5707-403c-93b9-e4fc43a1e3bb 2026-06-15 03:54:08,830 - DEBUG - VM ID 34afe8d3-5707-403c-93b9-e4fc43a1e3bb of VM ctest-vm2-99697257 is still found in API Server 2026-06-15 03:54:10,832 - DEBUG - Verifying in api server 10.0.0.65 2026-06-15 03:54:10,832 - DEBUG - Requesting: http://10.0.0.65:8082/virtual-machine/34afe8d3-5707-403c-93b9-e4fc43a1e3bb 2026-06-15 03:54:10,841 - DEBUG - VM ID 34afe8d3-5707-403c-93b9-e4fc43a1e3bb of VM ctest-vm2-99697257 is still found in API Server 2026-06-15 03:54:12,842 - DEBUG - Verifying in api server 10.0.0.65 2026-06-15 03:54:12,842 - DEBUG - Requesting: http://10.0.0.65:8082/virtual-machine/34afe8d3-5707-403c-93b9-e4fc43a1e3bb 2026-06-15 03:54:12,850 - DEBUG - Response Code: 404 2026-06-15 03:54:12,850 - DEBUG - Requesting: http://10.0.0.65:8082/virtual-machine/34afe8d3-5707-403c-93b9-e4fc43a1e3bb 2026-06-15 03:54:12,857 - DEBUG - Response Code: 404 2026-06-15 03:54:12,857 - DEBUG - Requesting: http://10.0.0.65:8082/virtual-machine/34afe8d3-5707-403c-93b9-e4fc43a1e3bb 2026-06-15 03:54:12,865 - DEBUG - Response Code: 404 2026-06-15 03:54:12,865 - INFO - VM ctest-vm2-99697257 is fully removed in API-Server 2026-06-15 03:54:13,023 - DEBUG - Skipping VM-deletion-check in nova db since mysql_token is not found 2026-06-15 03:54:13,067 - DEBUG - Validated that vrouter 10.0.0.141 does not have VMs route for VN default-domain:ctest-TestDropStats-57016829:ctest-vn-72875685 2026-06-15 03:54:13,078 - DEBUG - Validated that vrouter 10.0.0.130 does not have VMs route for VN default-domain:ctest-TestDropStats-57016829:ctest-vn-72875685 2026-06-15 03:54:13,078 - INFO - Validated that all vrouters do not have VMs route for VN default-domain:ctest-TestDropStats-57016829:ctest-vn-72875685 2026-06-15 03:54:13,079 - INFO - VM ctest-vm2-99697257 is removed in Compute, and routes are removed in all compute nodes 2026-06-15 03:54:13,305 - INFO - Routes for VM ctest-vm2-99697257 is removed in all control-nodes 2026-06-15 03:54:13,305 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-5940-2:Compute:contrail-vrouter-agent:0?flat 2026-06-15 03:54:13,357 - DEBUG - Requesting: http://10.0.0.65:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-5940-2?flat 2026-06-15 03:54:13,384 - DEBUG - VM 34afe8d3-5707-403c-93b9-e4fc43a1e3bb is still present in vrouter an-jenkins-deploy-platform-ansible-os-5940-2 uve 2026-06-15 03:54:13,384 - DEBUG - Verifying if the vm interface deleted from vroter uve... 2026-06-15 03:54:13,384 - DEBUG - interface for vm 34afe8d3-5707-403c-93b9-e4fc43a1e3bb deleted from vrouter uve ... 2026-06-15 03:54:13,384 - DEBUG - Deleted interface not in error interface list ... 2026-06-15 03:54:16,385 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-5940-2:Compute:contrail-vrouter-agent:0?flat 2026-06-15 03:54:16,437 - DEBUG - Requesting: http://10.0.0.65:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-5940-2?flat 2026-06-15 03:54:16,473 - DEBUG - VM 34afe8d3-5707-403c-93b9-e4fc43a1e3bb is not present in vrouter an-jenkins-deploy-platform-ansible-os-5940-2 uve 2026-06-15 03:54:16,473 - DEBUG - Verifying if the vm interface deleted from vroter uve... 2026-06-15 03:54:16,473 - DEBUG - interface for vm 34afe8d3-5707-403c-93b9-e4fc43a1e3bb deleted from vrouter uve ... 2026-06-15 03:54:16,473 - DEBUG - Deleted interface not in error interface list ... 2026-06-15 03:54:16,473 - DEBUG - Validated that VM 34afe8d3-5707-403c-93b9-e4fc43a1e3bb is removed in Vrouter UVE 2026-06-15 03:54:16,473 - DEBUG - Verifying through opserver in 10.0.0.23 2026-06-15 03:54:16,473 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-TestDropStats-57016829:ctest-vn-72875685?flat 2026-06-15 03:54:16,493 - DEBUG - 34afe8d3-5707-403c-93b9-e4fc43a1e3bb BM not in default-domain:ctest-TestDropStats-57016829:ctest-vn-72875685 uve 2026-06-15 03:54:16,493 - DEBUG - Validated that VM 34afe8d3-5707-403c-93b9-e4fc43a1e3bb is not present in the VN UVE 2026-06-15 03:54:16,493 - DEBUG - Verifying the 34afe8d3-5707-403c-93b9-e4fc43a1e3bb virtual network link through opserver 10.0.0.23 2026-06-15 03:54:16,493 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-machines 2026-06-15 03:54:16,508 - INFO - 34afe8d3-5707-403c-93b9-e4fc43a1e3bb vm uve deleted from opserver 2026-06-15 03:54:16,509 - DEBUG - Verifying the 34afe8d3-5707-403c-93b9-e4fc43a1e3bb virtual network link through opserver 10.0.0.33 2026-06-15 03:54:16,509 - DEBUG - Requesting: http://10.0.0.33:8081/analytics/uves/virtual-machines 2026-06-15 03:54:16,558 - INFO - 34afe8d3-5707-403c-93b9-e4fc43a1e3bb vm uve deleted from opserver 2026-06-15 03:54:16,558 - DEBUG - Verifying the 34afe8d3-5707-403c-93b9-e4fc43a1e3bb virtual network link through opserver 10.0.0.65 2026-06-15 03:54:16,558 - DEBUG - Requesting: http://10.0.0.65:8081/analytics/uves/virtual-machines 2026-06-15 03:54:16,571 - INFO - 34afe8d3-5707-403c-93b9-e4fc43a1e3bb vm uve deleted from opserver 2026-06-15 03:54:16,571 - INFO - Validated that VM 34afe8d3-5707-403c-93b9-e4fc43a1e3bb is removed from Opserver 2026-06-15 03:54:16,571 - INFO - Deleting VM ctest-vm1-47907004 2026-06-15 03:54:16,662 - DEBUG - Verifying in api server 10.0.0.65 2026-06-15 03:54:16,662 - DEBUG - Requesting: http://10.0.0.65:8082/virtual-machine/b987a1cd-ef58-4565-9f6b-90d220408483 2026-06-15 03:54:16,670 - DEBUG - VM ID b987a1cd-ef58-4565-9f6b-90d220408483 of VM ctest-vm1-47907004 is still found in API Server 2026-06-15 03:54:18,671 - DEBUG - Verifying in api server 10.0.0.65 2026-06-15 03:54:18,671 - DEBUG - Requesting: http://10.0.0.65:8082/virtual-machine/b987a1cd-ef58-4565-9f6b-90d220408483 2026-06-15 03:54:18,697 - DEBUG - VM ID b987a1cd-ef58-4565-9f6b-90d220408483 of VM ctest-vm1-47907004 is still found in API Server 2026-06-15 03:54:20,697 - DEBUG - Verifying in api server 10.0.0.65 2026-06-15 03:54:20,698 - DEBUG - Requesting: http://10.0.0.65:8082/virtual-machine/b987a1cd-ef58-4565-9f6b-90d220408483 2026-06-15 03:54:20,704 - DEBUG - Response Code: 404 2026-06-15 03:54:20,704 - DEBUG - Requesting: http://10.0.0.65:8082/virtual-machine/b987a1cd-ef58-4565-9f6b-90d220408483 2026-06-15 03:54:20,711 - DEBUG - Response Code: 404 2026-06-15 03:54:20,711 - DEBUG - Requesting: http://10.0.0.65:8082/virtual-machine/b987a1cd-ef58-4565-9f6b-90d220408483 2026-06-15 03:54:20,716 - DEBUG - Response Code: 404 2026-06-15 03:54:20,716 - INFO - VM ctest-vm1-47907004 is fully removed in API-Server 2026-06-15 03:54:20,868 - DEBUG - Skipping VM-deletion-check in nova db since mysql_token is not found 2026-06-15 03:54:20,906 - DEBUG - Validated that vrouter 10.0.0.141 does not have VMs route for VN default-domain:ctest-TestDropStats-57016829:ctest-vn-01937378 2026-06-15 03:54:20,918 - DEBUG - Validated that vrouter 10.0.0.130 does not have VMs route for VN default-domain:ctest-TestDropStats-57016829:ctest-vn-01937378 2026-06-15 03:54:20,918 - INFO - Validated that all vrouters do not have VMs route for VN default-domain:ctest-TestDropStats-57016829:ctest-vn-01937378 2026-06-15 03:54:20,918 - INFO - VM ctest-vm1-47907004 is removed in Compute, and routes are removed in all compute nodes 2026-06-15 03:54:21,157 - INFO - Routes for VM ctest-vm1-47907004 is removed in all control-nodes 2026-06-15 03:54:21,157 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-5940-1:Compute:contrail-vrouter-agent:0?flat 2026-06-15 03:54:21,204 - DEBUG - Requesting: http://10.0.0.65:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-5940-1?flat 2026-06-15 03:54:21,235 - DEBUG - VM b987a1cd-ef58-4565-9f6b-90d220408483 is still present in vrouter an-jenkins-deploy-platform-ansible-os-5940-1 uve 2026-06-15 03:54:21,235 - DEBUG - Verifying if the vm interface deleted from vroter uve... 2026-06-15 03:54:21,235 - DEBUG - interface for vm b987a1cd-ef58-4565-9f6b-90d220408483 deleted from vrouter uve ... 2026-06-15 03:54:21,235 - DEBUG - Deleted interface not in error interface list ... 2026-06-15 03:54:24,236 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-5940-1:Compute:contrail-vrouter-agent:0?flat 2026-06-15 03:54:24,267 - DEBUG - Requesting: http://10.0.0.65:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-5940-1?flat 2026-06-15 03:54:24,297 - DEBUG - VM b987a1cd-ef58-4565-9f6b-90d220408483 is still present in vrouter an-jenkins-deploy-platform-ansible-os-5940-1 uve 2026-06-15 03:54:24,297 - DEBUG - Verifying if the vm interface deleted from vroter uve... 2026-06-15 03:54:24,297 - DEBUG - interface for vm b987a1cd-ef58-4565-9f6b-90d220408483 deleted from vrouter uve ... 2026-06-15 03:54:24,297 - DEBUG - Deleted interface not in error interface list ... 2026-06-15 03:54:27,297 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-5940-1:Compute:contrail-vrouter-agent:0?flat 2026-06-15 03:54:27,346 - DEBUG - Requesting: http://10.0.0.65:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-5940-1?flat 2026-06-15 03:54:27,384 - DEBUG - VM b987a1cd-ef58-4565-9f6b-90d220408483 is still present in vrouter an-jenkins-deploy-platform-ansible-os-5940-1 uve 2026-06-15 03:54:27,384 - DEBUG - Verifying if the vm interface deleted from vroter uve... 2026-06-15 03:54:27,384 - DEBUG - interface for vm b987a1cd-ef58-4565-9f6b-90d220408483 deleted from vrouter uve ... 2026-06-15 03:54:27,384 - DEBUG - Deleted interface not in error interface list ... 2026-06-15 03:54:30,384 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-5940-1:Compute:contrail-vrouter-agent:0?flat 2026-06-15 03:54:30,416 - DEBUG - Requesting: http://10.0.0.65:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-5940-1?flat 2026-06-15 03:54:30,445 - DEBUG - VM b987a1cd-ef58-4565-9f6b-90d220408483 is still present in vrouter an-jenkins-deploy-platform-ansible-os-5940-1 uve 2026-06-15 03:54:30,445 - DEBUG - Verifying if the vm interface deleted from vroter uve... 2026-06-15 03:54:30,445 - DEBUG - interface for vm b987a1cd-ef58-4565-9f6b-90d220408483 deleted from vrouter uve ... 2026-06-15 03:54:30,445 - DEBUG - Deleted interface not in error interface list ... 2026-06-15 03:54:33,446 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-5940-1:Compute:contrail-vrouter-agent:0?flat 2026-06-15 03:54:33,478 - DEBUG - Requesting: http://10.0.0.65:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-5940-1?flat 2026-06-15 03:54:33,506 - DEBUG - VM b987a1cd-ef58-4565-9f6b-90d220408483 is still present in vrouter an-jenkins-deploy-platform-ansible-os-5940-1 uve 2026-06-15 03:54:33,507 - DEBUG - Verifying if the vm interface deleted from vroter uve... 2026-06-15 03:54:33,507 - DEBUG - interface for vm b987a1cd-ef58-4565-9f6b-90d220408483 deleted from vrouter uve ... 2026-06-15 03:54:33,507 - DEBUG - Deleted interface not in error interface list ... 2026-06-15 03:54:36,507 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-5940-1:Compute:contrail-vrouter-agent:0?flat 2026-06-15 03:54:36,540 - DEBUG - Requesting: http://10.0.0.65:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-5940-1?flat 2026-06-15 03:54:36,567 - DEBUG - VM b987a1cd-ef58-4565-9f6b-90d220408483 is still present in vrouter an-jenkins-deploy-platform-ansible-os-5940-1 uve 2026-06-15 03:54:36,567 - DEBUG - Verifying if the vm interface deleted from vroter uve... 2026-06-15 03:54:36,567 - DEBUG - interface for vm b987a1cd-ef58-4565-9f6b-90d220408483 deleted from vrouter uve ... 2026-06-15 03:54:36,567 - DEBUG - Deleted interface not in error interface list ... 2026-06-15 03:54:39,568 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-5940-1:Compute:contrail-vrouter-agent:0?flat 2026-06-15 03:54:39,605 - DEBUG - Requesting: http://10.0.0.65:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-5940-1?flat 2026-06-15 03:54:39,640 - DEBUG - VM b987a1cd-ef58-4565-9f6b-90d220408483 is not present in vrouter an-jenkins-deploy-platform-ansible-os-5940-1 uve 2026-06-15 03:54:39,640 - DEBUG - Verifying if the vm interface deleted from vroter uve... 2026-06-15 03:54:39,640 - DEBUG - interface for vm b987a1cd-ef58-4565-9f6b-90d220408483 deleted from vrouter uve ... 2026-06-15 03:54:39,640 - DEBUG - Deleted interface not in error interface list ... 2026-06-15 03:54:39,640 - DEBUG - Validated that VM b987a1cd-ef58-4565-9f6b-90d220408483 is removed in Vrouter UVE 2026-06-15 03:54:39,640 - DEBUG - Verifying through opserver in 10.0.0.23 2026-06-15 03:54:39,640 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-TestDropStats-57016829:ctest-vn-01937378?flat 2026-06-15 03:54:39,659 - DEBUG - b987a1cd-ef58-4565-9f6b-90d220408483 BM not in default-domain:ctest-TestDropStats-57016829:ctest-vn-01937378 uve 2026-06-15 03:54:39,659 - DEBUG - Validated that VM b987a1cd-ef58-4565-9f6b-90d220408483 is not present in the VN UVE 2026-06-15 03:54:39,659 - DEBUG - Verifying the b987a1cd-ef58-4565-9f6b-90d220408483 virtual network link through opserver 10.0.0.23 2026-06-15 03:54:39,659 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-machines 2026-06-15 03:54:39,675 - INFO - b987a1cd-ef58-4565-9f6b-90d220408483 vm uve deleted from opserver 2026-06-15 03:54:39,675 - DEBUG - Verifying the b987a1cd-ef58-4565-9f6b-90d220408483 virtual network link through opserver 10.0.0.33 2026-06-15 03:54:39,675 - DEBUG - Requesting: http://10.0.0.33:8081/analytics/uves/virtual-machines 2026-06-15 03:54:39,688 - INFO - b987a1cd-ef58-4565-9f6b-90d220408483 vm uve deleted from opserver 2026-06-15 03:54:39,688 - DEBUG - Verifying the b987a1cd-ef58-4565-9f6b-90d220408483 virtual network link through opserver 10.0.0.65 2026-06-15 03:54:39,689 - DEBUG - Requesting: http://10.0.0.65:8081/analytics/uves/virtual-machines 2026-06-15 03:54:39,703 - INFO - b987a1cd-ef58-4565-9f6b-90d220408483 vm uve deleted from opserver 2026-06-15 03:54:39,703 - INFO - Validated that VM b987a1cd-ef58-4565-9f6b-90d220408483 is removed from Opserver 2026-06-15 03:54:39,704 - INFO - Deleting VN ctest-vn-72875685 2026-06-15 03:54:39,859 - DEBUG - Response for deleting network () 2026-06-15 03:54:39,859 - INFO - Deleting VN ctest-vn-01937378 2026-06-15 03:54:40,069 - DEBUG - Response for deleting network () 2026-06-15 03:54:41,333 - DEBUG - Skipping xmpp flap check
2026-06-15 03:54:41,334 - INFO - END TEST : test_flow_action_drop_stats : PASSED[0:01:28]
2026-06-15 03:54:41,334 - INFO - -------------------------------------------------------------------------------- 2026-06-15 03:54:42,091 - INFO - Deleted project: ctest-TestDropStats-57016829, ID : c8280ec8-3916-4d47-844f-212e1318f1db