2026-03-19 14:26:17,437 - DEBUG - [10.0.0.56]: Running cmd : hostname 2026-03-19 14:26:17,701 - DEBUG - Output : cn-jenkins-deploy-platform-juju-k8s-1938-1 2026-03-19 14:26:17,701 - DEBUG - [10.0.0.56]: Running cmd : hostname -f 2026-03-19 14:26:17,736 - DEBUG - Output : cn-jenkins-deploy-platform-juju-k8s-1938-1 2026-03-19 14:26:17,736 - DEBUG - [10.0.0.56]: Running cmd : docker ps 2>/dev/null | grep -v "/pause\|/usr/bin/pod\|nova_api_\|contrail.*init\|init.*contrail\|provisioner\|placement" | awk '{print $NF}' 2026-03-19 14:26:17,844 - DEBUG - Output : NAMES contrail_test_kh6D5Xqc7 contrail-kubernetes-master-kubemanager-1 analytics_database-query-engine-1 analytics_database-nodemgr-1 analytics_database-cassandra-1 analytics_snmp-snmp-collector-1 analytics_snmp-topology-1 analytics_snmp-nodemgr-1 analytics_alarm-kafka-1 analytics_alarm-alarm-gen-1 analytics_alarm-nodemgr-1 analytics-api-1 analytics-nodemgr-1 analytics-collector-1 redis-redis-1 webui-web-1 webui-job-1 control-named-1 control-dns-1 control-nodemgr-1 control-control-1 config_database-cassandra-1 config_database-zookeeper-1 config_database-nodemgr-1 config_database-rabbitmq-1 config_api-devicemgr-1 config_api-dnsmasq-1 config_api-nodemgr-1 config_api-schema-1 config_api-svcmonitor-1 config_api-api-1 2026-03-19 14:26:17,844 - DEBUG - [10.0.0.56]: Running cmd : ip -4 -o addr show | awk '{print $4}' 2026-03-19 14:26:17,877 - DEBUG - Output : 127.0.0.1/8 10.0.0.56/24 252.56.0.1/8 10.193.211.1/24 172.17.0.1/16 2026-03-19 14:26:17,877 - DEBUG - [10.0.0.56]: Running cmd : getent hosts 10.0.0.56 | head -n 1 | awk '{print $2}' 2026-03-19 14:26:17,910 - DEBUG - Output : cn-jenkins-deploy-platform-juju-k8s-1938-1 2026-03-19 14:26:18,004 - DEBUG - [10.0.0.61]: Running cmd : hostname 2026-03-19 14:26:18,243 - DEBUG - Output : an-jenkins-deploy-platform-juju-k8s-1938-1 2026-03-19 14:26:18,243 - DEBUG - [10.0.0.61]: Running cmd : hostname -f 2026-03-19 14:26:18,268 - DEBUG - Output : an-jenkins-deploy-platform-juju-k8s-1938-1 2026-03-19 14:26:18,268 - DEBUG - [10.0.0.61]: Running cmd : docker ps 2>/dev/null | grep -v "/pause\|/usr/bin/pod\|nova_api_\|contrail.*init\|init.*contrail\|provisioner\|placement" | awk '{print $NF}' 2026-03-19 14:26:18,353 - DEBUG - Output : NAMES vrouter-vrouter-agent-1 vrouter-nodemgr-1 2026-03-19 14:26:18,353 - DEBUG - [10.0.0.61]: Running cmd : ip -4 -o addr show dev vhost0 | awk '{print $4}' 2026-03-19 14:26:18,387 - DEBUG - Output : 10.0.0.61/24 2026-03-19 14:26:18,387 - DEBUG - [10.0.0.61]: Running cmd : getent hosts 10.0.0.61 | head -n 1 | awk '{print $2}' 2026-03-19 14:26:18,422 - DEBUG - Output : an-jenkins-deploy-platform-juju-k8s-1938-1 2026-03-19 14:26:18,501 - INFO - Using existing project ['default-domain', 'k8s-default'](958a78a1-43a0-4463-92b1-1a98c02e74fe) 2026-03-19 14:26:18,611 - INFO - Using existing project ['default-domain', 'k8s-default'](958a78a1-43a0-4463-92b1-1a98c02e74fe) 2026-03-19 14:26:18,719 - INFO - ================================================================================ 2026-03-19 14:26:18,720 - INFO - STARTING TEST : test_ingress_ip_assignment 2026-03-19 14:26:18,720 - INFO - TEST DESCRIPTION : Verify that Ingress gets a CLuster IP which is reachable to Pods in same namespace. Also verify that a Floating IP is assigned to the Ingress from the Public FIP poo. Steps: 1. Create a service with 2 pods running nginx 2. Create an ingress out of this service 3. From another Pod do a wget on the ingress Cluster ip Validate that Ingress get a IP from Public FIP pool which might/might not be accessible. Validate that service and its loadbalancing work 2026-03-19 14:26:18,964 - DEBUG - Requesting: http://10.0.0.61:8085/Snh_AgentXmppConnectionStatusReq? 2026-03-19 14:26:18,972 - DEBUG - Nothing to compare xmpp stats {'10.0.0.61': {'10.0.0.56': '0'}} with 2026-03-19 14:26:18,972 - INFO - Initial checks done. Running the testcase now 2026-03-19 14:26:18,972 - INFO - 2026-03-19 14:26:18,976 - INFO - Namespace default already exists 2026-03-19 14:26:18,976 - DEBUG - [10.0.0.56]: Running cmd : docker exec --privileged -it contrail-kubernetes-master-kubemanager-1 /bin/bash -c 'grep "^[ ]*cluster_project" /etc/contrail/contrail-kubernetes.conf' 2026-03-19 14:26:19,082 - DEBUG - Output : cluster_project={} 2026-03-19 14:26:19,123 - INFO - Project default-domain:k8s-default for Namespace default is seen in contrail-api 2026-03-19 14:26:19,123 - DEBUG - Project uuid in Contrail is 958a78a1-43a0-4463-92b1-1a98c02e74fe 2026-03-19 14:26:19,124 - DEBUG - [10.0.0.56]: Running cmd : docker exec --privileged -it contrail-kubernetes-master-kubemanager-1 /bin/bash -c 'netstat -antp | grep :8108 | grep LISTEN' 2026-03-19 14:26:19,256 - DEBUG - Output : tcp 0 0 0.0.0.0:8108 0.0.0.0:* LISTEN 1/python3 2026-03-19 14:26:19,256 - DEBUG - Requesting: http://10.0.0.56:8108/Snh_NamespaceDatabaseList?namespace_uuid=e6991feb-43aa-4fcc-9f29-2a19c48e0d54 2026-03-19 14:26:19,261 - INFO - Namespace default with uuid e6991feb-43aa-4fcc-9f29-2a19c48e0d54 found in kube manager 2026-03-19 14:26:19,261 - INFO - Namespace default verification passed 2026-03-19 14:26:19,264 - DEBUG - Service ctest-nginx-svc-20069857 not present 2026-03-19 14:26:19,264 - INFO - Creating service ctest-nginx-svc-20069857 2026-03-19 14:26:19,287 - INFO - Created Service ctest-nginx-svc-20069857 2026-03-19 14:26:19,299 - DEBUG - Pod ctest-nginx-pod-38883926 not present 2026-03-19 14:26:19,300 - INFO - Creating Pod ctest-nginx-pod-38883926 2026-03-19 14:26:19,367 - DEBUG - Pod : ctest-nginx-pod-38883926 UUID is 631bab7a-8194-417d-a5f2-241e154b049b 2026-03-19 14:26:19,371 - DEBUG - Pod ctest-nginx-pod-98063544 not present 2026-03-19 14:26:19,371 - INFO - Creating Pod ctest-nginx-pod-98063544 2026-03-19 14:26:19,399 - DEBUG - Pod : ctest-nginx-pod-98063544 UUID is 34eade9e-4557-43b6-bdd2-07fc5ec75cde 2026-03-19 14:26:19,560 - INFO - Created VN __public__, UUID :9eda3ff7-7a06-4765-9b5b-37ce5ced4c7a 2026-03-19 14:26:19,761 - DEBUG - Requesting: http://10.0.0.56:8082/domains 2026-03-19 14:26:19,766 - DEBUG - Requesting: http://10.0.0.56:8082/domain/dd081525-3d7c-4426-ba32-bdafdad1a0a6 2026-03-19 14:26:19,797 - DEBUG - Requesting: http://10.0.0.56:8082/project/958a78a1-43a0-4463-92b1-1a98c02e74fe 2026-03-19 14:26:19,834 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/9eda3ff7-7a06-4765-9b5b-37ce5ced4c7a 2026-03-19 14:26:19,843 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/9eda3ff7-7a06-4765-9b5b-37ce5ced4c7a 2026-03-19 14:26:19,854 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/2b3f0b0e-8ee8-4f3d-9f7f-e36aedf3d839 2026-03-19 14:26:19,862 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/2b3f0b0e-8ee8-4f3d-9f7f-e36aedf3d839 2026-03-19 14:26:19,867 - DEBUG - Route Targets: [] 2026-03-19 14:26:19,867 - DEBUG - RT names not yet present for VN __public__ 2026-03-19 14:26:24,868 - DEBUG - Requesting: http://10.0.0.56:8082/domains 2026-03-19 14:26:24,878 - DEBUG - Requesting: http://10.0.0.56:8082/domain/dd081525-3d7c-4426-ba32-bdafdad1a0a6 2026-03-19 14:26:24,885 - DEBUG - Requesting: http://10.0.0.56:8082/project/958a78a1-43a0-4463-92b1-1a98c02e74fe 2026-03-19 14:26:24,923 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/9eda3ff7-7a06-4765-9b5b-37ce5ced4c7a 2026-03-19 14:26:24,932 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/9eda3ff7-7a06-4765-9b5b-37ce5ced4c7a 2026-03-19 14:26:24,940 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/2b3f0b0e-8ee8-4f3d-9f7f-e36aedf3d839 2026-03-19 14:26:24,949 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/2b3f0b0e-8ee8-4f3d-9f7f-e36aedf3d839 2026-03-19 14:26:24,963 - DEBUG - Requesting: http://10.0.0.56:8082/route-target/695d086b-c05f-495b-8d60-7116504e3b00 2026-03-19 14:26:24,971 - DEBUG - Route Targets: ['target:64512:8000006'] 2026-03-19 14:26:24,971 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/9eda3ff7-7a06-4765-9b5b-37ce5ced4c7a 2026-03-19 14:26:24,981 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/2b3f0b0e-8ee8-4f3d-9f7f-e36aedf3d839 2026-03-19 14:26:24,996 - INFO - Verified VN network id 13 for VN 9eda3ff7-7a06-4765-9b5b-37ce5ced4c7a 2026-03-19 14:26:24,997 - INFO - Verifications in API Server for VN __public__ passed 2026-03-19 14:26:24,997 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/9eda3ff7-7a06-4765-9b5b-37ce5ced4c7a 2026-03-19 14:26:25,012 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/2b3f0b0e-8ee8-4f3d-9f7f-e36aedf3d839 2026-03-19 14:26:25,019 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/2b3f0b0e-8ee8-4f3d-9f7f-e36aedf3d839 2026-03-19 14:26:25,028 - DEBUG - Requesting: http://10.0.0.56:8082/route-target/695d086b-c05f-495b-8d60-7116504e3b00 2026-03-19 14:26:25,035 - DEBUG - Requesting: http://10.0.0.56:8083/Snh_IFMapTableShowReq?table_name=virtual-network&search_string=virtual-network:default-domain:k8s-default:__public__ 2026-03-19 14:26:25,041 - DEBUG - Control-node 10.0.0.56 : VN object is : {'node_name': 'virtual-network:default-domain:k8s-default:__public__', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'cloud-admin', 'owner-access': '7', 'group': 'cloud-admin-group', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '11446531735052240741', 'uuid-lslong': '11194602658221345914'}, 'enable': 'true', 'created': '2026-03-19T14:26:19', 'last-modified': '2026-03-19T14:26:19', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:05.338314'} 2026-03-19 14:26:25,042 - DEBUG - Requesting: http://10.0.0.56:8083/Snh_ShowRoutingInstanceReq?name=default-domain:k8s-default:__public__:__public__ 2026-03-19 14:26:25,052 - DEBUG - Route Targets: ['target:64512:8000006'] 2026-03-19 14:26:25,052 - INFO - On all control nodes, Config, RI and RT verification for VN __public__ passed 2026-03-19 14:26:25,053 - DEBUG - ====Verifying policy data for __public__ in API_Server ====== 2026-03-19 14:26:25,053 - DEBUG - Requesting: http://10.0.0.56:8082/domains 2026-03-19 14:26:25,057 - DEBUG - Requesting: http://10.0.0.56:8082/domain/dd081525-3d7c-4426-ba32-bdafdad1a0a6 2026-03-19 14:26:25,064 - DEBUG - Requesting: http://10.0.0.56:8082/project/958a78a1-43a0-4463-92b1-1a98c02e74fe 2026-03-19 14:26:25,113 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/9eda3ff7-7a06-4765-9b5b-37ce5ced4c7a 2026-03-19 14:26:25,127 - DEBUG - =>VN __public__ has no policy to be verified 2026-03-19 14:26:25,127 - DEBUG - Verifying the vn in opserver 2026-03-19 14:26:25,127 - DEBUG - Verifying the default-domain:k8s-default:__public__ virtual network link through opserver 10.0.0.56 2026-03-19 14:26:25,127 - DEBUG - Requesting: http://10.0.0.56:8081/analytics/uves/virtual-networks 2026-03-19 14:26:25,133 - DEBUG - vn link and name as {'name': 'default-domain:k8s-default:__public__', 'href': 'http://10.0.0.56:8081/analytics/uves/virtual-network/default-domain:k8s-default:__public__?flat'} 2026-03-19 14:26:25,133 - INFO - Validated that VN default-domain:k8s-default:__public__ is found in opserver 2026-03-19 14:26:25,133 - DEBUG - Requesting: http://10.0.0.61:8085/Snh_VrfListReq?name=default-domain:k8s-default:__public__:__public__ 2026-03-19 14:26:25,141 - DEBUG - Requesting: http://10.0.0.61:8085/Snh_VrfListReq?name=default-domain:k8s-default:__public__:__public__ 2026-03-19 14:26:25,146 - DEBUG - Do not have enough data to verify VN in agent 2026-03-19 14:26:25,146 - DEBUG - Requesting: http://10.0.0.61:8085/Snh_VrfListReq?name=default-domain:k8s-default:__public__:__public__ 2026-03-19 14:26:25,153 - DEBUG - VRF ids for VN __public__: {} 2026-03-19 14:26:25,254 - INFO - Creating Floating IP pool __fip_pool_public__ in API Server 2026-03-19 14:26:25,344 - DEBUG - Requesting: http://10.0.0.56:8082/domains 2026-03-19 14:26:25,349 - DEBUG - Requesting: http://10.0.0.56:8082/domain/dd081525-3d7c-4426-ba32-bdafdad1a0a6 2026-03-19 14:26:25,360 - DEBUG - Requesting: http://10.0.0.56:8082/project/958a78a1-43a0-4463-92b1-1a98c02e74fe 2026-03-19 14:26:25,400 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/9eda3ff7-7a06-4765-9b5b-37ce5ced4c7a 2026-03-19 14:26:25,409 - DEBUG - Requesting: http://10.0.0.56:8082/floating-ip-pool/e5beb700-d9f8-490c-8e8c-0a62a4fc8805 2026-03-19 14:26:25,415 - DEBUG - Requesting: http://10.0.0.56:8082/domains 2026-03-19 14:26:25,419 - DEBUG - Requesting: http://10.0.0.56:8082/domain/dd081525-3d7c-4426-ba32-bdafdad1a0a6 2026-03-19 14:26:25,426 - DEBUG - Requesting: http://10.0.0.56:8082/project/958a78a1-43a0-4463-92b1-1a98c02e74fe 2026-03-19 14:26:25,474 - DEBUG - Requesting: http://10.0.0.56:8082/virtual-network/9eda3ff7-7a06-4765-9b5b-37ce5ced4c7a 2026-03-19 14:26:25,482 - DEBUG - FIP Pool __fip_pool_public__ found in API Server 2026-03-19 14:26:25,482 - INFO - Verification for FIP pool __fip_pool_public__ in API Server passed 2026-03-19 14:26:25,488 - DEBUG - Requesting: http://10.0.0.56:8083/Snh_IFMapTableShowReq?table_name=floating-ip-pool&search_string=floating-ip-pool:default-domain:k8s-default:__public__:__fip_pool_public__ 2026-03-19 14:26:25,492 - DEBUG - Control-node Ifmap-view has FIP pool __fip_pool_public__ information 2026-03-19 14:26:25,492 - INFO - Verification for FIP pool __fip_pool_public__ in Control-node passed 2026-03-19 14:26:25,495 - DEBUG - Ingress ctest-nginx-ingress-69400930 not present 2026-03-19 14:26:25,496 - INFO - Creating Ingress ctest-nginx-ingress-69400930 2026-03-19 14:26:25,504 - INFO - Created Ingress ctest-nginx-ingress-69400930 2026-03-19 14:26:25,507 - DEBUG - Cluster IP not yet seen for Ingress ctest-nginx-ingress-69400930 2026-03-19 14:26:28,512 - DEBUG - For Ingress ctest-nginx-ingress-69400930, Cluster IP: 10.47.255.249, External IPs ['175.181.155.124'] 2026-03-19 14:26:28,512 - INFO - Verifications in k8s passed for Ingress ctest-nginx-ingress-69400930 2026-03-19 14:26:28,513 - DEBUG - Requesting: http://10.0.0.56:8108/Snh_LoadbalancerDatabaseList?x=ed49344e-099b-4483-90b1-aef5a470adb9 2026-03-19 14:26:28,517 - INFO - Ingress ctest-nginx-ingress-69400930 with uuid ed49344e-099b-4483-90b1-aef5a470adb9 found in kube manager 2026-03-19 14:26:28,517 - INFO - Ingress ctest-nginx-ingress-69400930 verification passed 2026-03-19 14:26:28,519 - DEBUG - Pod ctest-busybox-pod-59504477 not present 2026-03-19 14:26:28,519 - INFO - Creating Pod ctest-busybox-pod-59504477 2026-03-19 14:26:28,561 - DEBUG - Pod : ctest-busybox-pod-59504477 UUID is c457f41c-4845-4b5f-89f2-b9325ec15f1b 2026-03-19 14:26:28,567 - DEBUG - Pod ctest-nginx-pod-38883926 not in running state.Currently in Pending 2026-03-19 14:26:33,575 - INFO - Pod ctest-nginx-pod-38883926 is in running state.Got IP 10.47.255.251 2026-03-19 14:26:33,640 - DEBUG - Pod ctest-nginx-pod-38883926 has vmi 999328d6-239f-11f1-ba89-fa163e8ff9cf 2026-03-19 14:26:33,640 - INFO - Verified pod ctest-nginx-pod-38883926 in contrail-api 2026-03-19 14:26:33,640 - DEBUG - Requesting: http://10.0.0.61:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0, 2026-03-19 14:26:33,650 - DEBUG - VMI 999328d6-239f-11f1-ba89-fa163e8ff9cf is active in agent 10.0.0.61 2026-03-19 14:26:33,650 - INFO - Verified Pod ctest-nginx-pod-38883926 in agent 10.0.0.61 2026-03-19 14:26:33,650 - DEBUG - Requesting: http://10.0.0.56:8108/Snh_VirtualMachineDatabaseList?x=631bab7a-8194-417d-a5f2-241e154b049b 2026-03-19 14:26:33,659 - INFO - Pod ctest-nginx-pod-38883926 with uuid 631bab7a-8194-417d-a5f2-241e154b049b found in kube manager 2026-03-19 14:26:33,659 - INFO - Pod ctest-nginx-pod-38883926 verification passed 2026-03-19 14:26:33,739 - DEBUG - [Pod ctest-nginx-pod-38883926] Cmd: echo ctest-nginx-pod-38883926 > /usr/share/nginx/html/index.html, Output: 2026-03-19 14:26:33,747 - INFO - Pod ctest-nginx-pod-98063544 is in running state.Got IP 10.47.255.250 2026-03-19 14:26:33,758 - DEBUG - Pod ctest-nginx-pod-98063544 has vmi 9a0be32a-239f-11f1-ba89-fa163e8ff9cf 2026-03-19 14:26:33,758 - INFO - Verified pod ctest-nginx-pod-98063544 in contrail-api 2026-03-19 14:26:33,758 - DEBUG - Requesting: http://10.0.0.61:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0, 2026-03-19 14:26:33,768 - DEBUG - VMI 9a0be32a-239f-11f1-ba89-fa163e8ff9cf is active in agent 10.0.0.61 2026-03-19 14:26:33,769 - INFO - Verified Pod ctest-nginx-pod-98063544 in agent 10.0.0.61 2026-03-19 14:26:33,769 - DEBUG - Requesting: http://10.0.0.56:8108/Snh_VirtualMachineDatabaseList?x=34eade9e-4557-43b6-bdd2-07fc5ec75cde 2026-03-19 14:26:33,774 - INFO - Pod ctest-nginx-pod-98063544 with uuid 34eade9e-4557-43b6-bdd2-07fc5ec75cde found in kube manager 2026-03-19 14:26:33,774 - INFO - Pod ctest-nginx-pod-98063544 verification passed 2026-03-19 14:26:33,829 - DEBUG - [Pod ctest-nginx-pod-98063544] Cmd: echo ctest-nginx-pod-98063544 > /usr/share/nginx/html/index.html, Output: 2026-03-19 14:26:33,835 - DEBUG - Pod ctest-busybox-pod-59504477 not in running state.Currently in Pending 2026-03-19 14:26:38,844 - DEBUG - Pod ctest-busybox-pod-59504477 not in running state.Currently in Pending 2026-03-19 14:26:43,852 - INFO - Pod ctest-busybox-pod-59504477 is in running state.Got IP 10.47.255.248 2026-03-19 14:26:43,864 - DEBUG - Pod ctest-busybox-pod-59504477 has vmi 9e429d9e-239f-11f1-ba89-fa163e8ff9cf 2026-03-19 14:26:43,864 - INFO - Verified pod ctest-busybox-pod-59504477 in contrail-api 2026-03-19 14:26:43,864 - DEBUG - Requesting: http://10.0.0.61:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0, 2026-03-19 14:26:43,874 - DEBUG - VMI 9e429d9e-239f-11f1-ba89-fa163e8ff9cf is active in agent 10.0.0.61 2026-03-19 14:26:43,874 - INFO - Verified Pod ctest-busybox-pod-59504477 in agent 10.0.0.61 2026-03-19 14:26:43,875 - DEBUG - Requesting: http://10.0.0.56:8108/Snh_VirtualMachineDatabaseList?x=c457f41c-4845-4b5f-89f2-b9325ec15f1b 2026-03-19 14:26:43,878 - INFO - Pod ctest-busybox-pod-59504477 with uuid c457f41c-4845-4b5f-89f2-b9325ec15f1b found in kube manager 2026-03-19 14:26:43,878 - INFO - Pod ctest-busybox-pod-59504477 verification passed 2026-03-19 14:26:43,936 - DEBUG - [Pod ctest-busybox-pod-59504477] Cmd: wget http://10.47.255.249:80/ -O - -T 5 -t 1 , Output: ctest-nginx-pod-38883926 Connecting to 10.47.255.249:80 (10.47.255.249:80) writing to stdout - 100% |********************************| 25 0:00:00 ETA written to stdout 2026-03-19 14:26:43,936 - DEBUG - [Pod Pod ctest-busybox-pod-59504477] Cmd wget http://10.47.255.249:80/ -O - -T 5 -t 1 passed 2026-03-19 14:26:43,936 - DEBUG - [Pod Pod ctest-busybox-pod-59504477] Cmd output: ctest-nginx-pod-38883926 Connecting to 10.47.255.249:80 (10.47.255.249:80) writing to stdout - 100% |********************************| 25 0:00:00 ETA written to stdout 2026-03-19 14:26:43,991 - DEBUG - [Pod ctest-busybox-pod-59504477] Cmd: wget http://10.47.255.249:80/ -O - -T 5 -t 1 , Output: ctest-nginx-pod-38883926 Connecting to 10.47.255.249:80 (10.47.255.249:80) writing to stdout - 100% |********************************| 25 0:00:00 ETA written to stdout 2026-03-19 14:26:43,991 - DEBUG - [Pod Pod ctest-busybox-pod-59504477] Cmd wget http://10.47.255.249:80/ -O - -T 5 -t 1 passed 2026-03-19 14:26:43,991 - DEBUG - [Pod Pod ctest-busybox-pod-59504477] Cmd output: ctest-nginx-pod-38883926 Connecting to 10.47.255.249:80 (10.47.255.249:80) writing to stdout - 100% |********************************| 25 0:00:00 ETA written to stdout 2026-03-19 14:26:44,042 - DEBUG - [Pod ctest-busybox-pod-59504477] Cmd: wget http://10.47.255.249:80/ -O - -T 5 -t 1 , Output: ctest-nginx-pod-38883926 Connecting to 10.47.255.249:80 (10.47.255.249:80) writing to stdout - 100% |********************************| 25 0:00:00 ETA written to stdout 2026-03-19 14:26:44,042 - DEBUG - [Pod Pod ctest-busybox-pod-59504477] Cmd wget http://10.47.255.249:80/ -O - -T 5 -t 1 passed 2026-03-19 14:26:44,042 - DEBUG - [Pod Pod ctest-busybox-pod-59504477] Cmd output: ctest-nginx-pod-38883926 Connecting to 10.47.255.249:80 (10.47.255.249:80) writing to stdout - 100% |********************************| 25 0:00:00 ETA written to stdout 2026-03-19 14:26:44,104 - DEBUG - [Pod ctest-busybox-pod-59504477] Cmd: wget http://10.47.255.249:80/ -O - -T 5 -t 1 , Output: ctest-nginx-pod-98063544 Connecting to 10.47.255.249:80 (10.47.255.249:80) writing to stdout - 100% |********************************| 25 0:00:00 ETA written to stdout 2026-03-19 14:26:44,104 - DEBUG - [Pod Pod ctest-busybox-pod-59504477] Cmd wget http://10.47.255.249:80/ -O - -T 5 -t 1 passed 2026-03-19 14:26:44,104 - DEBUG - [Pod Pod ctest-busybox-pod-59504477] Cmd output: ctest-nginx-pod-98063544 Connecting to 10.47.255.249:80 (10.47.255.249:80) writing to stdout - 100% |********************************| 25 0:00:00 ETA written to stdout 2026-03-19 14:26:44,104 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-nginx-pod-38883926': 3, 'ctest-nginx-pod-98063544': 1} 2026-03-19 14:26:44,104 - WARNING - Nothing to delete parallely 2026-03-19 14:26:44,105 - INFO - Deleting pod default:ctest-busybox-pod-59504477 2026-03-19 14:26:44,138 - DEBUG - Requesting: http://10.0.0.61:8085/Snh_VmListReq?uuid=c457f41c-4845-4b5f-89f2-b9325ec15f1b 2026-03-19 14:26:44,144 - WARNING - Pod uuid c457f41c-4845-4b5f-89f2-b9325ec15f1b is still seen in agent 10.0.0.61 VM list 2026-03-19 14:26:49,145 - DEBUG - Requesting: http://10.0.0.61:8085/Snh_VmListReq?uuid=c457f41c-4845-4b5f-89f2-b9325ec15f1b 2026-03-19 14:26:49,151 - WARNING - Pod uuid c457f41c-4845-4b5f-89f2-b9325ec15f1b is still seen in agent 10.0.0.61 VM list 2026-03-19 14:26:54,152 - DEBUG - Requesting: http://10.0.0.61:8085/Snh_VmListReq?uuid=c457f41c-4845-4b5f-89f2-b9325ec15f1b 2026-03-19 14:26:54,159 - WARNING - Pod uuid c457f41c-4845-4b5f-89f2-b9325ec15f1b is still seen in agent 10.0.0.61 VM list 2026-03-19 14:26:59,160 - DEBUG - Requesting: http://10.0.0.61:8085/Snh_VmListReq?uuid=c457f41c-4845-4b5f-89f2-b9325ec15f1b 2026-03-19 14:26:59,168 - WARNING - Pod uuid c457f41c-4845-4b5f-89f2-b9325ec15f1b is still seen in agent 10.0.0.61 VM list 2026-03-19 14:27:04,169 - DEBUG - Requesting: http://10.0.0.61:8085/Snh_VmListReq?uuid=c457f41c-4845-4b5f-89f2-b9325ec15f1b 2026-03-19 14:27:04,177 - WARNING - Pod uuid c457f41c-4845-4b5f-89f2-b9325ec15f1b is still seen in agent 10.0.0.61 VM list 2026-03-19 14:27:09,179 - DEBUG - Requesting: http://10.0.0.61:8085/Snh_VmListReq?uuid=c457f41c-4845-4b5f-89f2-b9325ec15f1b 2026-03-19 14:27:09,188 - WARNING - Pod uuid c457f41c-4845-4b5f-89f2-b9325ec15f1b is still seen in agent 10.0.0.61 VM list 2026-03-19 14:27:14,188 - DEBUG - Requesting: http://10.0.0.61:8085/Snh_VmListReq?uuid=c457f41c-4845-4b5f-89f2-b9325ec15f1b 2026-03-19 14:27:14,197 - WARNING - Pod uuid c457f41c-4845-4b5f-89f2-b9325ec15f1b is still seen in agent 10.0.0.61 VM list 2026-03-19 14:27:19,198 - DEBUG - Requesting: http://10.0.0.61:8085/Snh_VmListReq?uuid=c457f41c-4845-4b5f-89f2-b9325ec15f1b 2026-03-19 14:27:19,203 - DEBUG - Pod c457f41c-4845-4b5f-89f2-b9325ec15f1b is not in agent 10.0.0.61 VM list 2026-03-19 14:27:19,204 - DEBUG - Requesting: http://10.0.0.61:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:9e429d9e-239f-11f1-ba89-fa163e8ff9cf 2026-03-19 14:27:19,208 - DEBUG - VMI 9e429d9e-239f-11f1-ba89-fa163e8ff9cf is removed from agent 10.0.0.61 2026-03-19 14:27:19,209 - INFO - Verified that pod ctest-busybox-pod-59504477 is removed in agent 2026-03-19 14:27:19,209 - INFO - Deleting Ingress : ctest-nginx-ingress-69400930 2026-03-19 14:27:19,218 - DEBUG - Requesting: http://10.0.0.56:8108/Snh_LoadbalancerDatabaseList?x=ed49344e-099b-4483-90b1-aef5a470adb9 2026-03-19 14:27:19,224 - ERROR - Ingress ctest-nginx-ingress-69400930 with uuid ed49344e-099b-4483-90b1-aef5a470adb9 still found in kube manager 2026-03-19 14:27:20,224 - DEBUG - Requesting: http://10.0.0.56:8108/Snh_LoadbalancerDatabaseList?x=ed49344e-099b-4483-90b1-aef5a470adb9 2026-03-19 14:27:20,228 - INFO - Ingress ctest-nginx-ingress-69400930 with uuid ed49344e-099b-4483-90b1-aef5a470adb9 deleted successfully from kube manager 2026-03-19 14:27:20,228 - INFO - Verifications on Ingress %s cleanup passed 2026-03-19 14:27:20,228 - INFO - Deleting the FIP pool __fip_pool_public__ 2026-03-19 14:27:20,374 - DEBUG - Requesting: http://10.0.0.56:8083/Snh_IFMapTableShowReq?table_name=floating-ip-pool&search_string=floating-ip-pool:default-domain:k8s-default:__public__:__fip_pool_public__ 2026-03-19 14:27:20,382 - DEBUG - Control-node Ifmap-view does not have FIP pool __fip_pool_public__ information 2026-03-19 14:27:20,382 - INFO - Deleting VN __public__ 2026-03-19 14:27:20,583 - DEBUG - Requesting: http://10.0.0.56:8082/routing-instance/2b3f0b0e-8ee8-4f3d-9f7f-e36aedf3d839 2026-03-19 14:27:20,601 - DEBUG - Response Code: 404 2026-03-19 14:27:20,601 - DEBUG - Requesting: http://10.0.0.56:8082/domains 2026-03-19 14:27:20,615 - DEBUG - Requesting: http://10.0.0.56:8082/domain/dd081525-3d7c-4426-ba32-bdafdad1a0a6 2026-03-19 14:27:20,630 - DEBUG - Requesting: http://10.0.0.56:8082/project/958a78a1-43a0-4463-92b1-1a98c02e74fe 2026-03-19 14:27:20,714 - INFO - Validated that VN __public__ is not found in API Server 2026-03-19 14:27:20,715 - DEBUG - Requesting: http://10.0.0.61:8085/Snh_VnListReq?name=default-domain:k8s-default:__public__ 2026-03-19 14:27:20,721 - DEBUG - Requesting: http://10.0.0.61:8085/Snh_VrfListReq?name=default-domain:k8s-default:__public__:__public__ 2026-03-19 14:27:20,728 - DEBUG - VN __public__ is not present in Agent 10.0.0.61 2026-03-19 14:27:20,728 - INFO - Validated that VN __public__ is not in any agent 2026-03-19 14:27:20,728 - DEBUG - Requesting: http://10.0.0.61:8085/Snh_VrfListReq?name=default-domain:k8s-default:__public__:__public__ 2026-03-19 14:27:20,735 - DEBUG - Requesting: http://10.0.0.56:8083/Snh_ShowRoutingInstanceReq?name=default-domain:k8s-default:__public__:__public__ 2026-03-19 14:27:20,741 - DEBUG - Requesting: http://10.0.0.56:8083/Snh_IFMapTableShowReq?table_name=virtual-network&search_string=virtual-network:default-domain:k8s-default:__public__ 2026-03-19 14:27:20,746 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN __public__ info 2026-03-19 14:27:20,747 - INFO - Deleting pod default:ctest-nginx-pod-98063544 2026-03-19 14:27:20,773 - DEBUG - Requesting: http://10.0.0.61:8085/Snh_VmListReq?uuid=34eade9e-4557-43b6-bdd2-07fc5ec75cde 2026-03-19 14:27:20,784 - WARNING - Pod uuid 34eade9e-4557-43b6-bdd2-07fc5ec75cde is still seen in agent 10.0.0.61 VM list 2026-03-19 14:27:25,785 - DEBUG - Requesting: http://10.0.0.61:8085/Snh_VmListReq?uuid=34eade9e-4557-43b6-bdd2-07fc5ec75cde 2026-03-19 14:27:25,791 - DEBUG - Pod 34eade9e-4557-43b6-bdd2-07fc5ec75cde is not in agent 10.0.0.61 VM list 2026-03-19 14:27:25,791 - DEBUG - Requesting: http://10.0.0.61:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:9a0be32a-239f-11f1-ba89-fa163e8ff9cf 2026-03-19 14:27:25,797 - DEBUG - VMI 9a0be32a-239f-11f1-ba89-fa163e8ff9cf is removed from agent 10.0.0.61 2026-03-19 14:27:25,797 - INFO - Verified that pod ctest-nginx-pod-98063544 is removed in agent 2026-03-19 14:27:25,797 - INFO - Deleting pod default:ctest-nginx-pod-38883926 2026-03-19 14:27:25,811 - DEBUG - Requesting: http://10.0.0.61:8085/Snh_VmListReq?uuid=631bab7a-8194-417d-a5f2-241e154b049b 2026-03-19 14:27:25,818 - WARNING - Pod uuid 631bab7a-8194-417d-a5f2-241e154b049b is still seen in agent 10.0.0.61 VM list 2026-03-19 14:27:30,819 - DEBUG - Requesting: http://10.0.0.61:8085/Snh_VmListReq?uuid=631bab7a-8194-417d-a5f2-241e154b049b 2026-03-19 14:27:30,832 - DEBUG - Pod 631bab7a-8194-417d-a5f2-241e154b049b is not in agent 10.0.0.61 VM list 2026-03-19 14:27:30,832 - DEBUG - Requesting: http://10.0.0.61:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:999328d6-239f-11f1-ba89-fa163e8ff9cf 2026-03-19 14:27:30,838 - DEBUG - VMI 999328d6-239f-11f1-ba89-fa163e8ff9cf is removed from agent 10.0.0.61 2026-03-19 14:27:30,838 - INFO - Verified that pod ctest-nginx-pod-38883926 is removed in agent 2026-03-19 14:27:30,838 - INFO - Deleting service : ctest-nginx-svc-20069857 2026-03-19 14:27:31,100 - DEBUG - Requesting: http://10.0.0.61:8085/Snh_AgentXmppConnectionStatusReq? 2026-03-19 14:27:31,110 - DEBUG - No XMPP flaps were noticed during the test