2026-02-12 12:17:18,417 - DEBUG - [10.0.0.22]: Running cmd : hostname 2026-02-12 12:17:18,817 - DEBUG - Output : cn-jenkins-deploy-platform-juju-k8s-1891-1 2026-02-12 12:17:18,817 - DEBUG - [10.0.0.22]: Running cmd : hostname -f 2026-02-12 12:17:18,877 - DEBUG - Output : cn-jenkins-deploy-platform-juju-k8s-1891-1 2026-02-12 12:17:18,877 - DEBUG - [10.0.0.22]: Running cmd : docker ps 2>/dev/null | grep -v "/pause\|/usr/bin/pod\|nova_api_\|contrail.*init\|init.*contrail\|provisioner\|placement" | awk '{print $NF}' 2026-02-12 12:17:19,018 - DEBUG - Output : NAMES contrail_test_oG7b73q4d contrail-kubernetes-master-kubemanager-1 analytics_database-cassandra-1 analytics_database-query-engine-1 analytics_database-nodemgr-1 analytics_snmp-snmp-collector-1 analytics_snmp-topology-1 analytics_snmp-nodemgr-1 analytics_alarm-alarm-gen-1 analytics_alarm-kafka-1 analytics_alarm-nodemgr-1 analytics-nodemgr-1 analytics-api-1 analytics-collector-1 redis-redis-1 webui-web-1 webui-job-1 control-named-1 control-dns-1 control-nodemgr-1 control-control-1 config_database-cassandra-1 config_database-zookeeper-1 config_database-rabbitmq-1 config_database-nodemgr-1 config_api-devicemgr-1 config_api-api-1 config_api-nodemgr-1 config_api-dnsmasq-1 config_api-schema-1 config_api-svcmonitor-1 2026-02-12 12:17:19,019 - DEBUG - [10.0.0.22]: Running cmd : ip -4 -o addr show | awk '{print $4}' 2026-02-12 12:17:19,076 - DEBUG - Output : 127.0.0.1/8 10.0.0.22/24 252.22.0.1/8 10.7.142.1/24 172.17.0.1/16 2026-02-12 12:17:19,076 - DEBUG - [10.0.0.22]: Running cmd : getent hosts 10.0.0.22 | head -n 1 | awk '{print $2}' 2026-02-12 12:17:19,177 - DEBUG - Output : cn-jenkins-deploy-platform-juju-k8s-1891-1 2026-02-12 12:17:19,294 - DEBUG - [10.0.0.249]: Running cmd : hostname 2026-02-12 12:17:19,575 - DEBUG - Output : an-jenkins-deploy-platform-juju-k8s-1891-1 2026-02-12 12:17:19,576 - DEBUG - [10.0.0.249]: Running cmd : hostname -f 2026-02-12 12:17:19,614 - DEBUG - Output : an-jenkins-deploy-platform-juju-k8s-1891-1 2026-02-12 12:17:19,615 - DEBUG - [10.0.0.249]: Running cmd : docker ps 2>/dev/null | grep -v "/pause\|/usr/bin/pod\|nova_api_\|contrail.*init\|init.*contrail\|provisioner\|placement" | awk '{print $NF}' 2026-02-12 12:17:19,706 - DEBUG - Output : NAMES vrouter-vrouter-agent-1 vrouter-nodemgr-1 2026-02-12 12:17:19,707 - DEBUG - [10.0.0.249]: Running cmd : ip -4 -o addr show dev vhost0 | awk '{print $4}' 2026-02-12 12:17:19,746 - DEBUG - Output : 10.0.0.249/24 2026-02-12 12:17:19,747 - DEBUG - [10.0.0.249]: Running cmd : getent hosts 10.0.0.249 | head -n 1 | awk '{print $2}' 2026-02-12 12:17:19,815 - DEBUG - Output : an-jenkins-deploy-platform-juju-k8s-1891-1 2026-02-12 12:17:19,985 - INFO - Using existing project ['default-domain', 'k8s-default'](3d7add91-f0e6-4cf3-9deb-70568bdf4098) 2026-02-12 12:17:20,111 - INFO - Using existing project ['default-domain', 'k8s-default'](3d7add91-f0e6-4cf3-9deb-70568bdf4098) 2026-02-12 12:17:20,278 - INFO - ================================================================================ 2026-02-12 12:17:20,278 - INFO - STARTING TEST : test_ingress_ip_assignment 2026-02-12 12:17:20,278 - INFO - TEST DESCRIPTION : Verify that Ingress gets a CLuster IP which is reachable to Pods in same namespace. Also verify that a Floating IP is assigned to the Ingress from the Public FIP poo. Steps: 1. Create a service with 2 pods running nginx 2. Create an ingress out of this service 3. From another Pod do a wget on the ingress Cluster ip Validate that Ingress get a IP from Public FIP pool which might/might not be accessible. Validate that service and its loadbalancing work 2026-02-12 12:17:20,542 - DEBUG - Requesting: http://10.0.0.249:8085/Snh_AgentXmppConnectionStatusReq? 2026-02-12 12:17:20,555 - DEBUG - Nothing to compare xmpp stats {'10.0.0.249': {'10.0.0.22': '0'}} with 2026-02-12 12:17:20,555 - INFO - Initial checks done. Running the testcase now 2026-02-12 12:17:20,555 - INFO - 2026-02-12 12:17:20,563 - INFO - Namespace default already exists 2026-02-12 12:17:20,564 - DEBUG - [10.0.0.22]: Running cmd : docker exec --privileged -it contrail-kubernetes-master-kubemanager-1 /bin/bash -c 'grep "^[ ]*cluster_project" /etc/contrail/contrail-kubernetes.conf' 2026-02-12 12:17:20,707 - DEBUG - Output : cluster_project={} 2026-02-12 12:17:20,776 - INFO - Project default-domain:k8s-default for Namespace default is seen in contrail-api 2026-02-12 12:17:20,776 - DEBUG - Project uuid in Contrail is 3d7add91-f0e6-4cf3-9deb-70568bdf4098 2026-02-12 12:17:20,776 - DEBUG - [10.0.0.22]: Running cmd : docker exec --privileged -it contrail-kubernetes-master-kubemanager-1 /bin/bash -c 'netstat -antp | grep :8108 | grep LISTEN' 2026-02-12 12:17:20,914 - DEBUG - Output : tcp 0 0 0.0.0.0:8108 0.0.0.0:* LISTEN 1/python3 2026-02-12 12:17:20,914 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_NamespaceDatabaseList?namespace_uuid=4265be52-acbe-49ff-aefa-26343456b563 2026-02-12 12:17:20,919 - INFO - Namespace default with uuid 4265be52-acbe-49ff-aefa-26343456b563 found in kube manager 2026-02-12 12:17:20,919 - INFO - Namespace default verification passed 2026-02-12 12:17:20,923 - DEBUG - Service ctest-nginx-svc-64978886 not present 2026-02-12 12:17:20,923 - INFO - Creating service ctest-nginx-svc-64978886 2026-02-12 12:17:20,952 - INFO - Created Service ctest-nginx-svc-64978886 2026-02-12 12:17:20,960 - DEBUG - Pod ctest-nginx-pod-54189690 not present 2026-02-12 12:17:20,962 - INFO - Creating Pod ctest-nginx-pod-54189690 2026-02-12 12:17:21,056 - DEBUG - Pod : ctest-nginx-pod-54189690 UUID is e43c61c9-cf7a-4e24-a037-92ac16478fc3 2026-02-12 12:17:21,068 - DEBUG - Pod ctest-nginx-pod-82886736 not present 2026-02-12 12:17:21,068 - INFO - Creating Pod ctest-nginx-pod-82886736 2026-02-12 12:17:21,103 - DEBUG - Pod : ctest-nginx-pod-82886736 UUID is 206e6c0d-1ebe-45ba-a838-c287160fed17 2026-02-12 12:17:21,310 - INFO - Created VN __public__, UUID :482c6aaa-f318-44c0-9417-a9caa04995d4 2026-02-12 12:17:21,575 - DEBUG - Requesting: http://10.0.0.22:8082/domains 2026-02-12 12:17:21,582 - DEBUG - Requesting: http://10.0.0.22:8082/domain/168fb2bf-c66e-49fd-b422-7e20e5b6e896 2026-02-12 12:17:21,595 - DEBUG - Requesting: http://10.0.0.22:8082/project/3d7add91-f0e6-4cf3-9deb-70568bdf4098 2026-02-12 12:17:21,643 - DEBUG - Requesting: http://10.0.0.22:8082/virtual-network/482c6aaa-f318-44c0-9417-a9caa04995d4 2026-02-12 12:17:21,717 - DEBUG - Requesting: http://10.0.0.22:8082/virtual-network/482c6aaa-f318-44c0-9417-a9caa04995d4 2026-02-12 12:17:21,743 - DEBUG - Requesting: http://10.0.0.22:8082/routing-instance/59414b9b-48a6-4341-bea8-82dccbbf6521 2026-02-12 12:17:21,750 - DEBUG - Requesting: http://10.0.0.22:8082/routing-instance/59414b9b-48a6-4341-bea8-82dccbbf6521 2026-02-12 12:17:21,761 - DEBUG - Route Targets: [] 2026-02-12 12:17:21,761 - DEBUG - RT names not yet present for VN __public__ 2026-02-12 12:17:26,762 - DEBUG - Requesting: http://10.0.0.22:8082/domains 2026-02-12 12:17:26,777 - DEBUG - Requesting: http://10.0.0.22:8082/domain/168fb2bf-c66e-49fd-b422-7e20e5b6e896 2026-02-12 12:17:26,794 - DEBUG - Requesting: http://10.0.0.22:8082/project/3d7add91-f0e6-4cf3-9deb-70568bdf4098 2026-02-12 12:17:26,862 - DEBUG - Requesting: http://10.0.0.22:8082/virtual-network/482c6aaa-f318-44c0-9417-a9caa04995d4 2026-02-12 12:17:26,872 - DEBUG - Requesting: http://10.0.0.22:8082/virtual-network/482c6aaa-f318-44c0-9417-a9caa04995d4 2026-02-12 12:17:26,882 - DEBUG - Requesting: http://10.0.0.22:8082/routing-instance/59414b9b-48a6-4341-bea8-82dccbbf6521 2026-02-12 12:17:26,893 - DEBUG - Requesting: http://10.0.0.22:8082/routing-instance/59414b9b-48a6-4341-bea8-82dccbbf6521 2026-02-12 12:17:26,901 - DEBUG - Requesting: http://10.0.0.22:8082/route-target/8128bfdd-1b6a-4ecf-a7bc-f3ec99d42ec0 2026-02-12 12:17:26,908 - DEBUG - Route Targets: ['target:64512:8000006'] 2026-02-12 12:17:26,908 - DEBUG - Requesting: http://10.0.0.22:8082/virtual-network/482c6aaa-f318-44c0-9417-a9caa04995d4 2026-02-12 12:17:26,918 - DEBUG - Requesting: http://10.0.0.22:8082/routing-instance/59414b9b-48a6-4341-bea8-82dccbbf6521 2026-02-12 12:17:26,934 - INFO - Verified VN network id 13 for VN 482c6aaa-f318-44c0-9417-a9caa04995d4 2026-02-12 12:17:26,935 - INFO - Verifications in API Server for VN __public__ passed 2026-02-12 12:17:26,935 - DEBUG - Requesting: http://10.0.0.22:8082/virtual-network/482c6aaa-f318-44c0-9417-a9caa04995d4 2026-02-12 12:17:26,949 - DEBUG - Requesting: http://10.0.0.22:8082/routing-instance/59414b9b-48a6-4341-bea8-82dccbbf6521 2026-02-12 12:17:26,959 - DEBUG - Requesting: http://10.0.0.22:8082/routing-instance/59414b9b-48a6-4341-bea8-82dccbbf6521 2026-02-12 12:17:26,967 - DEBUG - Requesting: http://10.0.0.22:8082/route-target/8128bfdd-1b6a-4ecf-a7bc-f3ec99d42ec0 2026-02-12 12:17:26,977 - DEBUG - Requesting: http://10.0.0.22:8083/Snh_IFMapTableShowReq?table_name=virtual-network&search_string=virtual-network:default-domain:k8s-default:__public__ 2026-02-12 12:17:26,988 - DEBUG - Control-node 10.0.0.22 : VN object is : {'node_name': 'virtual-network:default-domain:k8s-default:__public__', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'cloud-admin', 'owner-access': '7', 'group': 'cloud-admin-group', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '5200648952161518784', 'uuid-lslong': '10671184529815344596'}, 'enable': 'true', 'created': '2026-02-12T12:17:21', 'last-modified': '2026-02-12T12:17:21', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:05.481973'} 2026-02-12 12:17:26,988 - DEBUG - Requesting: http://10.0.0.22:8083/Snh_ShowRoutingInstanceReq?name=default-domain:k8s-default:__public__:__public__ 2026-02-12 12:17:27,004 - DEBUG - Route Targets: ['target:64512:8000006'] 2026-02-12 12:17:27,005 - INFO - On all control nodes, Config, RI and RT verification for VN __public__ passed 2026-02-12 12:17:27,005 - DEBUG - ====Verifying policy data for __public__ in API_Server ====== 2026-02-12 12:17:27,005 - DEBUG - Requesting: http://10.0.0.22:8082/domains 2026-02-12 12:17:27,010 - DEBUG - Requesting: http://10.0.0.22:8082/domain/168fb2bf-c66e-49fd-b422-7e20e5b6e896 2026-02-12 12:17:27,018 - DEBUG - Requesting: http://10.0.0.22:8082/project/3d7add91-f0e6-4cf3-9deb-70568bdf4098 2026-02-12 12:17:27,061 - DEBUG - Requesting: http://10.0.0.22:8082/virtual-network/482c6aaa-f318-44c0-9417-a9caa04995d4 2026-02-12 12:17:27,071 - DEBUG - =>VN __public__ has no policy to be verified 2026-02-12 12:17:27,071 - DEBUG - Verifying the vn in opserver 2026-02-12 12:17:27,071 - DEBUG - Verifying the default-domain:k8s-default:__public__ virtual network link through opserver 10.0.0.22 2026-02-12 12:17:27,071 - DEBUG - Requesting: http://10.0.0.22:8081/analytics/uves/virtual-networks 2026-02-12 12:17:27,077 - DEBUG - vn link and name as {'name': 'default-domain:k8s-default:__public__', 'href': 'http://10.0.0.22:8081/analytics/uves/virtual-network/default-domain:k8s-default:__public__?flat'} 2026-02-12 12:17:27,077 - INFO - Validated that VN default-domain:k8s-default:__public__ is found in opserver 2026-02-12 12:17:27,077 - DEBUG - Requesting: http://10.0.0.249:8085/Snh_VrfListReq?name=default-domain:k8s-default:__public__:__public__ 2026-02-12 12:17:27,086 - DEBUG - Requesting: http://10.0.0.249:8085/Snh_VrfListReq?name=default-domain:k8s-default:__public__:__public__ 2026-02-12 12:17:27,096 - DEBUG - Do not have enough data to verify VN in agent 2026-02-12 12:17:27,096 - DEBUG - Requesting: http://10.0.0.249:8085/Snh_VrfListReq?name=default-domain:k8s-default:__public__:__public__ 2026-02-12 12:17:27,103 - DEBUG - VRF ids for VN __public__: {} 2026-02-12 12:17:27,195 - INFO - Creating Floating IP pool __fip_pool_public__ in API Server 2026-02-12 12:17:27,358 - DEBUG - Requesting: http://10.0.0.22:8082/domains 2026-02-12 12:17:27,364 - DEBUG - Requesting: http://10.0.0.22:8082/domain/168fb2bf-c66e-49fd-b422-7e20e5b6e896 2026-02-12 12:17:27,371 - DEBUG - Requesting: http://10.0.0.22:8082/project/3d7add91-f0e6-4cf3-9deb-70568bdf4098 2026-02-12 12:17:27,421 - DEBUG - Requesting: http://10.0.0.22:8082/virtual-network/482c6aaa-f318-44c0-9417-a9caa04995d4 2026-02-12 12:17:27,433 - DEBUG - Requesting: http://10.0.0.22:8082/floating-ip-pool/2fc2a0bf-980e-4e75-bf30-dc3f1f8f271e 2026-02-12 12:17:27,445 - DEBUG - Requesting: http://10.0.0.22:8082/domains 2026-02-12 12:17:27,451 - DEBUG - Requesting: http://10.0.0.22:8082/domain/168fb2bf-c66e-49fd-b422-7e20e5b6e896 2026-02-12 12:17:27,459 - DEBUG - Requesting: http://10.0.0.22:8082/project/3d7add91-f0e6-4cf3-9deb-70568bdf4098 2026-02-12 12:17:27,501 - DEBUG - Requesting: http://10.0.0.22:8082/virtual-network/482c6aaa-f318-44c0-9417-a9caa04995d4 2026-02-12 12:17:27,510 - DEBUG - FIP Pool __fip_pool_public__ found in API Server 2026-02-12 12:17:27,510 - INFO - Verification for FIP pool __fip_pool_public__ in API Server passed 2026-02-12 12:17:27,516 - DEBUG - Requesting: http://10.0.0.22:8083/Snh_IFMapTableShowReq?table_name=floating-ip-pool&search_string=floating-ip-pool:default-domain:k8s-default:__public__:__fip_pool_public__ 2026-02-12 12:17:27,521 - DEBUG - Control-node Ifmap-view has FIP pool __fip_pool_public__ information 2026-02-12 12:17:27,521 - INFO - Verification for FIP pool __fip_pool_public__ in Control-node passed 2026-02-12 12:17:27,524 - DEBUG - Ingress ctest-nginx-ingress-99513266 not present 2026-02-12 12:17:27,525 - INFO - Creating Ingress ctest-nginx-ingress-99513266 2026-02-12 12:17:27,533 - INFO - Created Ingress ctest-nginx-ingress-99513266 2026-02-12 12:17:27,537 - DEBUG - Cluster IP not yet seen for Ingress ctest-nginx-ingress-99513266 2026-02-12 12:17:30,543 - DEBUG - For Ingress ctest-nginx-ingress-99513266, Cluster IP: 10.47.255.249, External IPs ['79.43.114.60'] 2026-02-12 12:17:30,543 - INFO - Verifications in k8s passed for Ingress ctest-nginx-ingress-99513266 2026-02-12 12:17:30,543 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_LoadbalancerDatabaseList?x=c606be1c-4c0f-4783-bbc2-d4ee8972f8db 2026-02-12 12:17:30,549 - INFO - Ingress ctest-nginx-ingress-99513266 with uuid c606be1c-4c0f-4783-bbc2-d4ee8972f8db found in kube manager 2026-02-12 12:17:30,549 - INFO - Ingress ctest-nginx-ingress-99513266 verification passed 2026-02-12 12:17:30,552 - DEBUG - Pod ctest-busybox-pod-91939562 not present 2026-02-12 12:17:30,553 - INFO - Creating Pod ctest-busybox-pod-91939562 2026-02-12 12:17:30,576 - DEBUG - Node for Pod ctest-busybox-pod-91939562 not yet populated 2026-02-12 12:17:33,587 - DEBUG - Pod : ctest-busybox-pod-91939562 UUID is d56a308b-151e-45e7-bbfd-f85931ba4ea8 2026-02-12 12:17:33,593 - DEBUG - Pod ctest-nginx-pod-54189690 not in running state.Currently in Pending 2026-02-12 12:17:38,600 - DEBUG - Pod ctest-nginx-pod-54189690 not in running state.Currently in Pending 2026-02-12 12:17:43,617 - INFO - Pod ctest-nginx-pod-54189690 is in running state.Got IP 10.47.255.251 2026-02-12 12:17:43,650 - DEBUG - Pod ctest-nginx-pod-54189690 has vmi c8d2d834-080c-11f1-b57a-fa163e345d18 2026-02-12 12:17:43,650 - INFO - Verified pod ctest-nginx-pod-54189690 in contrail-api 2026-02-12 12:17:43,650 - DEBUG - Requesting: http://10.0.0.249:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0, 2026-02-12 12:17:43,669 - DEBUG - VMI c8d2d834-080c-11f1-b57a-fa163e345d18 is active in agent 10.0.0.249 2026-02-12 12:17:43,670 - INFO - Verified Pod ctest-nginx-pod-54189690 in agent 10.0.0.249 2026-02-12 12:17:43,670 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_VirtualMachineDatabaseList?x=e43c61c9-cf7a-4e24-a037-92ac16478fc3 2026-02-12 12:17:43,677 - INFO - Pod ctest-nginx-pod-54189690 with uuid e43c61c9-cf7a-4e24-a037-92ac16478fc3 found in kube manager 2026-02-12 12:17:43,677 - INFO - Pod ctest-nginx-pod-54189690 verification passed 2026-02-12 12:17:43,789 - DEBUG - [Pod ctest-nginx-pod-54189690] Cmd: echo ctest-nginx-pod-54189690 > /usr/share/nginx/html/index.html, Output: 2026-02-12 12:17:43,815 - INFO - Pod ctest-nginx-pod-82886736 is in running state.Got IP 10.47.255.250 2026-02-12 12:17:43,839 - DEBUG - Pod ctest-nginx-pod-82886736 has vmi c93ff45a-080c-11f1-b57a-fa163e345d18 2026-02-12 12:17:43,839 - INFO - Verified pod ctest-nginx-pod-82886736 in contrail-api 2026-02-12 12:17:43,840 - DEBUG - Requesting: http://10.0.0.249:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0, 2026-02-12 12:17:43,857 - DEBUG - VMI c93ff45a-080c-11f1-b57a-fa163e345d18 is active in agent 10.0.0.249 2026-02-12 12:17:43,857 - INFO - Verified Pod ctest-nginx-pod-82886736 in agent 10.0.0.249 2026-02-12 12:17:43,857 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_VirtualMachineDatabaseList?x=206e6c0d-1ebe-45ba-a838-c287160fed17 2026-02-12 12:17:43,865 - INFO - Pod ctest-nginx-pod-82886736 with uuid 206e6c0d-1ebe-45ba-a838-c287160fed17 found in kube manager 2026-02-12 12:17:43,865 - INFO - Pod ctest-nginx-pod-82886736 verification passed 2026-02-12 12:17:43,928 - DEBUG - [Pod ctest-nginx-pod-82886736] Cmd: echo ctest-nginx-pod-82886736 > /usr/share/nginx/html/index.html, Output: 2026-02-12 12:17:43,938 - DEBUG - Pod ctest-busybox-pod-91939562 not in running state.Currently in Pending 2026-02-12 12:17:48,948 - INFO - Pod ctest-busybox-pod-91939562 is in running state.Got IP 10.47.255.248 2026-02-12 12:17:48,962 - DEBUG - Pod ctest-busybox-pod-91939562 has vmi cda19d96-080c-11f1-b57a-fa163e345d18 2026-02-12 12:17:48,962 - INFO - Verified pod ctest-busybox-pod-91939562 in contrail-api 2026-02-12 12:17:48,962 - DEBUG - Requesting: http://10.0.0.249:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0, 2026-02-12 12:17:48,983 - DEBUG - VMI cda19d96-080c-11f1-b57a-fa163e345d18 is active in agent 10.0.0.249 2026-02-12 12:17:48,983 - INFO - Verified Pod ctest-busybox-pod-91939562 in agent 10.0.0.249 2026-02-12 12:17:48,984 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_VirtualMachineDatabaseList?x=d56a308b-151e-45e7-bbfd-f85931ba4ea8 2026-02-12 12:17:48,993 - INFO - Pod ctest-busybox-pod-91939562 with uuid d56a308b-151e-45e7-bbfd-f85931ba4ea8 found in kube manager 2026-02-12 12:17:48,993 - INFO - Pod ctest-busybox-pod-91939562 verification passed 2026-02-12 12:17:49,071 - DEBUG - [Pod ctest-busybox-pod-91939562] Cmd: wget http://10.47.255.249:80/ -O - -T 5 -t 1 , Output: ctest-nginx-pod-82886736 Connecting to 10.47.255.249:80 (10.47.255.249:80) writing to stdout - 100% |********************************| 25 0:00:00 ETA written to stdout 2026-02-12 12:17:49,071 - DEBUG - [Pod Pod ctest-busybox-pod-91939562] Cmd wget http://10.47.255.249:80/ -O - -T 5 -t 1 passed 2026-02-12 12:17:49,071 - DEBUG - [Pod Pod ctest-busybox-pod-91939562] Cmd output: ctest-nginx-pod-82886736 Connecting to 10.47.255.249:80 (10.47.255.249:80) writing to stdout - 100% |********************************| 25 0:00:00 ETA written to stdout 2026-02-12 12:17:49,143 - DEBUG - [Pod ctest-busybox-pod-91939562] Cmd: wget http://10.47.255.249:80/ -O - -T 5 -t 1 , Output: ctest-nginx-pod-82886736 Connecting to 10.47.255.249:80 (10.47.255.249:80) writing to stdout - 100% |********************************| 25 0:00:00 ETA written to stdout 2026-02-12 12:17:49,143 - DEBUG - [Pod Pod ctest-busybox-pod-91939562] Cmd wget http://10.47.255.249:80/ -O - -T 5 -t 1 passed 2026-02-12 12:17:49,144 - DEBUG - [Pod Pod ctest-busybox-pod-91939562] Cmd output: ctest-nginx-pod-82886736 Connecting to 10.47.255.249:80 (10.47.255.249:80) writing to stdout - 100% |********************************| 25 0:00:00 ETA written to stdout 2026-02-12 12:17:49,220 - DEBUG - [Pod ctest-busybox-pod-91939562] Cmd: wget http://10.47.255.249:80/ -O - -T 5 -t 1 , Output: ctest-nginx-pod-54189690 Connecting to 10.47.255.249:80 (10.47.255.249:80) writing to stdout - 100% |********************************| 25 0:00:00 ETA written to stdout 2026-02-12 12:17:49,220 - DEBUG - [Pod Pod ctest-busybox-pod-91939562] Cmd wget http://10.47.255.249:80/ -O - -T 5 -t 1 passed 2026-02-12 12:17:49,220 - DEBUG - [Pod Pod ctest-busybox-pod-91939562] Cmd output: ctest-nginx-pod-54189690 Connecting to 10.47.255.249:80 (10.47.255.249:80) writing to stdout - 100% |********************************| 25 0:00:00 ETA written to stdout 2026-02-12 12:17:49,220 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-nginx-pod-54189690': 1, 'ctest-nginx-pod-82886736': 2} 2026-02-12 12:17:49,221 - WARNING - Nothing to delete parallely 2026-02-12 12:17:49,222 - INFO - Deleting pod default:ctest-busybox-pod-91939562 2026-02-12 12:17:49,252 - DEBUG - Requesting: http://10.0.0.249:8085/Snh_VmListReq?uuid=d56a308b-151e-45e7-bbfd-f85931ba4ea8 2026-02-12 12:17:49,260 - WARNING - Pod uuid d56a308b-151e-45e7-bbfd-f85931ba4ea8 is still seen in agent 10.0.0.249 VM list 2026-02-12 12:17:54,261 - DEBUG - Requesting: http://10.0.0.249:8085/Snh_VmListReq?uuid=d56a308b-151e-45e7-bbfd-f85931ba4ea8 2026-02-12 12:17:54,270 - WARNING - Pod uuid d56a308b-151e-45e7-bbfd-f85931ba4ea8 is still seen in agent 10.0.0.249 VM list 2026-02-12 12:17:59,271 - DEBUG - Requesting: http://10.0.0.249:8085/Snh_VmListReq?uuid=d56a308b-151e-45e7-bbfd-f85931ba4ea8 2026-02-12 12:17:59,283 - WARNING - Pod uuid d56a308b-151e-45e7-bbfd-f85931ba4ea8 is still seen in agent 10.0.0.249 VM list 2026-02-12 12:18:04,284 - DEBUG - Requesting: http://10.0.0.249:8085/Snh_VmListReq?uuid=d56a308b-151e-45e7-bbfd-f85931ba4ea8 2026-02-12 12:18:04,294 - WARNING - Pod uuid d56a308b-151e-45e7-bbfd-f85931ba4ea8 is still seen in agent 10.0.0.249 VM list 2026-02-12 12:18:09,295 - DEBUG - Requesting: http://10.0.0.249:8085/Snh_VmListReq?uuid=d56a308b-151e-45e7-bbfd-f85931ba4ea8 2026-02-12 12:18:09,306 - WARNING - Pod uuid d56a308b-151e-45e7-bbfd-f85931ba4ea8 is still seen in agent 10.0.0.249 VM list 2026-02-12 12:18:14,306 - DEBUG - Requesting: http://10.0.0.249:8085/Snh_VmListReq?uuid=d56a308b-151e-45e7-bbfd-f85931ba4ea8 2026-02-12 12:18:14,316 - WARNING - Pod uuid d56a308b-151e-45e7-bbfd-f85931ba4ea8 is still seen in agent 10.0.0.249 VM list 2026-02-12 12:18:19,317 - DEBUG - Requesting: http://10.0.0.249:8085/Snh_VmListReq?uuid=d56a308b-151e-45e7-bbfd-f85931ba4ea8 2026-02-12 12:18:19,325 - WARNING - Pod uuid d56a308b-151e-45e7-bbfd-f85931ba4ea8 is still seen in agent 10.0.0.249 VM list 2026-02-12 12:18:24,327 - DEBUG - Requesting: http://10.0.0.249:8085/Snh_VmListReq?uuid=d56a308b-151e-45e7-bbfd-f85931ba4ea8 2026-02-12 12:18:24,337 - DEBUG - Pod d56a308b-151e-45e7-bbfd-f85931ba4ea8 is not in agent 10.0.0.249 VM list 2026-02-12 12:18:24,337 - DEBUG - Requesting: http://10.0.0.249:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:cda19d96-080c-11f1-b57a-fa163e345d18 2026-02-12 12:18:24,347 - DEBUG - VMI cda19d96-080c-11f1-b57a-fa163e345d18 is removed from agent 10.0.0.249 2026-02-12 12:18:24,347 - INFO - Verified that pod ctest-busybox-pod-91939562 is removed in agent 2026-02-12 12:18:24,347 - INFO - Deleting Ingress : ctest-nginx-ingress-99513266 2026-02-12 12:18:24,377 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_LoadbalancerDatabaseList?x=c606be1c-4c0f-4783-bbc2-d4ee8972f8db 2026-02-12 12:18:24,387 - ERROR - Ingress ctest-nginx-ingress-99513266 with uuid c606be1c-4c0f-4783-bbc2-d4ee8972f8db still found in kube manager 2026-02-12 12:18:25,388 - DEBUG - Requesting: http://10.0.0.22:8108/Snh_LoadbalancerDatabaseList?x=c606be1c-4c0f-4783-bbc2-d4ee8972f8db 2026-02-12 12:18:25,395 - INFO - Ingress ctest-nginx-ingress-99513266 with uuid c606be1c-4c0f-4783-bbc2-d4ee8972f8db deleted successfully from kube manager 2026-02-12 12:18:25,397 - INFO - Verifications on Ingress %s cleanup passed 2026-02-12 12:18:25,397 - INFO - Deleting the FIP pool __fip_pool_public__ 2026-02-12 12:18:25,628 - DEBUG - Requesting: http://10.0.0.22:8083/Snh_IFMapTableShowReq?table_name=floating-ip-pool&search_string=floating-ip-pool:default-domain:k8s-default:__public__:__fip_pool_public__ 2026-02-12 12:18:25,638 - DEBUG - Control-node Ifmap-view does not have FIP pool __fip_pool_public__ information 2026-02-12 12:18:25,639 - INFO - Deleting VN __public__ 2026-02-12 12:18:25,845 - DEBUG - Requesting: http://10.0.0.22:8082/routing-instance/59414b9b-48a6-4341-bea8-82dccbbf6521 2026-02-12 12:18:25,858 - DEBUG - Response Code: 404 2026-02-12 12:18:25,858 - DEBUG - Requesting: http://10.0.0.22:8082/domains 2026-02-12 12:18:25,877 - DEBUG - Requesting: http://10.0.0.22:8082/domain/168fb2bf-c66e-49fd-b422-7e20e5b6e896 2026-02-12 12:18:25,902 - DEBUG - Requesting: http://10.0.0.22:8082/project/3d7add91-f0e6-4cf3-9deb-70568bdf4098 2026-02-12 12:18:26,023 - INFO - Validated that VN __public__ is not found in API Server 2026-02-12 12:18:26,023 - DEBUG - Requesting: http://10.0.0.249:8085/Snh_VnListReq?name=default-domain:k8s-default:__public__ 2026-02-12 12:18:26,030 - DEBUG - Requesting: http://10.0.0.249:8085/Snh_VrfListReq?name=default-domain:k8s-default:__public__:__public__ 2026-02-12 12:18:26,037 - DEBUG - VN __public__ is not present in Agent 10.0.0.249 2026-02-12 12:18:26,037 - INFO - Validated that VN __public__ is not in any agent 2026-02-12 12:18:26,038 - DEBUG - Requesting: http://10.0.0.249:8085/Snh_VrfListReq?name=default-domain:k8s-default:__public__:__public__ 2026-02-12 12:18:26,046 - DEBUG - Requesting: http://10.0.0.22:8083/Snh_ShowRoutingInstanceReq?name=default-domain:k8s-default:__public__:__public__ 2026-02-12 12:18:26,052 - DEBUG - Requesting: http://10.0.0.22:8083/Snh_IFMapTableShowReq?table_name=virtual-network&search_string=virtual-network:default-domain:k8s-default:__public__ 2026-02-12 12:18:26,059 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN __public__ info 2026-02-12 12:18:26,059 - INFO - Deleting pod default:ctest-nginx-pod-82886736 2026-02-12 12:18:26,092 - DEBUG - Requesting: http://10.0.0.249:8085/Snh_VmListReq?uuid=206e6c0d-1ebe-45ba-a838-c287160fed17 2026-02-12 12:18:26,115 - WARNING - Pod uuid 206e6c0d-1ebe-45ba-a838-c287160fed17 is still seen in agent 10.0.0.249 VM list 2026-02-12 12:18:31,115 - DEBUG - Requesting: http://10.0.0.249:8085/Snh_VmListReq?uuid=206e6c0d-1ebe-45ba-a838-c287160fed17 2026-02-12 12:18:31,125 - DEBUG - Pod 206e6c0d-1ebe-45ba-a838-c287160fed17 is not in agent 10.0.0.249 VM list 2026-02-12 12:18:31,125 - DEBUG - Requesting: http://10.0.0.249:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:c93ff45a-080c-11f1-b57a-fa163e345d18 2026-02-12 12:18:31,133 - DEBUG - VMI c93ff45a-080c-11f1-b57a-fa163e345d18 is removed from agent 10.0.0.249 2026-02-12 12:18:31,133 - INFO - Verified that pod ctest-nginx-pod-82886736 is removed in agent 2026-02-12 12:18:31,135 - INFO - Deleting pod default:ctest-nginx-pod-54189690 2026-02-12 12:18:31,153 - DEBUG - Requesting: http://10.0.0.249:8085/Snh_VmListReq?uuid=e43c61c9-cf7a-4e24-a037-92ac16478fc3 2026-02-12 12:18:31,171 - WARNING - Pod uuid e43c61c9-cf7a-4e24-a037-92ac16478fc3 is still seen in agent 10.0.0.249 VM list 2026-02-12 12:18:36,172 - DEBUG - Requesting: http://10.0.0.249:8085/Snh_VmListReq?uuid=e43c61c9-cf7a-4e24-a037-92ac16478fc3 2026-02-12 12:18:36,189 - DEBUG - Pod e43c61c9-cf7a-4e24-a037-92ac16478fc3 is not in agent 10.0.0.249 VM list 2026-02-12 12:18:36,189 - DEBUG - Requesting: http://10.0.0.249:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:c8d2d834-080c-11f1-b57a-fa163e345d18 2026-02-12 12:18:36,197 - DEBUG - VMI c8d2d834-080c-11f1-b57a-fa163e345d18 is removed from agent 10.0.0.249 2026-02-12 12:18:36,197 - INFO - Verified that pod ctest-nginx-pod-54189690 is removed in agent 2026-02-12 12:18:36,197 - INFO - Deleting service : ctest-nginx-svc-64978886 2026-02-12 12:18:36,494 - DEBUG - Requesting: http://10.0.0.249:8085/Snh_AgentXmppConnectionStatusReq? 2026-02-12 12:18:36,507 - DEBUG - No XMPP flaps were noticed during the test