2025-10-31 00:00:24,129 - DEBUG - [10.0.0.240]: Running cmd : hostname 2025-10-31 00:00:24,384 - DEBUG - Output : cn-jenkins-deploy-platform-juju-k8s-1660-1 2025-10-31 00:00:24,384 - DEBUG - [10.0.0.240]: Running cmd : hostname -f 2025-10-31 00:00:24,422 - DEBUG - Output : cn-jenkins-deploy-platform-juju-k8s-1660-1 2025-10-31 00:00:24,422 - DEBUG - [10.0.0.240]: Running cmd : docker ps 2>/dev/null | grep -v "/pause\|/usr/bin/pod\|nova_api_\|contrail.*init\|init.*contrail\|provisioner\|placement" | awk '{print $NF}' 2025-10-31 00:00:24,487 - DEBUG - Output : NAMES contrail_test_pQsl5Dp8W contrail-kubernetes-master-kubemanager-1 webui-web-1 webui-job-1 analytics_database-query-engine-1 analytics_database-cassandra-1 analytics_database-nodemgr-1 analytics_snmp-topology-1 analytics_snmp-nodemgr-1 analytics_snmp-snmp-collector-1 analytics_alarm-kafka-1 analytics_alarm-nodemgr-1 analytics_alarm-alarm-gen-1 analytics-collector-1 analytics-nodemgr-1 analytics-api-1 redis-redis-1 control-named-1 control-control-1 control-dns-1 control-nodemgr-1 config_database-rabbitmq-1 config_database-nodemgr-1 config_database-cassandra-1 config_database-zookeeper-1 config_api-devicemgr-1 config_api-nodemgr-1 config_api-dnsmasq-1 config_api-api-1 config_api-schema-1 config_api-svcmonitor-1 2025-10-31 00:00:24,487 - DEBUG - [10.0.0.240]: Running cmd : ip -4 -o addr show | awk '{print $4}' 2025-10-31 00:00:24,558 - DEBUG - Output : 127.0.0.1/8 10.0.0.240/24 252.240.0.1/8 10.180.237.1/24 172.17.0.1/16 2025-10-31 00:00:24,558 - DEBUG - [10.0.0.240]: Running cmd : getent hosts 10.0.0.240 | head -n 1 | awk '{print $2}' 2025-10-31 00:00:24,603 - DEBUG - Output : cn-jenkins-deploy-platform-juju-k8s-1660-1 2025-10-31 00:00:24,703 - DEBUG - [10.0.0.155]: Running cmd : hostname 2025-10-31 00:00:24,939 - DEBUG - Output : an-jenkins-deploy-platform-juju-k8s-1660-1 2025-10-31 00:00:24,939 - DEBUG - [10.0.0.155]: Running cmd : hostname -f 2025-10-31 00:00:24,973 - DEBUG - Output : an-jenkins-deploy-platform-juju-k8s-1660-1 2025-10-31 00:00:24,973 - DEBUG - [10.0.0.155]: Running cmd : docker ps 2>/dev/null | grep -v "/pause\|/usr/bin/pod\|nova_api_\|contrail.*init\|init.*contrail\|provisioner\|placement" | awk '{print $NF}' 2025-10-31 00:00:25,051 - DEBUG - Output : NAMES vrouter-vrouter-agent-1 vrouter-nodemgr-1 2025-10-31 00:00:25,051 - DEBUG - [10.0.0.155]: Running cmd : ip -4 -o addr show dev vhost0 | awk '{print $4}' 2025-10-31 00:00:25,085 - DEBUG - Output : 10.0.0.155/24 2025-10-31 00:00:25,085 - DEBUG - [10.0.0.155]: Running cmd : getent hosts 10.0.0.155 | head -n 1 | awk '{print $2}' 2025-10-31 00:00:25,119 - DEBUG - Output : an-jenkins-deploy-platform-juju-k8s-1660-1 2025-10-31 00:00:25,228 - INFO - Using existing project ['default-domain', 'k8s-default'](a6bdc041-319f-481f-ba83-5cbb1f96e58f) 2025-10-31 00:00:25,329 - INFO - Using existing project ['default-domain', 'k8s-default'](a6bdc041-319f-481f-ba83-5cbb1f96e58f) 2025-10-31 00:00:25,435 - INFO - ================================================================================ 2025-10-31 00:00:25,435 - INFO - STARTING TEST : test_ingress_ip_assignment 2025-10-31 00:00:25,436 - INFO - TEST DESCRIPTION : Verify that Ingress gets a CLuster IP which is reachable to Pods in same namespace. Also verify that a Floating IP is assigned to the Ingress from the Public FIP poo. Steps: 1. Create a service with 2 pods running nginx 2. Create an ingress out of this service 3. From another Pod do a wget on the ingress Cluster ip Validate that Ingress get a IP from Public FIP pool which might/might not be accessible. Validate that service and its loadbalancing work 2025-10-31 00:00:25,629 - DEBUG - Requesting: http://10.0.0.155:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-31 00:00:25,635 - DEBUG - Nothing to compare xmpp stats {'10.0.0.155': {'10.0.0.240': '0'}} with 2025-10-31 00:00:25,635 - INFO - Initial checks done. Running the testcase now 2025-10-31 00:00:25,635 - INFO - 2025-10-31 00:00:25,638 - INFO - Namespace default already exists 2025-10-31 00:00:25,639 - DEBUG - [10.0.0.240]: Running cmd : docker exec --privileged -it contrail-kubernetes-master-kubemanager-1 /bin/bash -c 'grep "^[ ]*cluster_project" /etc/contrail/contrail-kubernetes.conf' 2025-10-31 00:00:25,747 - DEBUG - Output : cluster_project={} 2025-10-31 00:00:25,786 - INFO - Project default-domain:k8s-default for Namespace default is seen in contrail-api 2025-10-31 00:00:25,787 - DEBUG - Project uuid in Contrail is a6bdc041-319f-481f-ba83-5cbb1f96e58f 2025-10-31 00:00:25,787 - DEBUG - [10.0.0.240]: Running cmd : docker exec --privileged -it contrail-kubernetes-master-kubemanager-1 /bin/bash -c 'netstat -antp | grep :8108 | grep LISTEN' 2025-10-31 00:00:25,903 - DEBUG - Output : tcp 0 0 0.0.0.0:8108 0.0.0.0:* LISTEN 1/python3 2025-10-31 00:00:25,903 - DEBUG - Requesting: http://10.0.0.240:8108/Snh_NamespaceDatabaseList?namespace_uuid=df49daf9-acb4-42e5-9a25-43b240ae4303 2025-10-31 00:00:25,907 - INFO - Namespace default with uuid df49daf9-acb4-42e5-9a25-43b240ae4303 found in kube manager 2025-10-31 00:00:25,907 - INFO - Namespace default verification passed 2025-10-31 00:00:25,913 - DEBUG - Service ctest-nginx-svc-88941458 not present 2025-10-31 00:00:25,913 - INFO - Creating service ctest-nginx-svc-88941458 2025-10-31 00:00:25,941 - INFO - Created Service ctest-nginx-svc-88941458 2025-10-31 00:00:25,949 - DEBUG - Pod ctest-nginx-pod-50356765 not present 2025-10-31 00:00:25,950 - INFO - Creating Pod ctest-nginx-pod-50356765 2025-10-31 00:00:26,027 - DEBUG - Pod : ctest-nginx-pod-50356765 UUID is 8f4d6573-5168-4a0b-870f-12a5c58cc94a 2025-10-31 00:00:26,031 - DEBUG - Pod ctest-nginx-pod-40880924 not present 2025-10-31 00:00:26,032 - INFO - Creating Pod ctest-nginx-pod-40880924 2025-10-31 00:00:26,075 - DEBUG - Pod : ctest-nginx-pod-40880924 UUID is 275bd465-ca05-4fe3-b3e8-387c6e847e40 2025-10-31 00:00:26,230 - INFO - Created VN __public__, UUID :b33ff998-b00e-4190-8e56-3574477ac8d1 2025-10-31 00:00:26,456 - DEBUG - Requesting: http://10.0.0.240:8082/domains 2025-10-31 00:00:26,461 - DEBUG - Requesting: http://10.0.0.240:8082/domain/f6be4a7b-7730-4452-98a2-b3a7d5551645 2025-10-31 00:00:26,475 - DEBUG - Requesting: http://10.0.0.240:8082/project/a6bdc041-319f-481f-ba83-5cbb1f96e58f 2025-10-31 00:00:26,530 - DEBUG - Requesting: http://10.0.0.240:8082/virtual-network/b33ff998-b00e-4190-8e56-3574477ac8d1 2025-10-31 00:00:26,539 - DEBUG - Requesting: http://10.0.0.240:8082/virtual-network/b33ff998-b00e-4190-8e56-3574477ac8d1 2025-10-31 00:00:26,546 - DEBUG - Requesting: http://10.0.0.240:8082/routing-instance/ca18061d-5068-44ca-83e0-c8f290d6d8ec 2025-10-31 00:00:26,551 - DEBUG - Requesting: http://10.0.0.240:8082/routing-instance/ca18061d-5068-44ca-83e0-c8f290d6d8ec 2025-10-31 00:00:26,560 - DEBUG - Route Targets: [] 2025-10-31 00:00:26,560 - DEBUG - RT names not yet present for VN __public__ 2025-10-31 00:00:31,561 - DEBUG - Requesting: http://10.0.0.240:8082/domains 2025-10-31 00:00:31,566 - DEBUG - Requesting: http://10.0.0.240:8082/domain/f6be4a7b-7730-4452-98a2-b3a7d5551645 2025-10-31 00:00:31,573 - DEBUG - Requesting: http://10.0.0.240:8082/project/a6bdc041-319f-481f-ba83-5cbb1f96e58f 2025-10-31 00:00:31,613 - DEBUG - Requesting: http://10.0.0.240:8082/virtual-network/b33ff998-b00e-4190-8e56-3574477ac8d1 2025-10-31 00:00:31,620 - DEBUG - Requesting: http://10.0.0.240:8082/virtual-network/b33ff998-b00e-4190-8e56-3574477ac8d1 2025-10-31 00:00:31,627 - DEBUG - Requesting: http://10.0.0.240:8082/routing-instance/ca18061d-5068-44ca-83e0-c8f290d6d8ec 2025-10-31 00:00:31,633 - DEBUG - Requesting: http://10.0.0.240:8082/routing-instance/ca18061d-5068-44ca-83e0-c8f290d6d8ec 2025-10-31 00:00:31,638 - DEBUG - Requesting: http://10.0.0.240:8082/route-target/4d455c53-7509-4824-8a16-556760c08e5c 2025-10-31 00:00:31,642 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-10-31 00:00:31,643 - DEBUG - Requesting: http://10.0.0.240:8082/virtual-network/b33ff998-b00e-4190-8e56-3574477ac8d1 2025-10-31 00:00:31,649 - DEBUG - Requesting: http://10.0.0.240:8082/routing-instance/ca18061d-5068-44ca-83e0-c8f290d6d8ec 2025-10-31 00:00:31,659 - INFO - Verified VN network id 13 for VN b33ff998-b00e-4190-8e56-3574477ac8d1 2025-10-31 00:00:31,659 - INFO - Verifications in API Server for VN __public__ passed 2025-10-31 00:00:31,659 - DEBUG - Requesting: http://10.0.0.240:8082/virtual-network/b33ff998-b00e-4190-8e56-3574477ac8d1 2025-10-31 00:00:31,665 - DEBUG - Requesting: http://10.0.0.240:8082/routing-instance/ca18061d-5068-44ca-83e0-c8f290d6d8ec 2025-10-31 00:00:31,670 - DEBUG - Requesting: http://10.0.0.240:8082/routing-instance/ca18061d-5068-44ca-83e0-c8f290d6d8ec 2025-10-31 00:00:31,674 - DEBUG - Requesting: http://10.0.0.240:8082/route-target/4d455c53-7509-4824-8a16-556760c08e5c 2025-10-31 00:00:31,679 - DEBUG - Requesting: http://10.0.0.240:8083/Snh_IFMapTableShowReq?table_name=virtual-network&search_string=virtual-network:default-domain:k8s-default:__public__ 2025-10-31 00:00:31,684 - DEBUG - Control-node 10.0.0.240 : VN object is : {'node_name': 'virtual-network:default-domain:k8s-default:__public__', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'cloud-admin', 'owner-access': '7', 'group': 'cloud-admin-group', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '12916316690505941392', 'uuid-lslong': '10256443974914590929'}, 'enable': 'true', 'created': '2025-10-31T00:00:26', 'last-modified': '2025-10-31T00:00:26', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:05.282204'} 2025-10-31 00:00:31,684 - DEBUG - Requesting: http://10.0.0.240:8083/Snh_ShowRoutingInstanceReq?name=default-domain:k8s-default:__public__:__public__ 2025-10-31 00:00:31,691 - DEBUG - Route Targets: ['target:64512:8000006'] 2025-10-31 00:00:31,691 - INFO - On all control nodes, Config, RI and RT verification for VN __public__ passed 2025-10-31 00:00:31,692 - DEBUG - ====Verifying policy data for __public__ in API_Server ====== 2025-10-31 00:00:31,692 - DEBUG - Requesting: http://10.0.0.240:8082/domains 2025-10-31 00:00:31,696 - DEBUG - Requesting: http://10.0.0.240:8082/domain/f6be4a7b-7730-4452-98a2-b3a7d5551645 2025-10-31 00:00:31,703 - DEBUG - Requesting: http://10.0.0.240:8082/project/a6bdc041-319f-481f-ba83-5cbb1f96e58f 2025-10-31 00:00:31,737 - DEBUG - Requesting: http://10.0.0.240:8082/virtual-network/b33ff998-b00e-4190-8e56-3574477ac8d1 2025-10-31 00:00:31,745 - DEBUG - =>VN __public__ has no policy to be verified 2025-10-31 00:00:31,745 - DEBUG - Verifying the vn in opserver 2025-10-31 00:00:31,745 - DEBUG - Verifying the default-domain:k8s-default:__public__ virtual network link through opserver 10.0.0.240 2025-10-31 00:00:31,745 - DEBUG - Requesting: http://10.0.0.240:8081/analytics/uves/virtual-networks 2025-10-31 00:00:31,749 - DEBUG - vn link and name as {'name': 'default-domain:k8s-default:__public__', 'href': 'http://10.0.0.240:8081/analytics/uves/virtual-network/default-domain:k8s-default:__public__?flat'} 2025-10-31 00:00:31,749 - INFO - Validated that VN default-domain:k8s-default:__public__ is found in opserver 2025-10-31 00:00:31,749 - DEBUG - Requesting: http://10.0.0.155:8085/Snh_VrfListReq?name=default-domain:k8s-default:__public__:__public__ 2025-10-31 00:00:31,757 - DEBUG - Requesting: http://10.0.0.155:8085/Snh_VrfListReq?name=default-domain:k8s-default:__public__:__public__ 2025-10-31 00:00:31,761 - DEBUG - Do not have enough data to verify VN in agent 2025-10-31 00:00:31,761 - DEBUG - Requesting: http://10.0.0.155:8085/Snh_VrfListReq?name=default-domain:k8s-default:__public__:__public__ 2025-10-31 00:00:31,766 - DEBUG - VRF ids for VN __public__: {} 2025-10-31 00:00:31,851 - INFO - Creating Floating IP pool __fip_pool_public__ in API Server 2025-10-31 00:00:31,997 - DEBUG - Requesting: http://10.0.0.240:8082/domains 2025-10-31 00:00:32,002 - DEBUG - Requesting: http://10.0.0.240:8082/domain/f6be4a7b-7730-4452-98a2-b3a7d5551645 2025-10-31 00:00:32,008 - DEBUG - Requesting: http://10.0.0.240:8082/project/a6bdc041-319f-481f-ba83-5cbb1f96e58f 2025-10-31 00:00:32,043 - DEBUG - Requesting: http://10.0.0.240:8082/virtual-network/b33ff998-b00e-4190-8e56-3574477ac8d1 2025-10-31 00:00:32,053 - DEBUG - Requesting: http://10.0.0.240:8082/floating-ip-pool/38eb3562-137a-4546-b119-9f00d06b7139 2025-10-31 00:00:32,058 - DEBUG - Requesting: http://10.0.0.240:8082/domains 2025-10-31 00:00:32,062 - DEBUG - Requesting: http://10.0.0.240:8082/domain/f6be4a7b-7730-4452-98a2-b3a7d5551645 2025-10-31 00:00:32,068 - DEBUG - Requesting: http://10.0.0.240:8082/project/a6bdc041-319f-481f-ba83-5cbb1f96e58f 2025-10-31 00:00:32,104 - DEBUG - Requesting: http://10.0.0.240:8082/virtual-network/b33ff998-b00e-4190-8e56-3574477ac8d1 2025-10-31 00:00:32,112 - DEBUG - FIP Pool __fip_pool_public__ found in API Server 2025-10-31 00:00:32,112 - INFO - Verification for FIP pool __fip_pool_public__ in API Server passed 2025-10-31 00:00:32,117 - DEBUG - Requesting: http://10.0.0.240:8083/Snh_IFMapTableShowReq?table_name=floating-ip-pool&search_string=floating-ip-pool:default-domain:k8s-default:__public__:__fip_pool_public__ 2025-10-31 00:00:32,123 - DEBUG - Control-node Ifmap-view has FIP pool __fip_pool_public__ information 2025-10-31 00:00:32,123 - INFO - Verification for FIP pool __fip_pool_public__ in Control-node passed 2025-10-31 00:00:32,125 - DEBUG - Ingress ctest-nginx-ingress-82067802 not present 2025-10-31 00:00:32,126 - INFO - Creating Ingress ctest-nginx-ingress-82067802 2025-10-31 00:00:32,134 - INFO - Created Ingress ctest-nginx-ingress-82067802 2025-10-31 00:00:32,137 - DEBUG - Cluster IP not yet seen for Ingress ctest-nginx-ingress-82067802 2025-10-31 00:00:35,142 - DEBUG - For Ingress ctest-nginx-ingress-82067802, Cluster IP: 10.47.255.249, External IPs ['166.119.203.188'] 2025-10-31 00:00:35,143 - INFO - Verifications in k8s passed for Ingress ctest-nginx-ingress-82067802 2025-10-31 00:00:35,143 - DEBUG - Requesting: http://10.0.0.240:8108/Snh_LoadbalancerDatabaseList?x=99c6eec2-b8bd-47c7-bdfe-52c345c5d251 2025-10-31 00:00:35,148 - INFO - Ingress ctest-nginx-ingress-82067802 with uuid 99c6eec2-b8bd-47c7-bdfe-52c345c5d251 found in kube manager 2025-10-31 00:00:35,149 - INFO - Ingress ctest-nginx-ingress-82067802 verification passed 2025-10-31 00:00:35,151 - DEBUG - Pod ctest-busybox-pod-51774350 not present 2025-10-31 00:00:35,151 - INFO - Creating Pod ctest-busybox-pod-51774350 2025-10-31 00:00:35,201 - DEBUG - Pod : ctest-busybox-pod-51774350 UUID is fd884bae-f130-4ee4-900c-8c6f35323784 2025-10-31 00:00:35,211 - DEBUG - Pod ctest-nginx-pod-50356765 not in running state.Currently in Pending 2025-10-31 00:00:40,219 - INFO - Pod ctest-nginx-pod-50356765 is in running state.Got IP 10.47.255.251 2025-10-31 00:00:40,232 - DEBUG - Pod ctest-nginx-pod-50356765 has vmi 9b828ef0-b5ec-11f0-9b35-fa163ecd12f8 2025-10-31 00:00:40,232 - INFO - Verified pod ctest-nginx-pod-50356765 in contrail-api 2025-10-31 00:00:40,232 - DEBUG - Requesting: http://10.0.0.155:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0, 2025-10-31 00:00:40,242 - DEBUG - VMI 9b828ef0-b5ec-11f0-9b35-fa163ecd12f8 is active in agent 10.0.0.155 2025-10-31 00:00:40,242 - INFO - Verified Pod ctest-nginx-pod-50356765 in agent 10.0.0.155 2025-10-31 00:00:40,242 - DEBUG - Requesting: http://10.0.0.240:8108/Snh_VirtualMachineDatabaseList?x=8f4d6573-5168-4a0b-870f-12a5c58cc94a 2025-10-31 00:00:40,246 - INFO - Pod ctest-nginx-pod-50356765 with uuid 8f4d6573-5168-4a0b-870f-12a5c58cc94a found in kube manager 2025-10-31 00:00:40,246 - INFO - Pod ctest-nginx-pod-50356765 verification passed 2025-10-31 00:00:40,315 - DEBUG - [Pod ctest-nginx-pod-50356765] Cmd: echo ctest-nginx-pod-50356765 > /usr/share/nginx/html/index.html, Output: 2025-10-31 00:00:40,321 - INFO - Pod ctest-nginx-pod-40880924 is in running state.Got IP 10.47.255.250 2025-10-31 00:00:40,332 - DEBUG - Pod ctest-nginx-pod-40880924 has vmi 9be8023a-b5ec-11f0-9b35-fa163ecd12f8 2025-10-31 00:00:40,332 - INFO - Verified pod ctest-nginx-pod-40880924 in contrail-api 2025-10-31 00:00:40,333 - DEBUG - Requesting: http://10.0.0.155:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0, 2025-10-31 00:00:40,342 - DEBUG - VMI 9be8023a-b5ec-11f0-9b35-fa163ecd12f8 is active in agent 10.0.0.155 2025-10-31 00:00:40,342 - INFO - Verified Pod ctest-nginx-pod-40880924 in agent 10.0.0.155 2025-10-31 00:00:40,342 - DEBUG - Requesting: http://10.0.0.240:8108/Snh_VirtualMachineDatabaseList?x=275bd465-ca05-4fe3-b3e8-387c6e847e40 2025-10-31 00:00:40,345 - INFO - Pod ctest-nginx-pod-40880924 with uuid 275bd465-ca05-4fe3-b3e8-387c6e847e40 found in kube manager 2025-10-31 00:00:40,346 - INFO - Pod ctest-nginx-pod-40880924 verification passed 2025-10-31 00:00:40,402 - DEBUG - [Pod ctest-nginx-pod-40880924] Cmd: echo ctest-nginx-pod-40880924 > /usr/share/nginx/html/index.html, Output: 2025-10-31 00:00:40,408 - DEBUG - Pod ctest-busybox-pod-51774350 not in running state.Currently in Pending 2025-10-31 00:00:45,422 - DEBUG - Pod ctest-busybox-pod-51774350 not in running state.Currently in Pending 2025-10-31 00:00:50,429 - DEBUG - Pod ctest-busybox-pod-51774350 not in running state.Currently in Pending 2025-10-31 00:00:55,438 - INFO - Pod ctest-busybox-pod-51774350 is in running state.Got IP 10.47.255.248 2025-10-31 00:00:55,489 - DEBUG - Pod ctest-busybox-pod-51774350 has vmi a0398660-b5ec-11f0-9b35-fa163ecd12f8 2025-10-31 00:00:55,489 - INFO - Verified pod ctest-busybox-pod-51774350 in contrail-api 2025-10-31 00:00:55,489 - DEBUG - Requesting: http://10.0.0.155:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0, 2025-10-31 00:00:55,499 - DEBUG - VMI a0398660-b5ec-11f0-9b35-fa163ecd12f8 is active in agent 10.0.0.155 2025-10-31 00:00:55,499 - INFO - Verified Pod ctest-busybox-pod-51774350 in agent 10.0.0.155 2025-10-31 00:00:55,499 - DEBUG - Requesting: http://10.0.0.240:8108/Snh_VirtualMachineDatabaseList?x=fd884bae-f130-4ee4-900c-8c6f35323784 2025-10-31 00:00:55,503 - INFO - Pod ctest-busybox-pod-51774350 with uuid fd884bae-f130-4ee4-900c-8c6f35323784 found in kube manager 2025-10-31 00:00:55,503 - INFO - Pod ctest-busybox-pod-51774350 verification passed 2025-10-31 00:00:55,556 - DEBUG - [Pod ctest-busybox-pod-51774350] Cmd: wget http://10.47.255.249:80/ -O - -T 5 -t 1 , Output: ctest-nginx-pod-40880924 Connecting to 10.47.255.249:80 (10.47.255.249:80) writing to stdout - 100% |********************************| 25 0:00:00 ETA written to stdout 2025-10-31 00:00:55,556 - DEBUG - [Pod Pod ctest-busybox-pod-51774350] Cmd wget http://10.47.255.249:80/ -O - -T 5 -t 1 passed 2025-10-31 00:00:55,556 - DEBUG - [Pod Pod ctest-busybox-pod-51774350] Cmd output: ctest-nginx-pod-40880924 Connecting to 10.47.255.249:80 (10.47.255.249:80) writing to stdout - 100% |********************************| 25 0:00:00 ETA written to stdout 2025-10-31 00:00:55,606 - DEBUG - [Pod ctest-busybox-pod-51774350] Cmd: wget http://10.47.255.249:80/ -O - -T 5 -t 1 , Output: Connecting to 10.47.255.249:80 (10.47.255.249:80) writing to stdout - 100% |********************************| 25 0:00:00 ETA written to stdout ctest-nginx-pod-40880924 2025-10-31 00:00:55,606 - DEBUG - [Pod Pod ctest-busybox-pod-51774350] Cmd wget http://10.47.255.249:80/ -O - -T 5 -t 1 passed 2025-10-31 00:00:55,606 - DEBUG - [Pod Pod ctest-busybox-pod-51774350] Cmd output: Connecting to 10.47.255.249:80 (10.47.255.249:80) writing to stdout - 100% |********************************| 25 0:00:00 ETA written to stdout ctest-nginx-pod-40880924 2025-10-31 00:00:55,660 - DEBUG - [Pod ctest-busybox-pod-51774350] Cmd: wget http://10.47.255.249:80/ -O - -T 5 -t 1 , Output: Connecting to 10.47.255.249:80 (10.47.255.249:80) writing to stdout - 100% |********************************| 25 0:00:00 ETA written to stdout ctest-nginx-pod-50356765 2025-10-31 00:00:55,662 - DEBUG - [Pod Pod ctest-busybox-pod-51774350] Cmd wget http://10.47.255.249:80/ -O - -T 5 -t 1 passed 2025-10-31 00:00:55,662 - DEBUG - [Pod Pod ctest-busybox-pod-51774350] Cmd output: Connecting to 10.47.255.249:80 (10.47.255.249:80) writing to stdout - 100% |********************************| 25 0:00:00 ETA written to stdout ctest-nginx-pod-50356765 2025-10-31 00:00:55,662 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-nginx-pod-50356765': 1, 'ctest-nginx-pod-40880924': 2} 2025-10-31 00:00:55,663 - WARNING - Nothing to delete parallely 2025-10-31 00:00:55,663 - INFO - Deleting pod default:ctest-busybox-pod-51774350 2025-10-31 00:00:55,675 - DEBUG - Requesting: http://10.0.0.155:8085/Snh_VmListReq?uuid=fd884bae-f130-4ee4-900c-8c6f35323784 2025-10-31 00:00:55,681 - WARNING - Pod uuid fd884bae-f130-4ee4-900c-8c6f35323784 is still seen in agent 10.0.0.155 VM list 2025-10-31 00:01:00,682 - DEBUG - Requesting: http://10.0.0.155:8085/Snh_VmListReq?uuid=fd884bae-f130-4ee4-900c-8c6f35323784 2025-10-31 00:01:00,688 - WARNING - Pod uuid fd884bae-f130-4ee4-900c-8c6f35323784 is still seen in agent 10.0.0.155 VM list 2025-10-31 00:01:05,689 - DEBUG - Requesting: http://10.0.0.155:8085/Snh_VmListReq?uuid=fd884bae-f130-4ee4-900c-8c6f35323784 2025-10-31 00:01:05,695 - WARNING - Pod uuid fd884bae-f130-4ee4-900c-8c6f35323784 is still seen in agent 10.0.0.155 VM list 2025-10-31 00:01:10,696 - DEBUG - Requesting: http://10.0.0.155:8085/Snh_VmListReq?uuid=fd884bae-f130-4ee4-900c-8c6f35323784 2025-10-31 00:01:10,705 - WARNING - Pod uuid fd884bae-f130-4ee4-900c-8c6f35323784 is still seen in agent 10.0.0.155 VM list 2025-10-31 00:01:15,706 - DEBUG - Requesting: http://10.0.0.155:8085/Snh_VmListReq?uuid=fd884bae-f130-4ee4-900c-8c6f35323784 2025-10-31 00:01:15,714 - WARNING - Pod uuid fd884bae-f130-4ee4-900c-8c6f35323784 is still seen in agent 10.0.0.155 VM list 2025-10-31 00:01:20,715 - DEBUG - Requesting: http://10.0.0.155:8085/Snh_VmListReq?uuid=fd884bae-f130-4ee4-900c-8c6f35323784 2025-10-31 00:01:20,721 - WARNING - Pod uuid fd884bae-f130-4ee4-900c-8c6f35323784 is still seen in agent 10.0.0.155 VM list 2025-10-31 00:01:25,722 - DEBUG - Requesting: http://10.0.0.155:8085/Snh_VmListReq?uuid=fd884bae-f130-4ee4-900c-8c6f35323784 2025-10-31 00:01:25,732 - WARNING - Pod uuid fd884bae-f130-4ee4-900c-8c6f35323784 is still seen in agent 10.0.0.155 VM list 2025-10-31 00:01:30,733 - DEBUG - Requesting: http://10.0.0.155:8085/Snh_VmListReq?uuid=fd884bae-f130-4ee4-900c-8c6f35323784 2025-10-31 00:01:30,739 - DEBUG - Pod fd884bae-f130-4ee4-900c-8c6f35323784 is not in agent 10.0.0.155 VM list 2025-10-31 00:01:30,739 - DEBUG - Requesting: http://10.0.0.155:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:a0398660-b5ec-11f0-9b35-fa163ecd12f8 2025-10-31 00:01:30,744 - DEBUG - VMI a0398660-b5ec-11f0-9b35-fa163ecd12f8 is removed from agent 10.0.0.155 2025-10-31 00:01:30,744 - INFO - Verified that pod ctest-busybox-pod-51774350 is removed in agent 2025-10-31 00:01:30,744 - INFO - Deleting Ingress : ctest-nginx-ingress-82067802 2025-10-31 00:01:30,753 - DEBUG - Requesting: http://10.0.0.240:8108/Snh_LoadbalancerDatabaseList?x=99c6eec2-b8bd-47c7-bdfe-52c345c5d251 2025-10-31 00:01:30,758 - ERROR - Ingress ctest-nginx-ingress-82067802 with uuid 99c6eec2-b8bd-47c7-bdfe-52c345c5d251 still found in kube manager 2025-10-31 00:01:31,759 - DEBUG - Requesting: http://10.0.0.240:8108/Snh_LoadbalancerDatabaseList?x=99c6eec2-b8bd-47c7-bdfe-52c345c5d251 2025-10-31 00:01:31,762 - INFO - Ingress ctest-nginx-ingress-82067802 with uuid 99c6eec2-b8bd-47c7-bdfe-52c345c5d251 deleted successfully from kube manager 2025-10-31 00:01:31,762 - INFO - Verifications on Ingress %s cleanup passed 2025-10-31 00:01:31,763 - INFO - Deleting the FIP pool __fip_pool_public__ 2025-10-31 00:01:31,860 - DEBUG - Requesting: http://10.0.0.240:8083/Snh_IFMapTableShowReq?table_name=floating-ip-pool&search_string=floating-ip-pool:default-domain:k8s-default:__public__:__fip_pool_public__ 2025-10-31 00:01:31,865 - DEBUG - Control-node Ifmap-view does not have FIP pool __fip_pool_public__ information 2025-10-31 00:01:31,865 - INFO - Deleting VN __public__ 2025-10-31 00:01:31,988 - DEBUG - Requesting: http://10.0.0.240:8082/routing-instance/ca18061d-5068-44ca-83e0-c8f290d6d8ec 2025-10-31 00:01:31,992 - DEBUG - Response Code: 404 2025-10-31 00:01:31,992 - DEBUG - Requesting: http://10.0.0.240:8082/domains 2025-10-31 00:01:31,996 - DEBUG - Requesting: http://10.0.0.240:8082/domain/f6be4a7b-7730-4452-98a2-b3a7d5551645 2025-10-31 00:01:32,015 - DEBUG - Requesting: http://10.0.0.240:8082/project/a6bdc041-319f-481f-ba83-5cbb1f96e58f 2025-10-31 00:01:32,060 - INFO - Validated that VN __public__ is not found in API Server 2025-10-31 00:01:32,060 - DEBUG - Requesting: http://10.0.0.155:8085/Snh_VnListReq?name=default-domain:k8s-default:__public__ 2025-10-31 00:01:32,068 - DEBUG - Requesting: http://10.0.0.155:8085/Snh_VrfListReq?name=default-domain:k8s-default:__public__:__public__ 2025-10-31 00:01:32,082 - DEBUG - VN __public__ is not present in Agent 10.0.0.155 2025-10-31 00:01:32,082 - INFO - Validated that VN __public__ is not in any agent 2025-10-31 00:01:32,083 - DEBUG - Requesting: http://10.0.0.155:8085/Snh_VrfListReq?name=default-domain:k8s-default:__public__:__public__ 2025-10-31 00:01:32,107 - DEBUG - Requesting: http://10.0.0.240:8083/Snh_ShowRoutingInstanceReq?name=default-domain:k8s-default:__public__:__public__ 2025-10-31 00:01:32,112 - DEBUG - Requesting: http://10.0.0.240:8083/Snh_IFMapTableShowReq?table_name=virtual-network&search_string=virtual-network:default-domain:k8s-default:__public__ 2025-10-31 00:01:32,120 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN __public__ info 2025-10-31 00:01:32,121 - INFO - Deleting pod default:ctest-nginx-pod-40880924 2025-10-31 00:01:32,147 - DEBUG - Requesting: http://10.0.0.155:8085/Snh_VmListReq?uuid=275bd465-ca05-4fe3-b3e8-387c6e847e40 2025-10-31 00:01:32,155 - WARNING - Pod uuid 275bd465-ca05-4fe3-b3e8-387c6e847e40 is still seen in agent 10.0.0.155 VM list 2025-10-31 00:01:37,156 - DEBUG - Requesting: http://10.0.0.155:8085/Snh_VmListReq?uuid=275bd465-ca05-4fe3-b3e8-387c6e847e40 2025-10-31 00:01:37,162 - DEBUG - Pod 275bd465-ca05-4fe3-b3e8-387c6e847e40 is not in agent 10.0.0.155 VM list 2025-10-31 00:01:37,162 - DEBUG - Requesting: http://10.0.0.155:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:9be8023a-b5ec-11f0-9b35-fa163ecd12f8 2025-10-31 00:01:37,167 - DEBUG - VMI 9be8023a-b5ec-11f0-9b35-fa163ecd12f8 is removed from agent 10.0.0.155 2025-10-31 00:01:37,167 - INFO - Verified that pod ctest-nginx-pod-40880924 is removed in agent 2025-10-31 00:01:37,168 - INFO - Deleting pod default:ctest-nginx-pod-50356765 2025-10-31 00:01:37,176 - DEBUG - Requesting: http://10.0.0.155:8085/Snh_VmListReq?uuid=8f4d6573-5168-4a0b-870f-12a5c58cc94a 2025-10-31 00:01:37,183 - WARNING - Pod uuid 8f4d6573-5168-4a0b-870f-12a5c58cc94a is still seen in agent 10.0.0.155 VM list 2025-10-31 00:01:42,184 - DEBUG - Requesting: http://10.0.0.155:8085/Snh_VmListReq?uuid=8f4d6573-5168-4a0b-870f-12a5c58cc94a 2025-10-31 00:01:42,189 - DEBUG - Pod 8f4d6573-5168-4a0b-870f-12a5c58cc94a is not in agent 10.0.0.155 VM list 2025-10-31 00:01:42,190 - DEBUG - Requesting: http://10.0.0.155:8085/Snh_PageReq?x=begin:-1,end:-1,table:db.interface.0,uuid:9b828ef0-b5ec-11f0-9b35-fa163ecd12f8 2025-10-31 00:01:42,195 - DEBUG - VMI 9b828ef0-b5ec-11f0-9b35-fa163ecd12f8 is removed from agent 10.0.0.155 2025-10-31 00:01:42,195 - INFO - Verified that pod ctest-nginx-pod-50356765 is removed in agent 2025-10-31 00:01:42,195 - INFO - Deleting service : ctest-nginx-svc-88941458 2025-10-31 00:01:42,387 - DEBUG - Requesting: http://10.0.0.155:8085/Snh_AgentXmppConnectionStatusReq? 2025-10-31 00:01:42,393 - DEBUG - No XMPP flaps were noticed during the test