2026-02-09 20:23:56,358 - INFO - Using existing project ['default-domain', 'k8s-default'](bfa9c0dc-1c9d-4d40-8bb4-c298630aa513) 2026-02-09 20:23:56,375 - DEBUG - Namespace ctest-new-default-17995578 not present 2026-02-09 20:23:56,383 - INFO - Created namespace ctest-new-default-17995578 2026-02-09 20:23:59,388 - DEBUG - Namespace ctest-non-default-61144430 not present 2026-02-09 20:23:59,396 - INFO - Created namespace ctest-non-default-61144430 2026-02-09 20:24:02,425 - DEBUG - Pod ctest-pod-45352157 not present 2026-02-09 20:24:02,426 - INFO - Creating Pod ctest-pod-45352157 2026-02-09 20:24:02,466 - DEBUG - Pod : ctest-pod-45352157 UUID is 8532f155-0b8f-4eb7-ac0f-4a5282be8209 2026-02-09 20:24:02,468 - DEBUG - Pod ctest-pod-49685032 not present 2026-02-09 20:24:02,469 - INFO - Creating Pod ctest-pod-49685032 2026-02-09 20:24:02,505 - DEBUG - Pod : ctest-pod-49685032 UUID is 8365d485-97af-49c3-bb3f-c1c11c957e63 2026-02-09 20:24:02,509 - DEBUG - Pod ctest-pod-20850468 not present 2026-02-09 20:24:02,509 - INFO - Creating Pod ctest-pod-20850468 2026-02-09 20:24:02,565 - DEBUG - Pod : ctest-pod-20850468 UUID is 7241caef-66e2-4183-a38c-0729485125d2 2026-02-09 20:24:02,568 - DEBUG - Pod ctest-pod-18766348 not present 2026-02-09 20:24:02,568 - INFO - Creating Pod ctest-pod-18766348 2026-02-09 20:24:02,596 - DEBUG - Pod : ctest-pod-18766348 UUID is 0863a1b1-c230-49e6-8c25-972c7914af8b 2026-02-09 20:24:02,599 - DEBUG - Pod ctest-pod-24649128 not present 2026-02-09 20:24:02,600 - INFO - Creating Pod ctest-pod-24649128 2026-02-09 20:24:02,622 - DEBUG - Pod : ctest-pod-24649128 UUID is 4ca7bc22-5402-4998-94af-0243066326c2 2026-02-09 20:24:02,624 - DEBUG - Pod ctest-pod-08453752 not present 2026-02-09 20:24:02,624 - INFO - Creating Pod ctest-pod-08453752 2026-02-09 20:24:02,647 - DEBUG - Pod : ctest-pod-08453752 UUID is 189c2c3c-6223-4797-9a0a-0aaaa5539a2d 2026-02-09 20:24:02,650 - DEBUG - Pod ctest-pod-65789357 not present 2026-02-09 20:24:02,650 - INFO - Creating Pod ctest-pod-65789357 2026-02-09 20:24:02,670 - DEBUG - Pod : ctest-pod-65789357 UUID is 008c7f82-fdb4-4e2d-a210-167a932d5391 2026-02-09 20:24:02,672 - DEBUG - Pod ctest-pod-74807892 not present 2026-02-09 20:24:02,672 - INFO - Creating Pod ctest-pod-74807892 2026-02-09 20:24:02,699 - DEBUG - Pod : ctest-pod-74807892 UUID is 057b877d-4550-44df-bc2c-1076d3b16874 2026-02-09 20:24:02,855 - INFO - Project default-domain:k8s-ctest-new-default-17995578 for Namespace ctest-new-default-17995578 is seen in contrail-api 2026-02-09 20:24:02,855 - DEBUG - Project uuid in Contrail is 4f0a88b0-368b-4667-8b64-ac10e5de003e 2026-02-09 20:24:03,027 - DEBUG - Requesting: http://10.0.0.33:8108/Snh_NamespaceDatabaseList?namespace_uuid=029bcc67-8a18-4503-88fb-03de5ea0738d 2026-02-09 20:24:03,032 - INFO - Namespace ctest-new-default-17995578 with uuid 029bcc67-8a18-4503-88fb-03de5ea0738d found in kube manager 2026-02-09 20:24:03,032 - INFO - Namespace ctest-new-default-17995578 verification passed 2026-02-09 20:24:03,215 - INFO - Project default-domain:k8s-ctest-non-default-61144430 for Namespace ctest-non-default-61144430 is seen in contrail-api 2026-02-09 20:24:03,215 - DEBUG - Project uuid in Contrail is 26058ae5-df7b-4c42-98a2-4cf7e77ed420 2026-02-09 20:24:03,216 - DEBUG - Requesting: http://10.0.0.33:8108/Snh_NamespaceDatabaseList?namespace_uuid=ed2f6a33-3a9a-49ae-a63c-24d523dba5a3 2026-02-09 20:24:03,220 - INFO - Namespace ctest-non-default-61144430 with uuid ed2f6a33-3a9a-49ae-a63c-24d523dba5a3 found in kube manager 2026-02-09 20:24:03,220 - INFO - Namespace ctest-non-default-61144430 verification passed 2026-02-09 20:24:03,226 - DEBUG - Pod ctest-pod-45352157 not in running state.Currently in Pending 2026-02-09 20:24:08,234 - DEBUG - Pod ctest-pod-45352157 not in running state.Currently in Pending 2026-02-09 20:24:13,248 - INFO - Pod ctest-pod-45352157 is in running state.Got IP 10.47.255.251 2026-02-09 20:24:13,260 - DEBUG - Pod ctest-pod-45352157 has vmi 46f69724-05f5-11f1-8620-fa163e686c0e 2026-02-09 20:24:13,260 - INFO - Verified pod ctest-pod-45352157 in contrail-api 2026-02-09 20:24:13,278 - DEBUG - VMI 46f69724-05f5-11f1-8620-fa163e686c0e is active in agent 10.0.0.19 2026-02-09 20:24:13,278 - INFO - Verified Pod ctest-pod-45352157 in agent 10.0.0.19 2026-02-09 20:24:13,278 - DEBUG - Requesting: http://10.0.0.33:8108/Snh_VirtualMachineDatabaseList?x=8532f155-0b8f-4eb7-ac0f-4a5282be8209 2026-02-09 20:24:13,283 - INFO - Pod ctest-pod-45352157 with uuid 8532f155-0b8f-4eb7-ac0f-4a5282be8209 found in kube manager 2026-02-09 20:24:13,283 - INFO - Pod ctest-pod-45352157 verification passed 2026-02-09 20:24:13,290 - DEBUG - Pod ctest-pod-49685032 not in running state.Currently in Pending 2026-02-09 20:24:18,298 - INFO - Pod ctest-pod-49685032 is in running state.Got IP 10.47.255.250 2026-02-09 20:24:18,353 - DEBUG - Pod ctest-pod-49685032 has vmi 47608eae-05f5-11f1-8620-fa163e686c0e 2026-02-09 20:24:18,353 - INFO - Verified pod ctest-pod-49685032 in contrail-api 2026-02-09 20:24:18,370 - DEBUG - VMI 47608eae-05f5-11f1-8620-fa163e686c0e is active in agent 10.0.0.19 2026-02-09 20:24:18,370 - INFO - Verified Pod ctest-pod-49685032 in agent 10.0.0.19 2026-02-09 20:24:18,370 - DEBUG - Requesting: http://10.0.0.33:8108/Snh_VirtualMachineDatabaseList?x=8365d485-97af-49c3-bb3f-c1c11c957e63 2026-02-09 20:24:18,374 - INFO - Pod ctest-pod-49685032 with uuid 8365d485-97af-49c3-bb3f-c1c11c957e63 found in kube manager 2026-02-09 20:24:18,374 - INFO - Pod ctest-pod-49685032 verification passed 2026-02-09 20:24:18,380 - INFO - Pod ctest-pod-20850468 is in running state.Got IP 10.47.255.249 2026-02-09 20:24:18,397 - DEBUG - Pod ctest-pod-20850468 has vmi 480e828e-05f5-11f1-8620-fa163e686c0e 2026-02-09 20:24:18,397 - INFO - Verified pod ctest-pod-20850468 in contrail-api 2026-02-09 20:24:18,412 - DEBUG - VMI 480e828e-05f5-11f1-8620-fa163e686c0e is active in agent 10.0.0.19 2026-02-09 20:24:18,412 - INFO - Verified Pod ctest-pod-20850468 in agent 10.0.0.19 2026-02-09 20:24:18,412 - DEBUG - Requesting: http://10.0.0.33:8108/Snh_VirtualMachineDatabaseList?x=7241caef-66e2-4183-a38c-0729485125d2 2026-02-09 20:24:18,416 - INFO - Pod ctest-pod-20850468 with uuid 7241caef-66e2-4183-a38c-0729485125d2 found in kube manager 2026-02-09 20:24:18,416 - INFO - Pod ctest-pod-20850468 verification passed 2026-02-09 20:24:18,422 - INFO - Pod ctest-pod-18766348 is in running state.Got IP 10.47.255.248 2026-02-09 20:24:18,475 - DEBUG - Pod ctest-pod-18766348 has vmi 488bc208-05f5-11f1-8620-fa163e686c0e 2026-02-09 20:24:18,476 - INFO - Verified pod ctest-pod-18766348 in contrail-api 2026-02-09 20:24:18,492 - DEBUG - VMI 488bc208-05f5-11f1-8620-fa163e686c0e is active in agent 10.0.0.19 2026-02-09 20:24:18,492 - INFO - Verified Pod ctest-pod-18766348 in agent 10.0.0.19 2026-02-09 20:24:18,492 - DEBUG - Requesting: http://10.0.0.33:8108/Snh_VirtualMachineDatabaseList?x=0863a1b1-c230-49e6-8c25-972c7914af8b 2026-02-09 20:24:18,496 - INFO - Pod ctest-pod-18766348 with uuid 0863a1b1-c230-49e6-8c25-972c7914af8b found in kube manager 2026-02-09 20:24:18,496 - INFO - Pod ctest-pod-18766348 verification passed 2026-02-09 20:24:18,503 - INFO - Pod ctest-pod-24649128 is in running state.Got IP 10.47.255.247 2026-02-09 20:24:18,557 - DEBUG - Pod ctest-pod-24649128 has vmi 49257ba0-05f5-11f1-8620-fa163e686c0e 2026-02-09 20:24:18,557 - INFO - Verified pod ctest-pod-24649128 in contrail-api 2026-02-09 20:24:18,571 - DEBUG - VMI 49257ba0-05f5-11f1-8620-fa163e686c0e is active in agent 10.0.0.19 2026-02-09 20:24:18,571 - INFO - Verified Pod ctest-pod-24649128 in agent 10.0.0.19 2026-02-09 20:24:18,571 - DEBUG - Requesting: http://10.0.0.33:8108/Snh_VirtualMachineDatabaseList?x=4ca7bc22-5402-4998-94af-0243066326c2 2026-02-09 20:24:18,575 - INFO - Pod ctest-pod-24649128 with uuid 4ca7bc22-5402-4998-94af-0243066326c2 found in kube manager 2026-02-09 20:24:18,575 - INFO - Pod ctest-pod-24649128 verification passed 2026-02-09 20:24:18,581 - INFO - Pod ctest-pod-65789357 is in running state.Got IP 10.47.255.245 2026-02-09 20:24:18,591 - DEBUG - Pod ctest-pod-65789357 has vmi 4a16e0ee-05f5-11f1-8620-fa163e686c0e 2026-02-09 20:24:18,591 - INFO - Verified pod ctest-pod-65789357 in contrail-api 2026-02-09 20:24:18,606 - DEBUG - VMI 4a16e0ee-05f5-11f1-8620-fa163e686c0e is active in agent 10.0.0.19 2026-02-09 20:24:18,606 - INFO - Verified Pod ctest-pod-65789357 in agent 10.0.0.19 2026-02-09 20:24:18,606 - DEBUG - Requesting: http://10.0.0.33:8108/Snh_VirtualMachineDatabaseList?x=008c7f82-fdb4-4e2d-a210-167a932d5391 2026-02-09 20:24:18,609 - INFO - Pod ctest-pod-65789357 with uuid 008c7f82-fdb4-4e2d-a210-167a932d5391 found in kube manager 2026-02-09 20:24:18,610 - INFO - Pod ctest-pod-65789357 verification passed 2026-02-09 20:24:18,616 - DEBUG - Pod ctest-pod-08453752 not in running state.Currently in Pending 2026-02-09 20:24:23,622 - INFO - Pod ctest-pod-08453752 is in running state.Got IP 10.47.255.246 2026-02-09 20:24:23,677 - DEBUG - Pod ctest-pod-08453752 has vmi 49a3aa52-05f5-11f1-8620-fa163e686c0e 2026-02-09 20:24:23,677 - INFO - Verified pod ctest-pod-08453752 in contrail-api 2026-02-09 20:24:23,690 - DEBUG - VMI 49a3aa52-05f5-11f1-8620-fa163e686c0e is active in agent 10.0.0.19 2026-02-09 20:24:23,690 - INFO - Verified Pod ctest-pod-08453752 in agent 10.0.0.19 2026-02-09 20:24:23,690 - DEBUG - Requesting: http://10.0.0.33:8108/Snh_VirtualMachineDatabaseList?x=189c2c3c-6223-4797-9a0a-0aaaa5539a2d 2026-02-09 20:24:23,694 - INFO - Pod ctest-pod-08453752 with uuid 189c2c3c-6223-4797-9a0a-0aaaa5539a2d found in kube manager 2026-02-09 20:24:23,695 - INFO - Pod ctest-pod-08453752 verification passed 2026-02-09 20:24:23,701 - INFO - Pod ctest-pod-74807892 is in running state.Got IP 10.47.255.244 2026-02-09 20:24:23,715 - DEBUG - Pod ctest-pod-74807892 has vmi 4a8a9020-05f5-11f1-8620-fa163e686c0e 2026-02-09 20:24:23,715 - INFO - Verified pod ctest-pod-74807892 in contrail-api 2026-02-09 20:24:23,735 - DEBUG - VMI 4a8a9020-05f5-11f1-8620-fa163e686c0e is active in agent 10.0.0.19 2026-02-09 20:24:23,735 - INFO - Verified Pod ctest-pod-74807892 in agent 10.0.0.19 2026-02-09 20:24:23,735 - DEBUG - Requesting: http://10.0.0.33:8108/Snh_VirtualMachineDatabaseList?x=057b877d-4550-44df-bc2c-1076d3b16874 2026-02-09 20:24:23,739 - INFO - Pod ctest-pod-74807892 with uuid 057b877d-4550-44df-bc2c-1076d3b16874 found in kube manager 2026-02-09 20:24:23,739 - INFO - Pod ctest-pod-74807892 verification passed 2026-02-09 20:24:23,739 - INFO - ================================================================================ 2026-02-09 20:24:23,739 - INFO - STARTING TEST : test_ingress_rule_on_pod_with_service 2026-02-09 20:24:23,739 - INFO - TEST DESCRIPTION : Verify that ingress rules on a pod work fine when pods are accessed via a service Steps: 1. Create a service in namespace 1 over a pod which are endpoints of a service 2. Create a network policy to allow 1 pod of namespace 1 as ingress. 3. Verify that service respond correctly to the specified pod in the policy and service do not respond to anyone else. 4. Extend the policy and add the namespaceselector as namespace 2. 5. Verify that service become accessible to the pods of namespace 2. 2026-02-09 20:24:24,037 - DEBUG - Nothing to compare xmpp stats {'10.0.0.19': {'10.0.0.33': '0'}} with 2026-02-09 20:24:24,037 - INFO - Initial checks done. Running the testcase now 2026-02-09 20:24:24,037 - INFO - 2026-02-09 20:24:24,139 - DEBUG - [Pod ctest-pod-45352157] Cmd: echo ctest-pod-45352157 > /usr/share/nginx/html/index.html, Output: 2026-02-09 20:24:24,231 - DEBUG - [Pod ctest-pod-49685032] Cmd: echo ctest-pod-49685032 > /usr/share/nginx/html/index.html, Output: 2026-02-09 20:24:24,313 - DEBUG - [Pod ctest-pod-20850468] Cmd: echo ctest-pod-20850468 > /usr/share/nginx/html/index.html, Output: 2026-02-09 20:24:24,404 - DEBUG - [Pod ctest-pod-18766348] Cmd: echo ctest-pod-18766348 > /usr/share/nginx/html/index.html, Output: 2026-02-09 20:24:24,416 - DEBUG - Service ctest-nginx-svc-09032750 not present 2026-02-09 20:24:24,416 - INFO - Creating service ctest-nginx-svc-09032750 2026-02-09 20:24:24,474 - INFO - Created Service ctest-nginx-svc-09032750 2026-02-09 20:24:24,487 - WARNING - Service UUID 134cfe3d-3fcf-489c-8fb9-edc4940428b8 not yet found in contrail-api 2026-02-09 20:24:25,493 - INFO - Validated that Service ctest-nginx-svc-09032750 is seen in contrail-api 2026-02-09 20:24:25,494 - DEBUG - Requesting: http://10.0.0.33:8108/Snh_LoadbalancerDatabaseList?x=134cfe3d-3fcf-489c-8fb9-edc4940428b8 2026-02-09 20:24:25,503 - INFO - Service ctest-nginx-svc-09032750 with uuid 134cfe3d-3fcf-489c-8fb9-edc4940428b8 found in kube manager 2026-02-09 20:24:25,503 - INFO - Service ctest-nginx-svc-09032750 verification passed 2026-02-09 20:24:26,626 - DEBUG - [Pod ctest-pod-24649128] Cmd: wget http://10.110.69.98:80/ -O - -T 5 -t 1 , Output: Connecting to 10.110.69.98:80 (10.110.69.98:80) ctest-pod-49685032 writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-02-09 20:24:26,626 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd wget http://10.110.69.98:80/ -O - -T 5 -t 1 passed 2026-02-09 20:24:26,626 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd output: Connecting to 10.110.69.98:80 (10.110.69.98:80) ctest-pod-49685032 writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-02-09 20:24:26,742 - DEBUG - [Pod ctest-pod-24649128] Cmd: wget http://10.110.69.98:80/ -O - -T 5 -t 1 , Output: ctest-pod-49685032 Connecting to 10.110.69.98:80 (10.110.69.98:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-02-09 20:24:26,742 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd wget http://10.110.69.98:80/ -O - -T 5 -t 1 passed 2026-02-09 20:24:26,742 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd output: ctest-pod-49685032 Connecting to 10.110.69.98:80 (10.110.69.98:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-02-09 20:24:26,830 - DEBUG - [Pod ctest-pod-24649128] Cmd: wget http://10.110.69.98:80/ -O - -T 5 -t 1 , Output: ctest-pod-45352157 Connecting to 10.110.69.98:80 (10.110.69.98:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-02-09 20:24:26,830 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd wget http://10.110.69.98:80/ -O - -T 5 -t 1 passed 2026-02-09 20:24:26,831 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd output: ctest-pod-45352157 Connecting to 10.110.69.98:80 (10.110.69.98:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-02-09 20:24:26,831 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-45352157': 1, 'ctest-pod-49685032': 2} 2026-02-09 20:24:26,916 - DEBUG - [Pod ctest-pod-65789357] Cmd: wget http://10.110.69.98:80/ -O - -T 5 -t 1 , Output: ctest-pod-49685032 Connecting to 10.110.69.98:80 (10.110.69.98:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-02-09 20:24:26,916 - DEBUG - [Pod Pod ctest-pod-65789357] Cmd wget http://10.110.69.98:80/ -O - -T 5 -t 1 passed 2026-02-09 20:24:26,916 - DEBUG - [Pod Pod ctest-pod-65789357] Cmd output: ctest-pod-49685032 Connecting to 10.110.69.98:80 (10.110.69.98:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-02-09 20:24:27,001 - DEBUG - [Pod ctest-pod-65789357] Cmd: wget http://10.110.69.98:80/ -O - -T 5 -t 1 , Output: Connecting to 10.110.69.98:80 (10.110.69.98:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-45352157 2026-02-09 20:24:27,001 - DEBUG - [Pod Pod ctest-pod-65789357] Cmd wget http://10.110.69.98:80/ -O - -T 5 -t 1 passed 2026-02-09 20:24:27,001 - DEBUG - [Pod Pod ctest-pod-65789357] Cmd output: Connecting to 10.110.69.98:80 (10.110.69.98:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-45352157 2026-02-09 20:24:27,001 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-45352157': 1, 'ctest-pod-49685032': 1} 2026-02-09 20:24:27,005 - DEBUG - Network policy ingress-on-service not present 2026-02-09 20:24:27,005 - INFO - Creating Network Policy ingress-on-service 2026-02-09 20:24:27,021 - INFO - Network policy found in k8s 2026-02-09 20:24:27,021 - DEBUG - Requesting: http://10.0.0.33:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=f8bc7fb0-167d-419c-94f4-8450bf648f45 2026-02-09 20:24:27,030 - INFO - Network Policy ingress-on-service with uuid f8bc7fb0-167d-419c-94f4-8450bf648f45 found in kube manager 2026-02-09 20:24:27,082 - INFO - Network policy with name default-policy-management:k8s-allowall found in agent 2026-02-09 20:24:27,082 - INFO - Network policy with name default-policy-management:k8s-allowall associated with default ks8 2026-02-09 20:24:27,082 - DEBUG - Requesting: http://10.0.0.33:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=f8bc7fb0-167d-419c-94f4-8450bf648f45 2026-02-09 20:24:27,124 - WARNING - Network policy with name ingress-on-service not found in agent 2026-02-09 20:24:29,125 - DEBUG - Requesting: http://10.0.0.33:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=f8bc7fb0-167d-419c-94f4-8450bf648f45 2026-02-09 20:24:29,151 - INFO - Network policy with name ingress-on-service found in agent 2026-02-09 20:24:29,151 - INFO - Network Policy ingress-on-service verification passed 2026-02-09 20:24:34,242 - DEBUG - [Pod ctest-pod-24649128] Cmd: wget http://10.110.69.98:80/ -O - -T 5 -t 1 , Output: Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:24:34,242 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd wget http://10.110.69.98:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:24:34,242 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd output: Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:24:39,334 - DEBUG - [Pod ctest-pod-24649128] Cmd: wget http://10.110.69.98:80/ -O - -T 5 -t 1 , Output: Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:24:39,334 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd wget http://10.110.69.98:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:24:39,334 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd output: Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:24:44,405 - DEBUG - [Pod ctest-pod-24649128] Cmd: wget http://10.110.69.98:80/ -O - -T 5 -t 1 , Output: Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:24:44,405 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd wget http://10.110.69.98:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:24:44,405 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd output: Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:24:49,468 - DEBUG - [Pod ctest-pod-24649128] Cmd: wget http://10.110.69.98:80/ -O - -T 5 -t 1 , Output: Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:24:49,468 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd wget http://10.110.69.98:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:24:49,468 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd output: Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:24:54,543 - DEBUG - [Pod ctest-pod-24649128] Cmd: wget http://10.110.69.98:80/ -O - -T 5 -t 1 , Output: Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:24:54,543 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd wget http://10.110.69.98:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:24:54,543 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd output: Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:24:59,628 - DEBUG - [Pod ctest-pod-24649128] Cmd: wget http://10.110.69.98:80/ -O - -T 5 -t 1 , Output: Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:24:59,628 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd wget http://10.110.69.98:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:24:59,628 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd output: Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:25:04,713 - DEBUG - [Pod ctest-pod-24649128] Cmd: wget http://10.110.69.98:80/ -O - -T 5 -t 1 , Output: Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:25:04,714 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd wget http://10.110.69.98:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:25:04,714 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd output: Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:25:09,788 - DEBUG - [Pod ctest-pod-24649128] Cmd: wget http://10.110.69.98:80/ -O - -T 5 -t 1 , Output: Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:25:09,788 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd wget http://10.110.69.98:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:25:09,788 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd output: Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:25:14,891 - DEBUG - [Pod ctest-pod-24649128] Cmd: wget http://10.110.69.98:80/ -O - -T 5 -t 1 , Output: Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:25:14,891 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd wget http://10.110.69.98:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:25:14,891 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd output: Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:25:19,956 - DEBUG - [Pod ctest-pod-24649128] Cmd: wget http://10.110.69.98:80/ -O - -T 5 -t 1 , Output: Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:25:19,956 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd wget http://10.110.69.98:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:25:19,956 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd output: Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:25:19,956 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-45352157': 0, 'ctest-pod-49685032': 0} 2026-02-09 20:25:20,041 - DEBUG - [Pod ctest-pod-08453752] Cmd: wget http://10.110.69.98:80/ -O - -T 5 -t 1 , Output: ctest-pod-49685032 Connecting to 10.110.69.98:80 (10.110.69.98:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-02-09 20:25:20,042 - DEBUG - [Pod Pod ctest-pod-08453752] Cmd wget http://10.110.69.98:80/ -O - -T 5 -t 1 passed 2026-02-09 20:25:20,042 - DEBUG - [Pod Pod ctest-pod-08453752] Cmd output: ctest-pod-49685032 Connecting to 10.110.69.98:80 (10.110.69.98:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-02-09 20:25:20,122 - DEBUG - [Pod ctest-pod-08453752] Cmd: wget http://10.110.69.98:80/ -O - -T 5 -t 1 , Output: Connecting to 10.110.69.98:80 (10.110.69.98:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-49685032 2026-02-09 20:25:20,123 - DEBUG - [Pod Pod ctest-pod-08453752] Cmd wget http://10.110.69.98:80/ -O - -T 5 -t 1 passed 2026-02-09 20:25:20,123 - DEBUG - [Pod Pod ctest-pod-08453752] Cmd output: Connecting to 10.110.69.98:80 (10.110.69.98:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-49685032 2026-02-09 20:25:20,209 - DEBUG - [Pod ctest-pod-08453752] Cmd: wget http://10.110.69.98:80/ -O - -T 5 -t 1 , Output: ctest-pod-45352157 Connecting to 10.110.69.98:80 (10.110.69.98:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-02-09 20:25:20,209 - DEBUG - [Pod Pod ctest-pod-08453752] Cmd wget http://10.110.69.98:80/ -O - -T 5 -t 1 passed 2026-02-09 20:25:20,209 - DEBUG - [Pod Pod ctest-pod-08453752] Cmd output: ctest-pod-45352157 Connecting to 10.110.69.98:80 (10.110.69.98:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-02-09 20:25:20,209 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-45352157': 1, 'ctest-pod-49685032': 2} 2026-02-09 20:25:25,276 - DEBUG - [Pod ctest-pod-65789357] Cmd: wget http://10.110.69.98:80/ -O - -T 5 -t 1 , Output: Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:25:25,276 - DEBUG - [Pod Pod ctest-pod-65789357] Cmd wget http://10.110.69.98:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:25:25,276 - DEBUG - [Pod Pod ctest-pod-65789357] Cmd output: Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:25:30,370 - DEBUG - [Pod ctest-pod-65789357] Cmd: wget http://10.110.69.98:80/ -O - -T 5 -t 1 , Output: Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:25:30,370 - DEBUG - [Pod Pod ctest-pod-65789357] Cmd wget http://10.110.69.98:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:25:30,370 - DEBUG - [Pod Pod ctest-pod-65789357] Cmd output: Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:25:35,450 - DEBUG - [Pod ctest-pod-65789357] Cmd: wget http://10.110.69.98:80/ -O - -T 5 -t 1 , Output: Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:25:35,450 - DEBUG - [Pod Pod ctest-pod-65789357] Cmd wget http://10.110.69.98:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:25:35,450 - DEBUG - [Pod Pod ctest-pod-65789357] Cmd output: Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:25:40,523 - DEBUG - [Pod ctest-pod-65789357] Cmd: wget http://10.110.69.98:80/ -O - -T 5 -t 1 , Output: Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:25:40,523 - DEBUG - [Pod Pod ctest-pod-65789357] Cmd wget http://10.110.69.98:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:25:40,523 - DEBUG - [Pod Pod ctest-pod-65789357] Cmd output: Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:25:45,623 - DEBUG - [Pod ctest-pod-65789357] Cmd: wget http://10.110.69.98:80/ -O - -T 5 -t 1 , Output: Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:25:45,623 - DEBUG - [Pod Pod ctest-pod-65789357] Cmd wget http://10.110.69.98:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:25:45,623 - DEBUG - [Pod Pod ctest-pod-65789357] Cmd output: Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:25:50,685 - DEBUG - [Pod ctest-pod-65789357] Cmd: wget http://10.110.69.98:80/ -O - -T 5 -t 1 , Output: Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:25:50,685 - DEBUG - [Pod Pod ctest-pod-65789357] Cmd wget http://10.110.69.98:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:25:50,685 - DEBUG - [Pod Pod ctest-pod-65789357] Cmd output: Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:25:55,786 - DEBUG - [Pod ctest-pod-65789357] Cmd: wget http://10.110.69.98:80/ -O - -T 5 -t 1 , Output: Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:25:55,786 - DEBUG - [Pod Pod ctest-pod-65789357] Cmd wget http://10.110.69.98:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:25:55,786 - DEBUG - [Pod Pod ctest-pod-65789357] Cmd output: Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:26:00,876 - DEBUG - [Pod ctest-pod-65789357] Cmd: wget http://10.110.69.98:80/ -O - -T 5 -t 1 , Output: Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:26:00,877 - DEBUG - [Pod Pod ctest-pod-65789357] Cmd wget http://10.110.69.98:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:26:00,877 - DEBUG - [Pod Pod ctest-pod-65789357] Cmd output: Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:26:05,977 - DEBUG - [Pod ctest-pod-65789357] Cmd: wget http://10.110.69.98:80/ -O - -T 5 -t 1 , Output: Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:26:05,977 - DEBUG - [Pod Pod ctest-pod-65789357] Cmd wget http://10.110.69.98:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:26:05,977 - DEBUG - [Pod Pod ctest-pod-65789357] Cmd output: Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:26:11,058 - DEBUG - [Pod ctest-pod-65789357] Cmd: wget http://10.110.69.98:80/ -O - -T 5 -t 1 , Output: Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:26:11,058 - DEBUG - [Pod Pod ctest-pod-65789357] Cmd wget http://10.110.69.98:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:26:11,059 - DEBUG - [Pod Pod ctest-pod-65789357] Cmd output: Connecting to 10.110.69.98:80 (10.110.69.98:80) wget: download timed out 2026-02-09 20:26:11,059 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-45352157': 0, 'ctest-pod-49685032': 0} 2026-02-09 20:26:13,129 - DEBUG - [Pod ctest-pod-08453752] Cmd: ping -c 3 10.47.255.247, Output: PING 10.47.255.247 (10.47.255.247): 56 data bytes 64 bytes from 10.47.255.247: seq=0 ttl=63 time=1.414 ms 64 bytes from 10.47.255.247: seq=1 ttl=63 time=0.089 ms 64 bytes from 10.47.255.247: seq=2 ttl=63 time=0.218 ms --- 10.47.255.247 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.089/0.573/1.414 ms 2026-02-09 20:26:13,129 - INFO - Ping check to IP 10.47.255.247 from pod ctest-pod-08453752 with expectation True passed 2026-02-09 20:26:15,207 - DEBUG - [Pod ctest-pod-24649128] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=0.681 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.092 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.104 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.092/0.292/0.681 ms 2026-02-09 20:26:15,207 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-24649128 with expectation True passed 2026-02-09 20:26:17,290 - DEBUG - [Pod ctest-pod-65789357] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=0.677 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.123 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.162 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.123/0.320/0.677 ms 2026-02-09 20:26:17,290 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-65789357 with expectation True passed 2026-02-09 20:26:17,398 - DEBUG - [Pod ctest-pod-08453752] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-09 20:26:17,399 - DEBUG - [Pod Pod ctest-pod-08453752] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2026-02-09 20:26:17,399 - DEBUG - [Pod Pod ctest-pod-08453752] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-09 20:26:17,399 - INFO - wget check of of http://10.47.255.251 from ctest-pod-08453752 passed 2026-02-09 20:26:17,481 - DEBUG - [Pod ctest-pod-08453752] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-09 20:26:17,481 - DEBUG - [Pod Pod ctest-pod-08453752] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2026-02-09 20:26:17,481 - DEBUG - [Pod Pod ctest-pod-08453752] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-09 20:26:17,481 - INFO - wget check of of http://10.47.255.250 from ctest-pod-08453752 passed 2026-02-09 20:26:17,555 - DEBUG - [Pod ctest-pod-24649128] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-09 20:26:17,555 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2026-02-09 20:26:17,555 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-09 20:26:17,555 - WARNING - wget check of of http://10.47.255.250 from ctest-pod-24649128 failed 2026-02-09 20:26:19,654 - DEBUG - [Pod ctest-pod-24649128] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-09 20:26:19,654 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2026-02-09 20:26:19,654 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-09 20:26:19,654 - WARNING - wget check of of http://10.47.255.250 from ctest-pod-24649128 failed 2026-02-09 20:26:21,736 - DEBUG - [Pod ctest-pod-24649128] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-09 20:26:21,736 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2026-02-09 20:26:21,736 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-09 20:26:21,736 - WARNING - wget check of of http://10.47.255.250 from ctest-pod-24649128 failed 2026-02-09 20:26:23,823 - DEBUG - [Pod ctest-pod-24649128] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-09 20:26:23,824 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2026-02-09 20:26:23,824 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-09 20:26:23,824 - WARNING - wget check of of http://10.47.255.250 from ctest-pod-24649128 failed 2026-02-09 20:26:25,935 - DEBUG - [Pod ctest-pod-24649128] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-09 20:26:25,935 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2026-02-09 20:26:25,935 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-09 20:26:25,935 - WARNING - wget check of of http://10.47.255.250 from ctest-pod-24649128 failed 2026-02-09 20:26:28,008 - DEBUG - [Pod ctest-pod-24649128] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-09 20:26:28,008 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2026-02-09 20:26:28,008 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-09 20:26:28,008 - WARNING - wget check of of http://10.47.255.250 from ctest-pod-24649128 failed 2026-02-09 20:26:30,091 - DEBUG - [Pod ctest-pod-24649128] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-09 20:26:30,091 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2026-02-09 20:26:30,091 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-09 20:26:30,091 - WARNING - wget check of of http://10.47.255.250 from ctest-pod-24649128 failed 2026-02-09 20:26:32,176 - DEBUG - [Pod ctest-pod-24649128] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-09 20:26:32,177 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2026-02-09 20:26:32,177 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-09 20:26:32,177 - WARNING - wget check of of http://10.47.255.250 from ctest-pod-24649128 failed 2026-02-09 20:26:34,286 - DEBUG - [Pod ctest-pod-24649128] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-09 20:26:34,286 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2026-02-09 20:26:34,286 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-09 20:26:34,286 - WARNING - wget check of of http://10.47.255.250 from ctest-pod-24649128 failed 2026-02-09 20:26:36,371 - DEBUG - [Pod ctest-pod-24649128] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-09 20:26:36,371 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2026-02-09 20:26:36,372 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-09 20:26:36,372 - WARNING - wget check of of http://10.47.255.250 from ctest-pod-24649128 failed 2026-02-09 20:26:38,488 - DEBUG - [Pod ctest-pod-24649128] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-09 20:26:38,488 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2026-02-09 20:26:38,488 - DEBUG - [Pod Pod ctest-pod-24649128] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-09 20:26:38,488 - WARNING - wget check of of http://10.47.255.250 from ctest-pod-24649128 failed 2026-02-09 20:26:38,496 - INFO - Deleting Network Policy : ingress-on-service 2026-02-09 20:26:38,497 - INFO - Deleting service : ctest-nginx-svc-09032750 2026-02-09 20:26:38,767 - ERROR - AssertionError Python 3.9.21: /usr/bin/python3 Mon Feb 9 20:26:38 2026 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={}) 78 log.info('Initial checks done. Running the testcase now') 79 log.info('') 80 result = function(self, *args, **kwargs) 81 if self.inputs.upgrade: 82 pid = os.getpid() result = None function = self = args = () kwargs = {} /contrail-test/scripts/k8s_scripts/test_policy.py in test_ingress_rule_on_pod_with_service(self=) 2859 assert self.validate_wget(self.client2_pod_ns1, url1) 2860 assert self.validate_wget(self.client2_pod_ns1, url2) 2861 assert self.validate_wget(self.client1_pod_ns1, url2, expectation=False) 2862 assert self.validate_wget(self.client2_pod_ns2, url1, expectation=False) 2863 # Updating the policy and adding the namespace selector self = self.validate_wget = .deco_retry....ice[ci_k8s_sanity,k8s_sanity] id=0x7f1eb2e06af0>> self.client1_pod_ns1 = url2 = 'http://10.47.255.250' expectation undefined AssertionError: __cause__ = None __class__ = __context__ = None __delattr__ = __dict__ = {} __dir__ = __doc__ = 'Assertion failed.' __eq__ = __format__ = __ge__ = __getattribute__ = __gt__ = __hash__ = __init__ = __init_subclass__ = __le__ = __lt__ = __ne__ = __new__ = __reduce__ = __reduce_ex__ = __repr__ = __setattr__ = __setstate__ = __sizeof__ = __str__ = __subclasshook__ = __suppress_context__ = False __traceback__ = args = () with_traceback = The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper result = function(self, *args, **kwargs) File "/contrail-test/scripts/k8s_scripts/test_policy.py", line 2861, in test_ingress_rule_on_pod_with_service assert self.validate_wget(self.client1_pod_ns1, url2, expectation=False) AssertionError 2026-02-09 20:26:38,781 - DEBUG - No XMPP flaps were noticed during the test 2026-02-09 20:26:38,781 - INFO -
2026-02-09 20:26:38,781 - INFO - END TEST : test_ingress_rule_on_pod_with_service : FAILED[0:02:15]
2026-02-09 20:26:38,781 - INFO - -------------------------------------------------------------------------------- 2026-02-09 20:26:38,786 - INFO - Deleting pod ctest-new-default-17995578:ctest-pod-45352157 2026-02-09 20:26:38,786 - INFO - Deleting pod ctest-new-default-17995578:ctest-pod-49685032 2026-02-09 20:26:38,787 - INFO - Deleting pod ctest-non-default-61144430:ctest-pod-20850468 2026-02-09 20:26:38,787 - INFO - Deleting pod ctest-non-default-61144430:ctest-pod-18766348 2026-02-09 20:26:38,788 - INFO - Deleting pod ctest-new-default-17995578:ctest-pod-24649128 2026-02-09 20:26:38,788 - INFO - Deleting pod ctest-new-default-17995578:ctest-pod-08453752 2026-02-09 20:26:38,788 - INFO - Deleting pod ctest-non-default-61144430:ctest-pod-65789357 2026-02-09 20:26:38,789 - INFO - Deleting pod ctest-non-default-61144430:ctest-pod-74807892 2026-02-09 20:26:38,898 - WARNING - Pod uuid 189c2c3c-6223-4797-9a0a-0aaaa5539a2d is still seen in agent 10.0.0.19 VM list 2026-02-09 20:26:38,901 - WARNING - Pod uuid 0863a1b1-c230-49e6-8c25-972c7914af8b is still seen in agent 10.0.0.19 VM list 2026-02-09 20:26:38,954 - WARNING - Pod uuid 8532f155-0b8f-4eb7-ac0f-4a5282be8209 is still seen in agent 10.0.0.19 VM list 2026-02-09 20:26:38,955 - WARNING - Pod uuid 4ca7bc22-5402-4998-94af-0243066326c2 is still seen in agent 10.0.0.19 VM list 2026-02-09 20:26:38,955 - WARNING - Pod uuid 008c7f82-fdb4-4e2d-a210-167a932d5391 is still seen in agent 10.0.0.19 VM list 2026-02-09 20:26:38,955 - WARNING - Pod uuid 8365d485-97af-49c3-bb3f-c1c11c957e63 is still seen in agent 10.0.0.19 VM list 2026-02-09 20:26:38,956 - WARNING - Pod uuid 7241caef-66e2-4183-a38c-0729485125d2 is still seen in agent 10.0.0.19 VM list 2026-02-09 20:26:38,956 - WARNING - Pod uuid 057b877d-4550-44df-bc2c-1076d3b16874 is still seen in agent 10.0.0.19 VM list 2026-02-09 20:26:43,911 - WARNING - Pod uuid 189c2c3c-6223-4797-9a0a-0aaaa5539a2d is still seen in agent 10.0.0.19 VM list 2026-02-09 20:26:43,912 - DEBUG - Pod 0863a1b1-c230-49e6-8c25-972c7914af8b is not in agent 10.0.0.19 VM list 2026-02-09 20:26:43,921 - DEBUG - VMI 488bc208-05f5-11f1-8620-fa163e686c0e is removed from agent 10.0.0.19 2026-02-09 20:26:43,922 - INFO - Verified that pod ctest-pod-18766348 is removed in agent 2026-02-09 20:26:43,983 - WARNING - Pod uuid 008c7f82-fdb4-4e2d-a210-167a932d5391 is still seen in agent 10.0.0.19 VM list 2026-02-09 20:26:43,983 - DEBUG - Pod 8532f155-0b8f-4eb7-ac0f-4a5282be8209 is not in agent 10.0.0.19 VM list 2026-02-09 20:26:43,985 - WARNING - Pod uuid 4ca7bc22-5402-4998-94af-0243066326c2 is still seen in agent 10.0.0.19 VM list 2026-02-09 20:26:43,987 - WARNING - Pod uuid 8365d485-97af-49c3-bb3f-c1c11c957e63 is still seen in agent 10.0.0.19 VM list 2026-02-09 20:26:43,987 - DEBUG - Pod 7241caef-66e2-4183-a38c-0729485125d2 is not in agent 10.0.0.19 VM list 2026-02-09 20:26:43,988 - WARNING - Pod uuid 057b877d-4550-44df-bc2c-1076d3b16874 is still seen in agent 10.0.0.19 VM list 2026-02-09 20:26:44,001 - DEBUG - VMI 46f69724-05f5-11f1-8620-fa163e686c0e is removed from agent 10.0.0.19 2026-02-09 20:26:44,001 - INFO - Verified that pod ctest-pod-45352157 is removed in agent 2026-02-09 20:26:44,002 - DEBUG - VMI 480e828e-05f5-11f1-8620-fa163e686c0e is removed from agent 10.0.0.19 2026-02-09 20:26:44,002 - INFO - Verified that pod ctest-pod-20850468 is removed in agent 2026-02-09 20:26:48,919 - WARNING - Pod uuid 189c2c3c-6223-4797-9a0a-0aaaa5539a2d is still seen in agent 10.0.0.19 VM list 2026-02-09 20:26:49,021 - WARNING - Pod uuid 4ca7bc22-5402-4998-94af-0243066326c2 is still seen in agent 10.0.0.19 VM list 2026-02-09 20:26:49,022 - WARNING - Pod uuid 008c7f82-fdb4-4e2d-a210-167a932d5391 is still seen in agent 10.0.0.19 VM list 2026-02-09 20:26:49,023 - WARNING - Pod uuid 8365d485-97af-49c3-bb3f-c1c11c957e63 is still seen in agent 10.0.0.19 VM list 2026-02-09 20:26:49,024 - WARNING - Pod uuid 057b877d-4550-44df-bc2c-1076d3b16874 is still seen in agent 10.0.0.19 VM list 2026-02-09 20:26:53,926 - WARNING - Pod uuid 189c2c3c-6223-4797-9a0a-0aaaa5539a2d is still seen in agent 10.0.0.19 VM list 2026-02-09 20:26:54,049 - WARNING - Pod uuid 8365d485-97af-49c3-bb3f-c1c11c957e63 is still seen in agent 10.0.0.19 VM list 2026-02-09 20:26:54,049 - WARNING - Pod uuid 008c7f82-fdb4-4e2d-a210-167a932d5391 is still seen in agent 10.0.0.19 VM list 2026-02-09 20:26:54,049 - WARNING - Pod uuid 057b877d-4550-44df-bc2c-1076d3b16874 is still seen in agent 10.0.0.19 VM list 2026-02-09 20:26:54,050 - WARNING - Pod uuid 4ca7bc22-5402-4998-94af-0243066326c2 is still seen in agent 10.0.0.19 VM list 2026-02-09 20:26:58,940 - WARNING - Pod uuid 189c2c3c-6223-4797-9a0a-0aaaa5539a2d is still seen in agent 10.0.0.19 VM list 2026-02-09 20:26:59,068 - WARNING - Pod uuid 008c7f82-fdb4-4e2d-a210-167a932d5391 is still seen in agent 10.0.0.19 VM list 2026-02-09 20:26:59,069 - WARNING - Pod uuid 8365d485-97af-49c3-bb3f-c1c11c957e63 is still seen in agent 10.0.0.19 VM list 2026-02-09 20:26:59,070 - WARNING - Pod uuid 057b877d-4550-44df-bc2c-1076d3b16874 is still seen in agent 10.0.0.19 VM list 2026-02-09 20:26:59,070 - WARNING - Pod uuid 4ca7bc22-5402-4998-94af-0243066326c2 is still seen in agent 10.0.0.19 VM list 2026-02-09 20:27:03,955 - WARNING - Pod uuid 189c2c3c-6223-4797-9a0a-0aaaa5539a2d is still seen in agent 10.0.0.19 VM list 2026-02-09 20:27:04,111 - WARNING - Pod uuid 8365d485-97af-49c3-bb3f-c1c11c957e63 is still seen in agent 10.0.0.19 VM list 2026-02-09 20:27:04,112 - WARNING - Pod uuid 008c7f82-fdb4-4e2d-a210-167a932d5391 is still seen in agent 10.0.0.19 VM list 2026-02-09 20:27:04,113 - WARNING - Pod uuid 057b877d-4550-44df-bc2c-1076d3b16874 is still seen in agent 10.0.0.19 VM list 2026-02-09 20:27:04,113 - WARNING - Pod uuid 4ca7bc22-5402-4998-94af-0243066326c2 is still seen in agent 10.0.0.19 VM list 2026-02-09 20:27:08,975 - WARNING - Pod uuid 189c2c3c-6223-4797-9a0a-0aaaa5539a2d is still seen in agent 10.0.0.19 VM list 2026-02-09 20:27:09,130 - WARNING - Pod uuid 8365d485-97af-49c3-bb3f-c1c11c957e63 is still seen in agent 10.0.0.19 VM list 2026-02-09 20:27:09,131 - WARNING - Pod uuid 057b877d-4550-44df-bc2c-1076d3b16874 is still seen in agent 10.0.0.19 VM list 2026-02-09 20:27:09,132 - WARNING - Pod uuid 4ca7bc22-5402-4998-94af-0243066326c2 is still seen in agent 10.0.0.19 VM list 2026-02-09 20:27:09,133 - WARNING - Pod uuid 008c7f82-fdb4-4e2d-a210-167a932d5391 is still seen in agent 10.0.0.19 VM list 2026-02-09 20:27:13,982 - DEBUG - Pod 189c2c3c-6223-4797-9a0a-0aaaa5539a2d is not in agent 10.0.0.19 VM list 2026-02-09 20:27:13,989 - DEBUG - VMI 49a3aa52-05f5-11f1-8620-fa163e686c0e is removed from agent 10.0.0.19 2026-02-09 20:27:13,989 - INFO - Verified that pod ctest-pod-08453752 is removed in agent 2026-02-09 20:27:14,151 - DEBUG - Pod 8365d485-97af-49c3-bb3f-c1c11c957e63 is not in agent 10.0.0.19 VM list 2026-02-09 20:27:14,155 - DEBUG - Pod 057b877d-4550-44df-bc2c-1076d3b16874 is not in agent 10.0.0.19 VM list 2026-02-09 20:27:14,170 - DEBUG - VMI 47608eae-05f5-11f1-8620-fa163e686c0e is removed from agent 10.0.0.19 2026-02-09 20:27:14,170 - INFO - Verified that pod ctest-pod-49685032 is removed in agent 2026-02-09 20:27:14,170 - DEBUG - VMI 4a8a9020-05f5-11f1-8620-fa163e686c0e is removed from agent 10.0.0.19 2026-02-09 20:27:14,170 - INFO - Verified that pod ctest-pod-74807892 is removed in agent 2026-02-09 20:27:14,170 - DEBUG - Pod 008c7f82-fdb4-4e2d-a210-167a932d5391 is not in agent 10.0.0.19 VM list 2026-02-09 20:27:14,172 - DEBUG - Pod 4ca7bc22-5402-4998-94af-0243066326c2 is not in agent 10.0.0.19 VM list 2026-02-09 20:27:14,186 - DEBUG - VMI 49257ba0-05f5-11f1-8620-fa163e686c0e is removed from agent 10.0.0.19 2026-02-09 20:27:14,186 - INFO - Verified that pod ctest-pod-24649128 is removed in agent 2026-02-09 20:27:14,189 - DEBUG - VMI 4a16e0ee-05f5-11f1-8620-fa163e686c0e is removed from agent 10.0.0.19 2026-02-09 20:27:14,189 - INFO - Verified that pod ctest-pod-65789357 is removed in agent 2026-02-09 20:27:14,189 - INFO - Deleting namespace ctest-non-default-61144430 2026-02-09 20:27:14,232 - DEBUG - Namespace ctest-non-default-61144430 still in k8s 2026-02-09 20:27:16,237 - DEBUG - Namespace ctest-non-default-61144430 still in k8s 2026-02-09 20:27:18,243 - DEBUG - Namespace ctest-non-default-61144430 still in k8s 2026-02-09 20:27:20,256 - DEBUG - Namespace ctest-non-default-61144430 is not in k8s 2026-02-09 20:27:20,256 - DEBUG - No need to check project deletion in contrail-api 2026-02-09 20:27:20,256 - INFO - Deleting namespace ctest-new-default-17995578 2026-02-09 20:27:20,267 - DEBUG - Namespace ctest-new-default-17995578 still in k8s 2026-02-09 20:27:22,274 - DEBUG - Namespace ctest-new-default-17995578 still in k8s 2026-02-09 20:27:24,279 - DEBUG - Namespace ctest-new-default-17995578 still in k8s 2026-02-09 20:27:26,284 - DEBUG - Namespace ctest-new-default-17995578 is not in k8s 2026-02-09 20:27:26,284 - DEBUG - No need to check project deletion in contrail-api