2026-01-29 04:36:23,459 - INFO - Using existing project ['default-domain', 'k8s-default'](0fdfa9f3-aeeb-43da-badc-04c8c92e6f7f) 2026-01-29 04:36:23,482 - DEBUG - Namespace ctest-new-default-73034959 not present 2026-01-29 04:36:23,494 - INFO - Created namespace ctest-new-default-73034959 2026-01-29 04:36:26,497 - DEBUG - Namespace ctest-non-default-71721767 not present 2026-01-29 04:36:26,504 - INFO - Created namespace ctest-non-default-71721767 2026-01-29 04:36:29,527 - DEBUG - Pod ctest-pod-77324757 not present 2026-01-29 04:36:29,528 - INFO - Creating Pod ctest-pod-77324757 2026-01-29 04:36:29,574 - DEBUG - Pod : ctest-pod-77324757 UUID is fec9c138-3410-4227-8b1c-39ffce8d51de 2026-01-29 04:36:29,579 - DEBUG - Pod ctest-pod-90385737 not present 2026-01-29 04:36:29,580 - INFO - Creating Pod ctest-pod-90385737 2026-01-29 04:36:29,602 - DEBUG - Pod : ctest-pod-90385737 UUID is c60f5652-347d-4061-95d3-b5b962223cfb 2026-01-29 04:36:29,604 - DEBUG - Pod ctest-pod-69918613 not present 2026-01-29 04:36:29,604 - INFO - Creating Pod ctest-pod-69918613 2026-01-29 04:36:29,648 - DEBUG - Pod : ctest-pod-69918613 UUID is 291d680d-c973-41af-8fdf-8e17db25ca12 2026-01-29 04:36:29,650 - DEBUG - Pod ctest-pod-43220972 not present 2026-01-29 04:36:29,650 - INFO - Creating Pod ctest-pod-43220972 2026-01-29 04:36:29,676 - DEBUG - Pod : ctest-pod-43220972 UUID is dc8b559a-5d2b-4b24-b878-5323632bcf94 2026-01-29 04:36:29,678 - DEBUG - Pod ctest-pod-42424437 not present 2026-01-29 04:36:29,679 - INFO - Creating Pod ctest-pod-42424437 2026-01-29 04:36:29,707 - DEBUG - Pod : ctest-pod-42424437 UUID is 10d4872a-e2a5-46a2-b6c8-add9ef531aea 2026-01-29 04:36:29,709 - DEBUG - Pod ctest-pod-17427988 not present 2026-01-29 04:36:29,710 - INFO - Creating Pod ctest-pod-17427988 2026-01-29 04:36:29,741 - DEBUG - Pod : ctest-pod-17427988 UUID is fa7ec9d4-325b-408b-a34e-253f4e35fd95 2026-01-29 04:36:29,744 - DEBUG - Pod ctest-pod-31022830 not present 2026-01-29 04:36:29,745 - INFO - Creating Pod ctest-pod-31022830 2026-01-29 04:36:29,763 - DEBUG - Pod : ctest-pod-31022830 UUID is 37305eaf-6339-4ed2-9a07-512c1e92dd3e 2026-01-29 04:36:29,765 - DEBUG - Pod ctest-pod-24981972 not present 2026-01-29 04:36:29,765 - INFO - Creating Pod ctest-pod-24981972 2026-01-29 04:36:29,781 - DEBUG - Pod : ctest-pod-24981972 UUID is 6cd24ec4-7caa-4781-97ae-ff1819ba0551 2026-01-29 04:36:29,984 - INFO - Project default-domain:k8s-ctest-new-default-73034959 for Namespace ctest-new-default-73034959 is seen in contrail-api 2026-01-29 04:36:29,984 - DEBUG - Project uuid in Contrail is c7dce712-b563-4a99-9067-d365a56591b0 2026-01-29 04:36:30,124 - DEBUG - Requesting: http://10.0.0.141:8108/Snh_NamespaceDatabaseList?namespace_uuid=72bc96ba-86f0-47de-9890-d962065d2d06 2026-01-29 04:36:30,127 - INFO - Namespace ctest-new-default-73034959 with uuid 72bc96ba-86f0-47de-9890-d962065d2d06 found in kube manager 2026-01-29 04:36:30,128 - INFO - Namespace ctest-new-default-73034959 verification passed 2026-01-29 04:36:30,301 - INFO - Project default-domain:k8s-ctest-non-default-71721767 for Namespace ctest-non-default-71721767 is seen in contrail-api 2026-01-29 04:36:30,302 - DEBUG - Project uuid in Contrail is 459c5cf9-b021-4d47-acbd-2bcce7202e10 2026-01-29 04:36:30,303 - DEBUG - Requesting: http://10.0.0.141:8108/Snh_NamespaceDatabaseList?namespace_uuid=316fd682-2afc-45b1-85e1-7b77d8bbf3fc 2026-01-29 04:36:30,312 - INFO - Namespace ctest-non-default-71721767 with uuid 316fd682-2afc-45b1-85e1-7b77d8bbf3fc found in kube manager 2026-01-29 04:36:30,312 - INFO - Namespace ctest-non-default-71721767 verification passed 2026-01-29 04:36:30,325 - DEBUG - Pod ctest-pod-77324757 not in running state.Currently in Pending 2026-01-29 04:36:35,332 - DEBUG - Pod ctest-pod-77324757 not in running state.Currently in Pending 2026-01-29 04:36:40,341 - INFO - Pod ctest-pod-77324757 is in running state.Got IP 10.47.255.251 2026-01-29 04:36:40,394 - DEBUG - Pod ctest-pod-77324757 has vmi 1567b0c6-fccc-11f0-b235-fa163ef15b3d 2026-01-29 04:36:40,394 - INFO - Verified pod ctest-pod-77324757 in contrail-api 2026-01-29 04:36:40,405 - DEBUG - VMI 1567b0c6-fccc-11f0-b235-fa163ef15b3d is active in agent 10.0.0.242 2026-01-29 04:36:40,405 - INFO - Verified Pod ctest-pod-77324757 in agent 10.0.0.242 2026-01-29 04:36:40,405 - DEBUG - Requesting: http://10.0.0.141:8108/Snh_VirtualMachineDatabaseList?x=fec9c138-3410-4227-8b1c-39ffce8d51de 2026-01-29 04:36:40,409 - INFO - Pod ctest-pod-77324757 with uuid fec9c138-3410-4227-8b1c-39ffce8d51de found in kube manager 2026-01-29 04:36:40,409 - INFO - Pod ctest-pod-77324757 verification passed 2026-01-29 04:36:40,415 - DEBUG - Pod ctest-pod-90385737 not in running state.Currently in Pending 2026-01-29 04:36:45,422 - INFO - Pod ctest-pod-90385737 is in running state.Got IP 10.47.255.250 2026-01-29 04:36:45,434 - DEBUG - Pod ctest-pod-90385737 has vmi 15d467b6-fccc-11f0-b235-fa163ef15b3d 2026-01-29 04:36:45,434 - INFO - Verified pod ctest-pod-90385737 in contrail-api 2026-01-29 04:36:45,447 - DEBUG - VMI 15d467b6-fccc-11f0-b235-fa163ef15b3d is active in agent 10.0.0.242 2026-01-29 04:36:45,447 - INFO - Verified Pod ctest-pod-90385737 in agent 10.0.0.242 2026-01-29 04:36:45,447 - DEBUG - Requesting: http://10.0.0.141:8108/Snh_VirtualMachineDatabaseList?x=c60f5652-347d-4061-95d3-b5b962223cfb 2026-01-29 04:36:45,451 - INFO - Pod ctest-pod-90385737 with uuid c60f5652-347d-4061-95d3-b5b962223cfb found in kube manager 2026-01-29 04:36:45,451 - INFO - Pod ctest-pod-90385737 verification passed 2026-01-29 04:36:45,457 - INFO - Pod ctest-pod-69918613 is in running state.Got IP 10.47.255.249 2026-01-29 04:36:45,510 - DEBUG - Pod ctest-pod-69918613 has vmi 165d70ec-fccc-11f0-b235-fa163ef15b3d 2026-01-29 04:36:45,510 - INFO - Verified pod ctest-pod-69918613 in contrail-api 2026-01-29 04:36:45,523 - DEBUG - VMI 165d70ec-fccc-11f0-b235-fa163ef15b3d is active in agent 10.0.0.242 2026-01-29 04:36:45,524 - INFO - Verified Pod ctest-pod-69918613 in agent 10.0.0.242 2026-01-29 04:36:45,524 - DEBUG - Requesting: http://10.0.0.141:8108/Snh_VirtualMachineDatabaseList?x=291d680d-c973-41af-8fdf-8e17db25ca12 2026-01-29 04:36:45,528 - INFO - Pod ctest-pod-69918613 with uuid 291d680d-c973-41af-8fdf-8e17db25ca12 found in kube manager 2026-01-29 04:36:45,528 - INFO - Pod ctest-pod-69918613 verification passed 2026-01-29 04:36:45,535 - INFO - Pod ctest-pod-43220972 is in running state.Got IP 10.47.255.248 2026-01-29 04:36:45,552 - DEBUG - Pod ctest-pod-43220972 has vmi 16c6ba48-fccc-11f0-b235-fa163ef15b3d 2026-01-29 04:36:45,552 - INFO - Verified pod ctest-pod-43220972 in contrail-api 2026-01-29 04:36:45,566 - DEBUG - VMI 16c6ba48-fccc-11f0-b235-fa163ef15b3d is active in agent 10.0.0.242 2026-01-29 04:36:45,566 - INFO - Verified Pod ctest-pod-43220972 in agent 10.0.0.242 2026-01-29 04:36:45,566 - DEBUG - Requesting: http://10.0.0.141:8108/Snh_VirtualMachineDatabaseList?x=dc8b559a-5d2b-4b24-b878-5323632bcf94 2026-01-29 04:36:45,570 - INFO - Pod ctest-pod-43220972 with uuid dc8b559a-5d2b-4b24-b878-5323632bcf94 found in kube manager 2026-01-29 04:36:45,570 - INFO - Pod ctest-pod-43220972 verification passed 2026-01-29 04:36:45,576 - DEBUG - Pod ctest-pod-42424437 not in running state.Currently in Pending 2026-01-29 04:36:50,583 - INFO - Pod ctest-pod-42424437 is in running state.Got IP 10.47.255.247 2026-01-29 04:36:50,638 - DEBUG - Pod ctest-pod-42424437 has vmi 173d7a84-fccc-11f0-b235-fa163ef15b3d 2026-01-29 04:36:50,638 - INFO - Verified pod ctest-pod-42424437 in contrail-api 2026-01-29 04:36:50,652 - DEBUG - VMI 173d7a84-fccc-11f0-b235-fa163ef15b3d is active in agent 10.0.0.242 2026-01-29 04:36:50,652 - INFO - Verified Pod ctest-pod-42424437 in agent 10.0.0.242 2026-01-29 04:36:50,652 - DEBUG - Requesting: http://10.0.0.141:8108/Snh_VirtualMachineDatabaseList?x=10d4872a-e2a5-46a2-b6c8-add9ef531aea 2026-01-29 04:36:50,656 - INFO - Pod ctest-pod-42424437 with uuid 10d4872a-e2a5-46a2-b6c8-add9ef531aea found in kube manager 2026-01-29 04:36:50,656 - INFO - Pod ctest-pod-42424437 verification passed 2026-01-29 04:36:50,664 - INFO - Pod ctest-pod-31022830 is in running state.Got IP 10.47.255.245 2026-01-29 04:36:50,675 - DEBUG - Pod ctest-pod-31022830 has vmi 181defe2-fccc-11f0-b235-fa163ef15b3d 2026-01-29 04:36:50,675 - INFO - Verified pod ctest-pod-31022830 in contrail-api 2026-01-29 04:36:50,688 - DEBUG - VMI 181defe2-fccc-11f0-b235-fa163ef15b3d is active in agent 10.0.0.242 2026-01-29 04:36:50,688 - INFO - Verified Pod ctest-pod-31022830 in agent 10.0.0.242 2026-01-29 04:36:50,688 - DEBUG - Requesting: http://10.0.0.141:8108/Snh_VirtualMachineDatabaseList?x=37305eaf-6339-4ed2-9a07-512c1e92dd3e 2026-01-29 04:36:50,692 - INFO - Pod ctest-pod-31022830 with uuid 37305eaf-6339-4ed2-9a07-512c1e92dd3e found in kube manager 2026-01-29 04:36:50,692 - INFO - Pod ctest-pod-31022830 verification passed 2026-01-29 04:36:50,698 - INFO - Pod ctest-pod-17427988 is in running state.Got IP 10.47.255.246 2026-01-29 04:36:50,759 - DEBUG - Pod ctest-pod-17427988 has vmi 17afd9c6-fccc-11f0-b235-fa163ef15b3d 2026-01-29 04:36:50,759 - INFO - Verified pod ctest-pod-17427988 in contrail-api 2026-01-29 04:36:50,771 - DEBUG - VMI 17afd9c6-fccc-11f0-b235-fa163ef15b3d is active in agent 10.0.0.242 2026-01-29 04:36:50,771 - INFO - Verified Pod ctest-pod-17427988 in agent 10.0.0.242 2026-01-29 04:36:50,771 - DEBUG - Requesting: http://10.0.0.141:8108/Snh_VirtualMachineDatabaseList?x=fa7ec9d4-325b-408b-a34e-253f4e35fd95 2026-01-29 04:36:50,775 - INFO - Pod ctest-pod-17427988 with uuid fa7ec9d4-325b-408b-a34e-253f4e35fd95 found in kube manager 2026-01-29 04:36:50,775 - INFO - Pod ctest-pod-17427988 verification passed 2026-01-29 04:36:50,782 - INFO - Pod ctest-pod-24981972 is in running state.Got IP 10.47.255.244 2026-01-29 04:36:50,834 - DEBUG - Pod ctest-pod-24981972 has vmi 188f12da-fccc-11f0-b235-fa163ef15b3d 2026-01-29 04:36:50,834 - INFO - Verified pod ctest-pod-24981972 in contrail-api 2026-01-29 04:36:50,846 - DEBUG - VMI 188f12da-fccc-11f0-b235-fa163ef15b3d is active in agent 10.0.0.242 2026-01-29 04:36:50,846 - INFO - Verified Pod ctest-pod-24981972 in agent 10.0.0.242 2026-01-29 04:36:50,846 - DEBUG - Requesting: http://10.0.0.141:8108/Snh_VirtualMachineDatabaseList?x=6cd24ec4-7caa-4781-97ae-ff1819ba0551 2026-01-29 04:36:50,850 - INFO - Pod ctest-pod-24981972 with uuid 6cd24ec4-7caa-4781-97ae-ff1819ba0551 found in kube manager 2026-01-29 04:36:50,850 - INFO - Pod ctest-pod-24981972 verification passed 2026-01-29 04:36:50,851 - INFO - ================================================================================ 2026-01-29 04:36:50,851 - INFO - STARTING TEST : test_ingress_rule_on_pod_with_service 2026-01-29 04:36:50,851 - INFO - TEST DESCRIPTION : Verify that ingress rules on a pod work fine when pods are accessed via a service Steps: 1. Create a service in namespace 1 over a pod which are endpoints of a service 2. Create a network policy to allow 1 pod of namespace 1 as ingress. 3. Verify that service respond correctly to the specified pod in the policy and service do not respond to anyone else. 4. Extend the policy and add the namespaceselector as namespace 2. 5. Verify that service become accessible to the pods of namespace 2. 2026-01-29 04:36:51,089 - DEBUG - Nothing to compare xmpp stats {'10.0.0.242': {'10.0.0.141': '0'}} with 2026-01-29 04:36:51,089 - INFO - Initial checks done. Running the testcase now 2026-01-29 04:36:51,089 - INFO - 2026-01-29 04:36:51,150 - DEBUG - [Pod ctest-pod-77324757] Cmd: echo ctest-pod-77324757 > /usr/share/nginx/html/index.html, Output: 2026-01-29 04:36:51,212 - DEBUG - [Pod ctest-pod-90385737] Cmd: echo ctest-pod-90385737 > /usr/share/nginx/html/index.html, Output: 2026-01-29 04:36:51,261 - DEBUG - [Pod ctest-pod-69918613] Cmd: echo ctest-pod-69918613 > /usr/share/nginx/html/index.html, Output: 2026-01-29 04:36:51,311 - DEBUG - [Pod ctest-pod-43220972] Cmd: echo ctest-pod-43220972 > /usr/share/nginx/html/index.html, Output: 2026-01-29 04:36:51,313 - DEBUG - Service ctest-nginx-svc-54400070 not present 2026-01-29 04:36:51,314 - INFO - Creating service ctest-nginx-svc-54400070 2026-01-29 04:36:51,337 - INFO - Created Service ctest-nginx-svc-54400070 2026-01-29 04:36:51,344 - WARNING - Service UUID 86cbaa68-00fa-487f-b82b-bdf387404d5d not yet found in contrail-api 2026-01-29 04:36:52,352 - INFO - Validated that Service ctest-nginx-svc-54400070 is seen in contrail-api 2026-01-29 04:36:52,352 - DEBUG - Requesting: http://10.0.0.141:8108/Snh_LoadbalancerDatabaseList?x=86cbaa68-00fa-487f-b82b-bdf387404d5d 2026-01-29 04:36:52,356 - INFO - Service ctest-nginx-svc-54400070 with uuid 86cbaa68-00fa-487f-b82b-bdf387404d5d found in kube manager 2026-01-29 04:36:52,356 - INFO - Service ctest-nginx-svc-54400070 verification passed 2026-01-29 04:36:52,424 - DEBUG - [Pod ctest-pod-42424437] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: ctest-pod-90385737 Connecting to 10.98.113.173:80 (10.98.113.173:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-01-29 04:36:52,424 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 passed 2026-01-29 04:36:52,424 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd output: ctest-pod-90385737 Connecting to 10.98.113.173:80 (10.98.113.173:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-01-29 04:36:52,480 - DEBUG - [Pod ctest-pod-42424437] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: ctest-pod-90385737 Connecting to 10.98.113.173:80 (10.98.113.173:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-01-29 04:36:52,480 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 passed 2026-01-29 04:36:52,480 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd output: ctest-pod-90385737 Connecting to 10.98.113.173:80 (10.98.113.173:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-01-29 04:36:52,539 - DEBUG - [Pod ctest-pod-42424437] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: ctest-pod-90385737 Connecting to 10.98.113.173:80 (10.98.113.173:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-01-29 04:36:52,539 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 passed 2026-01-29 04:36:52,539 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd output: ctest-pod-90385737 Connecting to 10.98.113.173:80 (10.98.113.173:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-01-29 04:36:52,584 - DEBUG - [Pod ctest-pod-42424437] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: ctest-pod-90385737 Connecting to 10.98.113.173:80 (10.98.113.173:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-01-29 04:36:52,584 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 passed 2026-01-29 04:36:52,584 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd output: ctest-pod-90385737 Connecting to 10.98.113.173:80 (10.98.113.173:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-01-29 04:36:52,635 - DEBUG - [Pod ctest-pod-42424437] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.113.173:80 (10.98.113.173:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-90385737 2026-01-29 04:36:52,636 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 passed 2026-01-29 04:36:52,636 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd output: Connecting to 10.98.113.173:80 (10.98.113.173:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-90385737 2026-01-29 04:36:52,696 - DEBUG - [Pod ctest-pod-42424437] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: ctest-pod-77324757 Connecting to 10.98.113.173:80 (10.98.113.173:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-01-29 04:36:52,697 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 passed 2026-01-29 04:36:52,697 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd output: ctest-pod-77324757 Connecting to 10.98.113.173:80 (10.98.113.173:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-01-29 04:36:52,697 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-77324757': 1, 'ctest-pod-90385737': 5} 2026-01-29 04:36:52,763 - DEBUG - [Pod ctest-pod-31022830] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: ctest-pod-77324757 Connecting to 10.98.113.173:80 (10.98.113.173:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-01-29 04:36:52,763 - DEBUG - [Pod Pod ctest-pod-31022830] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 passed 2026-01-29 04:36:52,763 - DEBUG - [Pod Pod ctest-pod-31022830] Cmd output: ctest-pod-77324757 Connecting to 10.98.113.173:80 (10.98.113.173:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-01-29 04:36:52,813 - DEBUG - [Pod ctest-pod-31022830] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: ctest-pod-90385737 Connecting to 10.98.113.173:80 (10.98.113.173:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-01-29 04:36:52,813 - DEBUG - [Pod Pod ctest-pod-31022830] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 passed 2026-01-29 04:36:52,813 - DEBUG - [Pod Pod ctest-pod-31022830] Cmd output: ctest-pod-90385737 Connecting to 10.98.113.173:80 (10.98.113.173:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-01-29 04:36:52,813 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-77324757': 1, 'ctest-pod-90385737': 1} 2026-01-29 04:36:52,816 - DEBUG - Network policy ingress-on-service not present 2026-01-29 04:36:52,817 - INFO - Creating Network Policy ingress-on-service 2026-01-29 04:36:52,827 - INFO - Network policy found in k8s 2026-01-29 04:36:52,827 - DEBUG - Requesting: http://10.0.0.141:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=67cc346c-2daa-4fd3-9a32-01315ad52fcd 2026-01-29 04:36:52,831 - INFO - Network Policy ingress-on-service with uuid 67cc346c-2daa-4fd3-9a32-01315ad52fcd found in kube manager 2026-01-29 04:36:52,862 - INFO - Network policy with name default-policy-management:k8s-allowall found in agent 2026-01-29 04:36:52,862 - INFO - Network policy with name default-policy-management:k8s-allowall associated with default ks8 2026-01-29 04:36:52,863 - DEBUG - Requesting: http://10.0.0.141:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=67cc346c-2daa-4fd3-9a32-01315ad52fcd 2026-01-29 04:36:52,886 - WARNING - Network policy with name ingress-on-service not found in agent 2026-01-29 04:36:54,887 - DEBUG - Requesting: http://10.0.0.141:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=67cc346c-2daa-4fd3-9a32-01315ad52fcd 2026-01-29 04:36:54,914 - INFO - Network policy with name ingress-on-service found in agent 2026-01-29 04:36:54,915 - INFO - Network Policy ingress-on-service verification passed 2026-01-29 04:36:59,976 - DEBUG - [Pod ctest-pod-42424437] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:36:59,976 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:36:59,976 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:37:05,034 - DEBUG - [Pod ctest-pod-42424437] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:37:05,035 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:37:05,035 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:37:10,111 - DEBUG - [Pod ctest-pod-42424437] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:37:10,111 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:37:10,111 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:37:15,164 - DEBUG - [Pod ctest-pod-42424437] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:37:15,164 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:37:15,164 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:37:20,224 - DEBUG - [Pod ctest-pod-42424437] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:37:20,224 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:37:20,225 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:37:25,274 - DEBUG - [Pod ctest-pod-42424437] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:37:25,274 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:37:25,274 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:37:30,324 - DEBUG - [Pod ctest-pod-42424437] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:37:30,324 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:37:30,324 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:37:35,383 - DEBUG - [Pod ctest-pod-42424437] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:37:35,383 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:37:35,383 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:37:40,439 - DEBUG - [Pod ctest-pod-42424437] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:37:40,439 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:37:40,439 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:37:45,491 - DEBUG - [Pod ctest-pod-42424437] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:37:45,491 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:37:45,491 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:37:45,491 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-77324757': 0, 'ctest-pod-90385737': 0} 2026-01-29 04:37:45,548 - DEBUG - [Pod ctest-pod-17427988] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: ctest-pod-90385737 Connecting to 10.98.113.173:80 (10.98.113.173:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-01-29 04:37:45,549 - DEBUG - [Pod Pod ctest-pod-17427988] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 passed 2026-01-29 04:37:45,549 - DEBUG - [Pod Pod ctest-pod-17427988] Cmd output: ctest-pod-90385737 Connecting to 10.98.113.173:80 (10.98.113.173:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-01-29 04:37:45,615 - DEBUG - [Pod ctest-pod-17427988] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: ctest-pod-77324757 Connecting to 10.98.113.173:80 (10.98.113.173:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-01-29 04:37:45,615 - DEBUG - [Pod Pod ctest-pod-17427988] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 passed 2026-01-29 04:37:45,615 - DEBUG - [Pod Pod ctest-pod-17427988] Cmd output: ctest-pod-77324757 Connecting to 10.98.113.173:80 (10.98.113.173:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-01-29 04:37:45,615 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-77324757': 1, 'ctest-pod-90385737': 1} 2026-01-29 04:37:50,682 - DEBUG - [Pod ctest-pod-31022830] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:37:50,682 - DEBUG - [Pod Pod ctest-pod-31022830] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:37:50,682 - DEBUG - [Pod Pod ctest-pod-31022830] Cmd output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:37:55,735 - DEBUG - [Pod ctest-pod-31022830] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:37:55,736 - DEBUG - [Pod Pod ctest-pod-31022830] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:37:55,736 - DEBUG - [Pod Pod ctest-pod-31022830] Cmd output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:38:00,786 - DEBUG - [Pod ctest-pod-31022830] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:38:00,786 - DEBUG - [Pod Pod ctest-pod-31022830] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:38:00,786 - DEBUG - [Pod Pod ctest-pod-31022830] Cmd output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:38:05,835 - DEBUG - [Pod ctest-pod-31022830] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:38:05,835 - DEBUG - [Pod Pod ctest-pod-31022830] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:38:05,835 - DEBUG - [Pod Pod ctest-pod-31022830] Cmd output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:38:10,887 - DEBUG - [Pod ctest-pod-31022830] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:38:10,887 - DEBUG - [Pod Pod ctest-pod-31022830] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:38:10,887 - DEBUG - [Pod Pod ctest-pod-31022830] Cmd output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:38:15,934 - DEBUG - [Pod ctest-pod-31022830] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:38:15,935 - DEBUG - [Pod Pod ctest-pod-31022830] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:38:15,935 - DEBUG - [Pod Pod ctest-pod-31022830] Cmd output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:38:20,987 - DEBUG - [Pod ctest-pod-31022830] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:38:20,987 - DEBUG - [Pod Pod ctest-pod-31022830] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:38:20,987 - DEBUG - [Pod Pod ctest-pod-31022830] Cmd output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:38:26,041 - DEBUG - [Pod ctest-pod-31022830] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:38:26,041 - DEBUG - [Pod Pod ctest-pod-31022830] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:38:26,041 - DEBUG - [Pod Pod ctest-pod-31022830] Cmd output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:38:31,090 - DEBUG - [Pod ctest-pod-31022830] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:38:31,090 - DEBUG - [Pod Pod ctest-pod-31022830] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:38:31,091 - DEBUG - [Pod Pod ctest-pod-31022830] Cmd output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:38:36,142 - DEBUG - [Pod ctest-pod-31022830] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:38:36,143 - DEBUG - [Pod Pod ctest-pod-31022830] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:38:36,143 - DEBUG - [Pod Pod ctest-pod-31022830] Cmd output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:38:36,143 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-77324757': 0, 'ctest-pod-90385737': 0} 2026-01-29 04:38:38,187 - DEBUG - [Pod ctest-pod-17427988] Cmd: ping -c 3 10.47.255.247, Output: PING 10.47.255.247 (10.47.255.247): 56 data bytes 64 bytes from 10.47.255.247: seq=0 ttl=63 time=0.972 ms 64 bytes from 10.47.255.247: seq=1 ttl=63 time=0.074 ms 64 bytes from 10.47.255.247: seq=2 ttl=63 time=0.090 ms --- 10.47.255.247 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.074/0.378/0.972 ms 2026-01-29 04:38:38,188 - INFO - Ping check to IP 10.47.255.247 from pod ctest-pod-17427988 with expectation True passed 2026-01-29 04:38:40,239 - DEBUG - [Pod ctest-pod-42424437] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=0.789 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.071 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.092 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.071/0.317/0.789 ms 2026-01-29 04:38:40,239 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-42424437 with expectation True passed 2026-01-29 04:38:42,288 - DEBUG - [Pod ctest-pod-31022830] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=1.913 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.068 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.076 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.068/0.685/1.913 ms 2026-01-29 04:38:42,288 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-31022830 with expectation True passed 2026-01-29 04:38:42,346 - DEBUG - [Pod ctest-pod-17427988] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-01-29 04:38:42,346 - DEBUG - [Pod Pod ctest-pod-17427988] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2026-01-29 04:38:42,346 - DEBUG - [Pod Pod ctest-pod-17427988] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-01-29 04:38:42,346 - INFO - wget check of of http://10.47.255.251 from ctest-pod-17427988 passed 2026-01-29 04:38:42,400 - DEBUG - [Pod ctest-pod-17427988] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-01-29 04:38:42,400 - DEBUG - [Pod Pod ctest-pod-17427988] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2026-01-29 04:38:42,400 - DEBUG - [Pod Pod ctest-pod-17427988] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-01-29 04:38:42,400 - INFO - wget check of of http://10.47.255.250 from ctest-pod-17427988 passed 2026-01-29 04:38:47,447 - DEBUG - [Pod ctest-pod-42424437] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2026-01-29 04:38:47,447 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2026-01-29 04:38:47,447 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2026-01-29 04:38:47,447 - INFO - wget check of of http://10.47.255.250 from ctest-pod-42424437 passed 2026-01-29 04:38:52,501 - DEBUG - [Pod ctest-pod-24981972] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) wget: download timed out 2026-01-29 04:38:52,501 - DEBUG - [Pod Pod ctest-pod-24981972] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.251 (10.47.255.251:80) wget: download timed out 2026-01-29 04:38:52,501 - DEBUG - [Pod Pod ctest-pod-24981972] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) wget: download timed out 2026-01-29 04:38:52,501 - INFO - wget check of of http://10.47.255.251 from ctest-pod-24981972 passed 2026-01-29 04:38:52,510 - INFO - Updating Network Policy ingress-on-service 2026-01-29 04:38:57,583 - DEBUG - [Pod ctest-pod-42424437] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:38:57,583 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:38:57,583 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:39:02,631 - DEBUG - [Pod ctest-pod-42424437] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:39:02,631 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:39:02,631 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:39:07,684 - DEBUG - [Pod ctest-pod-42424437] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:39:07,684 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:39:07,684 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:39:12,743 - DEBUG - [Pod ctest-pod-42424437] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:39:12,744 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:39:12,744 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:39:17,791 - DEBUG - [Pod ctest-pod-42424437] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:39:17,791 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:39:17,791 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:39:22,848 - DEBUG - [Pod ctest-pod-42424437] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:39:22,848 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:39:22,848 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:39:27,914 - DEBUG - [Pod ctest-pod-42424437] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:39:27,914 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:39:27,914 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:39:28,884 - WARNING - ApiException is caught (0) Reason: Connection is already closed. . Retrying... 2026-01-29 04:39:39,210 - DEBUG - [Pod ctest-pod-42424437] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:39:39,210 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:39:39,210 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:39:44,259 - DEBUG - [Pod ctest-pod-42424437] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:39:44,260 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:39:44,260 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:39:49,312 - DEBUG - [Pod ctest-pod-42424437] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:39:49,312 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:39:49,312 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd output: Connecting to 10.98.113.173:80 (10.98.113.173:80) wget: download timed out 2026-01-29 04:39:49,312 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-77324757': 0, 'ctest-pod-90385737': 0} 2026-01-29 04:39:49,365 - DEBUG - [Pod ctest-pod-17427988] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: ctest-pod-90385737 Connecting to 10.98.113.173:80 (10.98.113.173:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-01-29 04:39:49,365 - DEBUG - [Pod Pod ctest-pod-17427988] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 passed 2026-01-29 04:39:49,365 - DEBUG - [Pod Pod ctest-pod-17427988] Cmd output: ctest-pod-90385737 Connecting to 10.98.113.173:80 (10.98.113.173:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-01-29 04:39:49,413 - DEBUG - [Pod ctest-pod-17427988] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: ctest-pod-90385737 Connecting to 10.98.113.173:80 (10.98.113.173:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-01-29 04:39:49,413 - DEBUG - [Pod Pod ctest-pod-17427988] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 passed 2026-01-29 04:39:49,413 - DEBUG - [Pod Pod ctest-pod-17427988] Cmd output: ctest-pod-90385737 Connecting to 10.98.113.173:80 (10.98.113.173:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-01-29 04:39:49,471 - DEBUG - [Pod ctest-pod-17427988] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: ctest-pod-90385737 Connecting to 10.98.113.173:80 (10.98.113.173:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-01-29 04:39:49,471 - DEBUG - [Pod Pod ctest-pod-17427988] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 passed 2026-01-29 04:39:49,472 - DEBUG - [Pod Pod ctest-pod-17427988] Cmd output: ctest-pod-90385737 Connecting to 10.98.113.173:80 (10.98.113.173:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-01-29 04:39:49,509 - DEBUG - [Pod ctest-pod-17427988] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: ctest-pod-90385737 Connecting to 10.98.113.173:80 (10.98.113.173:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-01-29 04:39:49,509 - DEBUG - [Pod Pod ctest-pod-17427988] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 passed 2026-01-29 04:39:49,509 - DEBUG - [Pod Pod ctest-pod-17427988] Cmd output: ctest-pod-90385737 Connecting to 10.98.113.173:80 (10.98.113.173:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-01-29 04:39:49,583 - DEBUG - [Pod ctest-pod-17427988] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: Connecting to 10.98.113.173:80 (10.98.113.173:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-90385737 2026-01-29 04:39:49,583 - DEBUG - [Pod Pod ctest-pod-17427988] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 passed 2026-01-29 04:39:49,583 - DEBUG - [Pod Pod ctest-pod-17427988] Cmd output: Connecting to 10.98.113.173:80 (10.98.113.173:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-90385737 2026-01-29 04:39:49,643 - DEBUG - [Pod ctest-pod-17427988] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: ctest-pod-77324757 Connecting to 10.98.113.173:80 (10.98.113.173:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-01-29 04:39:49,643 - DEBUG - [Pod Pod ctest-pod-17427988] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 passed 2026-01-29 04:39:49,643 - DEBUG - [Pod Pod ctest-pod-17427988] Cmd output: ctest-pod-77324757 Connecting to 10.98.113.173:80 (10.98.113.173:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-01-29 04:39:49,643 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-77324757': 1, 'ctest-pod-90385737': 5} 2026-01-29 04:39:49,718 - DEBUG - [Pod ctest-pod-31022830] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: ctest-pod-77324757 Connecting to 10.98.113.173:80 (10.98.113.173:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-01-29 04:39:49,718 - DEBUG - [Pod Pod ctest-pod-31022830] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 passed 2026-01-29 04:39:49,718 - DEBUG - [Pod Pod ctest-pod-31022830] Cmd output: ctest-pod-77324757 Connecting to 10.98.113.173:80 (10.98.113.173:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-01-29 04:39:49,774 - DEBUG - [Pod ctest-pod-31022830] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: ctest-pod-77324757 Connecting to 10.98.113.173:80 (10.98.113.173:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-01-29 04:39:49,775 - DEBUG - [Pod Pod ctest-pod-31022830] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 passed 2026-01-29 04:39:49,775 - DEBUG - [Pod Pod ctest-pod-31022830] Cmd output: ctest-pod-77324757 Connecting to 10.98.113.173:80 (10.98.113.173:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-01-29 04:39:49,836 - DEBUG - [Pod ctest-pod-31022830] Cmd: wget http://10.98.113.173:80/ -O - -T 5 -t 1 , Output: ctest-pod-90385737 Connecting to 10.98.113.173:80 (10.98.113.173:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-01-29 04:39:49,836 - DEBUG - [Pod Pod ctest-pod-31022830] Cmd wget http://10.98.113.173:80/ -O - -T 5 -t 1 passed 2026-01-29 04:39:49,837 - DEBUG - [Pod Pod ctest-pod-31022830] Cmd output: ctest-pod-90385737 Connecting to 10.98.113.173:80 (10.98.113.173:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-01-29 04:39:49,837 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-77324757': 2, 'ctest-pod-90385737': 1} 2026-01-29 04:39:51,886 - DEBUG - [Pod ctest-pod-17427988] Cmd: ping -c 3 10.47.255.247, Output: PING 10.47.255.247 (10.47.255.247): 56 data bytes 64 bytes from 10.47.255.247: seq=0 ttl=63 time=0.708 ms 64 bytes from 10.47.255.247: seq=1 ttl=63 time=0.079 ms 64 bytes from 10.47.255.247: seq=2 ttl=63 time=0.090 ms --- 10.47.255.247 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.079/0.292/0.708 ms 2026-01-29 04:39:51,886 - INFO - Ping check to IP 10.47.255.247 from pod ctest-pod-17427988 with expectation True passed 2026-01-29 04:39:53,942 - DEBUG - [Pod ctest-pod-42424437] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=0.362 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.076 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.080 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.076/0.172/0.362 ms 2026-01-29 04:39:53,943 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-42424437 with expectation True passed 2026-01-29 04:39:55,992 - DEBUG - [Pod ctest-pod-31022830] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=0.801 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.077 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.095 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.077/0.324/0.801 ms 2026-01-29 04:39:55,992 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-31022830 with expectation True passed 2026-01-29 04:39:56,058 - DEBUG - [Pod ctest-pod-17427988] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-01-29 04:39:56,058 - DEBUG - [Pod Pod ctest-pod-17427988] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2026-01-29 04:39:56,058 - DEBUG - [Pod Pod ctest-pod-17427988] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-01-29 04:39:56,058 - INFO - wget check of of http://10.47.255.251 from ctest-pod-17427988 passed 2026-01-29 04:39:56,109 - DEBUG - [Pod ctest-pod-17427988] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-01-29 04:39:56,109 - DEBUG - [Pod Pod ctest-pod-17427988] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2026-01-29 04:39:56,109 - DEBUG - [Pod Pod ctest-pod-17427988] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-01-29 04:39:56,109 - INFO - wget check of of http://10.47.255.250 from ctest-pod-17427988 passed 2026-01-29 04:40:01,159 - DEBUG - [Pod ctest-pod-42424437] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2026-01-29 04:40:01,159 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2026-01-29 04:40:01,159 - DEBUG - [Pod Pod ctest-pod-42424437] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2026-01-29 04:40:01,159 - INFO - wget check of of http://10.47.255.250 from ctest-pod-42424437 passed 2026-01-29 04:40:01,215 - DEBUG - [Pod ctest-pod-24981972] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-01-29 04:40:01,215 - DEBUG - [Pod Pod ctest-pod-24981972] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2026-01-29 04:40:01,215 - DEBUG - [Pod Pod ctest-pod-24981972] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-01-29 04:40:01,216 - INFO - wget check of of http://10.47.255.251 from ctest-pod-24981972 passed 2026-01-29 04:40:01,216 - INFO - Deleting Network Policy : ingress-on-service 2026-01-29 04:40:01,216 - INFO - Deleting service : ctest-nginx-svc-54400070 2026-01-29 04:40:01,468 - DEBUG - No XMPP flaps were noticed during the test