2026-02-10 04:04:46,378 - INFO - Using existing project ['default-domain', 'k8s-default'](4fecdb7c-3687-43b7-a903-3380b23449b1) 2026-02-10 04:04:46,400 - DEBUG - Namespace ctest-new-default-83327604 not present 2026-02-10 04:04:46,408 - INFO - Created namespace ctest-new-default-83327604 2026-02-10 04:04:49,415 - DEBUG - Namespace ctest-non-default-33960794 not present 2026-02-10 04:04:49,432 - INFO - Created namespace ctest-non-default-33960794 2026-02-10 04:04:52,451 - DEBUG - Pod ctest-pod-32123737 not present 2026-02-10 04:04:52,452 - INFO - Creating Pod ctest-pod-32123737 2026-02-10 04:04:52,495 - DEBUG - Pod : ctest-pod-32123737 UUID is 5ad221b4-abb7-4f4e-9bfe-12a3eeb0cad1 2026-02-10 04:04:52,508 - DEBUG - Pod ctest-pod-94493639 not present 2026-02-10 04:04:52,509 - INFO - Creating Pod ctest-pod-94493639 2026-02-10 04:04:52,568 - DEBUG - Pod : ctest-pod-94493639 UUID is bc6402d5-235d-4c3e-a45e-1f7a24e2fbe0 2026-02-10 04:04:52,581 - DEBUG - Pod ctest-pod-30974977 not present 2026-02-10 04:04:52,582 - INFO - Creating Pod ctest-pod-30974977 2026-02-10 04:04:52,629 - DEBUG - Pod : ctest-pod-30974977 UUID is cb1caaa1-7663-4685-a32c-ef842ba3b4ea 2026-02-10 04:04:52,632 - DEBUG - Pod ctest-pod-25237338 not present 2026-02-10 04:04:52,633 - INFO - Creating Pod ctest-pod-25237338 2026-02-10 04:04:52,663 - DEBUG - Pod : ctest-pod-25237338 UUID is 8f1448a2-8f7a-49d2-a015-234f5a5ffe53 2026-02-10 04:04:52,665 - DEBUG - Pod ctest-pod-57887898 not present 2026-02-10 04:04:52,666 - INFO - Creating Pod ctest-pod-57887898 2026-02-10 04:04:52,701 - DEBUG - Pod : ctest-pod-57887898 UUID is d2d23d92-231d-41c4-bb97-437a795dc1fc 2026-02-10 04:04:52,704 - DEBUG - Pod ctest-pod-58728610 not present 2026-02-10 04:04:52,704 - INFO - Creating Pod ctest-pod-58728610 2026-02-10 04:04:52,733 - DEBUG - Pod : ctest-pod-58728610 UUID is e3987d54-1bd3-4c18-8b66-3968afad7a6b 2026-02-10 04:04:52,746 - DEBUG - Pod ctest-pod-66196454 not present 2026-02-10 04:04:52,746 - INFO - Creating Pod ctest-pod-66196454 2026-02-10 04:04:52,784 - DEBUG - Pod : ctest-pod-66196454 UUID is 109b7879-1a6d-4836-b84b-28394237a805 2026-02-10 04:04:52,787 - DEBUG - Pod ctest-pod-85178032 not present 2026-02-10 04:04:52,788 - INFO - Creating Pod ctest-pod-85178032 2026-02-10 04:04:52,812 - DEBUG - Pod : ctest-pod-85178032 UUID is f4fe607b-529f-4c65-b4af-e3502eb59623 2026-02-10 04:04:53,044 - INFO - Project default-domain:k8s-ctest-new-default-83327604 for Namespace ctest-new-default-83327604 is seen in contrail-api 2026-02-10 04:04:53,044 - DEBUG - Project uuid in Contrail is dcdf0f1d-6593-4a42-aa92-2568caf1d8ce 2026-02-10 04:04:53,265 - DEBUG - Requesting: http://10.0.0.242:8108/Snh_NamespaceDatabaseList?namespace_uuid=15e26db0-9f1b-452e-8247-57322bae74cb 2026-02-10 04:04:53,269 - INFO - Namespace ctest-new-default-83327604 with uuid 15e26db0-9f1b-452e-8247-57322bae74cb found in kube manager 2026-02-10 04:04:53,269 - INFO - Namespace ctest-new-default-83327604 verification passed 2026-02-10 04:04:53,481 - INFO - Project default-domain:k8s-ctest-non-default-33960794 for Namespace ctest-non-default-33960794 is seen in contrail-api 2026-02-10 04:04:53,481 - DEBUG - Project uuid in Contrail is c9804e38-b06b-40bd-9d79-f1b9f4901aee 2026-02-10 04:04:53,481 - DEBUG - Requesting: http://10.0.0.242:8108/Snh_NamespaceDatabaseList?namespace_uuid=6a82d95d-fdfb-4780-b028-c29e7676a2af 2026-02-10 04:04:53,487 - INFO - Namespace ctest-non-default-33960794 with uuid 6a82d95d-fdfb-4780-b028-c29e7676a2af found in kube manager 2026-02-10 04:04:53,488 - INFO - Namespace ctest-non-default-33960794 verification passed 2026-02-10 04:04:53,497 - DEBUG - Pod ctest-pod-32123737 not in running state.Currently in Pending 2026-02-10 04:04:58,506 - DEBUG - Pod ctest-pod-32123737 not in running state.Currently in Pending 2026-02-10 04:05:03,517 - INFO - Pod ctest-pod-32123737 is in running state.Got IP 10.47.255.251 2026-02-10 04:05:03,577 - DEBUG - Pod ctest-pod-32123737 has vmi a7cd184e-0635-11f1-8b30-fa163e9732a7 2026-02-10 04:05:03,577 - INFO - Verified pod ctest-pod-32123737 in contrail-api 2026-02-10 04:05:03,592 - DEBUG - VMI a7cd184e-0635-11f1-8b30-fa163e9732a7 is active in agent 10.0.0.23 2026-02-10 04:05:03,592 - INFO - Verified Pod ctest-pod-32123737 in agent 10.0.0.23 2026-02-10 04:05:03,592 - DEBUG - Requesting: http://10.0.0.242:8108/Snh_VirtualMachineDatabaseList?x=5ad221b4-abb7-4f4e-9bfe-12a3eeb0cad1 2026-02-10 04:05:03,597 - INFO - Pod ctest-pod-32123737 with uuid 5ad221b4-abb7-4f4e-9bfe-12a3eeb0cad1 found in kube manager 2026-02-10 04:05:03,597 - INFO - Pod ctest-pod-32123737 verification passed 2026-02-10 04:05:03,605 - INFO - Pod ctest-pod-94493639 is in running state.Got IP 10.47.255.250 2026-02-10 04:05:03,620 - DEBUG - Pod ctest-pod-94493639 has vmi a844b642-0635-11f1-8b30-fa163e9732a7 2026-02-10 04:05:03,620 - INFO - Verified pod ctest-pod-94493639 in contrail-api 2026-02-10 04:05:03,635 - DEBUG - VMI a844b642-0635-11f1-8b30-fa163e9732a7 is active in agent 10.0.0.23 2026-02-10 04:05:03,635 - INFO - Verified Pod ctest-pod-94493639 in agent 10.0.0.23 2026-02-10 04:05:03,635 - DEBUG - Requesting: http://10.0.0.242:8108/Snh_VirtualMachineDatabaseList?x=bc6402d5-235d-4c3e-a45e-1f7a24e2fbe0 2026-02-10 04:05:03,641 - INFO - Pod ctest-pod-94493639 with uuid bc6402d5-235d-4c3e-a45e-1f7a24e2fbe0 found in kube manager 2026-02-10 04:05:03,642 - INFO - Pod ctest-pod-94493639 verification passed 2026-02-10 04:05:03,651 - INFO - Pod ctest-pod-30974977 is in running state.Got IP 10.47.255.249 2026-02-10 04:05:03,710 - DEBUG - Pod ctest-pod-30974977 has vmi a8ec5a82-0635-11f1-8b30-fa163e9732a7 2026-02-10 04:05:03,710 - INFO - Verified pod ctest-pod-30974977 in contrail-api 2026-02-10 04:05:03,725 - DEBUG - VMI a8ec5a82-0635-11f1-8b30-fa163e9732a7 is active in agent 10.0.0.23 2026-02-10 04:05:03,725 - INFO - Verified Pod ctest-pod-30974977 in agent 10.0.0.23 2026-02-10 04:05:03,725 - DEBUG - Requesting: http://10.0.0.242:8108/Snh_VirtualMachineDatabaseList?x=cb1caaa1-7663-4685-a32c-ef842ba3b4ea 2026-02-10 04:05:03,730 - INFO - Pod ctest-pod-30974977 with uuid cb1caaa1-7663-4685-a32c-ef842ba3b4ea found in kube manager 2026-02-10 04:05:03,730 - INFO - Pod ctest-pod-30974977 verification passed 2026-02-10 04:05:03,737 - DEBUG - Pod ctest-pod-25237338 not in running state.Currently in Pending 2026-02-10 04:05:08,747 - INFO - Pod ctest-pod-25237338 is in running state.Got IP 10.47.255.248 2026-02-10 04:05:08,760 - DEBUG - Pod ctest-pod-25237338 has vmi a96df4ac-0635-11f1-8b30-fa163e9732a7 2026-02-10 04:05:08,760 - INFO - Verified pod ctest-pod-25237338 in contrail-api 2026-02-10 04:05:08,776 - DEBUG - VMI a96df4ac-0635-11f1-8b30-fa163e9732a7 is active in agent 10.0.0.23 2026-02-10 04:05:08,776 - INFO - Verified Pod ctest-pod-25237338 in agent 10.0.0.23 2026-02-10 04:05:08,776 - DEBUG - Requesting: http://10.0.0.242:8108/Snh_VirtualMachineDatabaseList?x=8f1448a2-8f7a-49d2-a015-234f5a5ffe53 2026-02-10 04:05:08,780 - INFO - Pod ctest-pod-25237338 with uuid 8f1448a2-8f7a-49d2-a015-234f5a5ffe53 found in kube manager 2026-02-10 04:05:08,781 - INFO - Pod ctest-pod-25237338 verification passed 2026-02-10 04:05:08,788 - DEBUG - Pod ctest-pod-57887898 not in running state.Currently in Pending 2026-02-10 04:05:13,797 - INFO - Pod ctest-pod-57887898 is in running state.Got IP 10.47.255.247 2026-02-10 04:05:13,853 - DEBUG - Pod ctest-pod-57887898 has vmi a9e74280-0635-11f1-8b30-fa163e9732a7 2026-02-10 04:05:13,853 - INFO - Verified pod ctest-pod-57887898 in contrail-api 2026-02-10 04:05:13,871 - DEBUG - VMI a9e74280-0635-11f1-8b30-fa163e9732a7 is active in agent 10.0.0.23 2026-02-10 04:05:13,871 - INFO - Verified Pod ctest-pod-57887898 in agent 10.0.0.23 2026-02-10 04:05:13,871 - DEBUG - Requesting: http://10.0.0.242:8108/Snh_VirtualMachineDatabaseList?x=d2d23d92-231d-41c4-bb97-437a795dc1fc 2026-02-10 04:05:13,876 - INFO - Pod ctest-pod-57887898 with uuid d2d23d92-231d-41c4-bb97-437a795dc1fc found in kube manager 2026-02-10 04:05:13,876 - INFO - Pod ctest-pod-57887898 verification passed 2026-02-10 04:05:13,883 - INFO - Pod ctest-pod-66196454 is in running state.Got IP 10.47.255.245 2026-02-10 04:05:13,897 - DEBUG - Pod ctest-pod-66196454 has vmi aaf2b2c2-0635-11f1-8b30-fa163e9732a7 2026-02-10 04:05:13,897 - INFO - Verified pod ctest-pod-66196454 in contrail-api 2026-02-10 04:05:13,911 - DEBUG - VMI aaf2b2c2-0635-11f1-8b30-fa163e9732a7 is active in agent 10.0.0.23 2026-02-10 04:05:13,911 - INFO - Verified Pod ctest-pod-66196454 in agent 10.0.0.23 2026-02-10 04:05:13,911 - DEBUG - Requesting: http://10.0.0.242:8108/Snh_VirtualMachineDatabaseList?x=109b7879-1a6d-4836-b84b-28394237a805 2026-02-10 04:05:13,916 - INFO - Pod ctest-pod-66196454 with uuid 109b7879-1a6d-4836-b84b-28394237a805 found in kube manager 2026-02-10 04:05:13,916 - INFO - Pod ctest-pod-66196454 verification passed 2026-02-10 04:05:13,926 - INFO - Pod ctest-pod-58728610 is in running state.Got IP 10.47.255.246 2026-02-10 04:05:13,989 - DEBUG - Pod ctest-pod-58728610 has vmi aa74e6da-0635-11f1-8b30-fa163e9732a7 2026-02-10 04:05:13,989 - INFO - Verified pod ctest-pod-58728610 in contrail-api 2026-02-10 04:05:14,003 - DEBUG - VMI aa74e6da-0635-11f1-8b30-fa163e9732a7 is active in agent 10.0.0.23 2026-02-10 04:05:14,003 - INFO - Verified Pod ctest-pod-58728610 in agent 10.0.0.23 2026-02-10 04:05:14,004 - DEBUG - Requesting: http://10.0.0.242:8108/Snh_VirtualMachineDatabaseList?x=e3987d54-1bd3-4c18-8b66-3968afad7a6b 2026-02-10 04:05:14,008 - INFO - Pod ctest-pod-58728610 with uuid e3987d54-1bd3-4c18-8b66-3968afad7a6b found in kube manager 2026-02-10 04:05:14,008 - INFO - Pod ctest-pod-58728610 verification passed 2026-02-10 04:05:14,017 - INFO - Pod ctest-pod-85178032 is in running state.Got IP 10.47.255.244 2026-02-10 04:05:14,073 - DEBUG - Pod ctest-pod-85178032 has vmi ab7235ba-0635-11f1-8b30-fa163e9732a7 2026-02-10 04:05:14,073 - INFO - Verified pod ctest-pod-85178032 in contrail-api 2026-02-10 04:05:14,101 - DEBUG - VMI ab7235ba-0635-11f1-8b30-fa163e9732a7 is active in agent 10.0.0.23 2026-02-10 04:05:14,101 - INFO - Verified Pod ctest-pod-85178032 in agent 10.0.0.23 2026-02-10 04:05:14,101 - DEBUG - Requesting: http://10.0.0.242:8108/Snh_VirtualMachineDatabaseList?x=f4fe607b-529f-4c65-b4af-e3502eb59623 2026-02-10 04:05:14,106 - INFO - Pod ctest-pod-85178032 with uuid f4fe607b-529f-4c65-b4af-e3502eb59623 found in kube manager 2026-02-10 04:05:14,106 - INFO - Pod ctest-pod-85178032 verification passed 2026-02-10 04:05:14,106 - INFO - ================================================================================ 2026-02-10 04:05:14,106 - INFO - STARTING TEST : test_ingress_rule_on_pod_with_service 2026-02-10 04:05:14,106 - INFO - TEST DESCRIPTION : Verify that ingress rules on a pod work fine when pods are accessed via a service Steps: 1. Create a service in namespace 1 over a pod which are endpoints of a service 2. Create a network policy to allow 1 pod of namespace 1 as ingress. 3. Verify that service respond correctly to the specified pod in the policy and service do not respond to anyone else. 4. Extend the policy and add the namespaceselector as namespace 2. 5. Verify that service become accessible to the pods of namespace 2. 2026-02-10 04:05:14,340 - DEBUG - Nothing to compare xmpp stats {'10.0.0.23': {'10.0.0.242': '0'}} with 2026-02-10 04:05:14,340 - INFO - Initial checks done. Running the testcase now 2026-02-10 04:05:14,340 - INFO - 2026-02-10 04:05:14,415 - DEBUG - [Pod ctest-pod-32123737] Cmd: echo ctest-pod-32123737 > /usr/share/nginx/html/index.html, Output: 2026-02-10 04:05:14,475 - DEBUG - [Pod ctest-pod-94493639] Cmd: echo ctest-pod-94493639 > /usr/share/nginx/html/index.html, Output: 2026-02-10 04:05:14,555 - DEBUG - [Pod ctest-pod-30974977] Cmd: echo ctest-pod-30974977 > /usr/share/nginx/html/index.html, Output: 2026-02-10 04:05:14,615 - DEBUG - [Pod ctest-pod-25237338] Cmd: echo ctest-pod-25237338 > /usr/share/nginx/html/index.html, Output: 2026-02-10 04:05:14,618 - DEBUG - Service ctest-nginx-svc-32827184 not present 2026-02-10 04:05:14,619 - INFO - Creating service ctest-nginx-svc-32827184 2026-02-10 04:05:14,644 - INFO - Created Service ctest-nginx-svc-32827184 2026-02-10 04:05:14,653 - WARNING - Service UUID ef76c29e-c5b7-4e5d-9391-39bfb5ca5885 not yet found in contrail-api 2026-02-10 04:05:15,667 - INFO - Validated that Service ctest-nginx-svc-32827184 is seen in contrail-api 2026-02-10 04:05:15,667 - DEBUG - Requesting: http://10.0.0.242:8108/Snh_LoadbalancerDatabaseList?x=ef76c29e-c5b7-4e5d-9391-39bfb5ca5885 2026-02-10 04:05:15,675 - INFO - Service ctest-nginx-svc-32827184 with uuid ef76c29e-c5b7-4e5d-9391-39bfb5ca5885 found in kube manager 2026-02-10 04:05:15,676 - INFO - Service ctest-nginx-svc-32827184 verification passed 2026-02-10 04:05:16,763 - DEBUG - [Pod ctest-pod-57887898] Cmd: wget http://10.109.194.90:80/ -O - -T 5 -t 1 , Output: Connecting to 10.109.194.90:80 (10.109.194.90:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-32123737 2026-02-10 04:05:16,763 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd wget http://10.109.194.90:80/ -O - -T 5 -t 1 passed 2026-02-10 04:05:16,763 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd output: Connecting to 10.109.194.90:80 (10.109.194.90:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-32123737 2026-02-10 04:05:16,819 - DEBUG - [Pod ctest-pod-57887898] Cmd: wget http://10.109.194.90:80/ -O - -T 5 -t 1 , Output: ctest-pod-94493639 Connecting to 10.109.194.90:80 (10.109.194.90:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-02-10 04:05:16,820 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd wget http://10.109.194.90:80/ -O - -T 5 -t 1 passed 2026-02-10 04:05:16,820 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd output: ctest-pod-94493639 Connecting to 10.109.194.90:80 (10.109.194.90:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-02-10 04:05:16,820 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-32123737': 1, 'ctest-pod-94493639': 1} 2026-02-10 04:05:16,886 - DEBUG - [Pod ctest-pod-66196454] Cmd: wget http://10.109.194.90:80/ -O - -T 5 -t 1 , Output: ctest-pod-94493639 Connecting to 10.109.194.90:80 (10.109.194.90:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-02-10 04:05:16,886 - DEBUG - [Pod Pod ctest-pod-66196454] Cmd wget http://10.109.194.90:80/ -O - -T 5 -t 1 passed 2026-02-10 04:05:16,887 - DEBUG - [Pod Pod ctest-pod-66196454] Cmd output: ctest-pod-94493639 Connecting to 10.109.194.90:80 (10.109.194.90:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-02-10 04:05:16,958 - DEBUG - [Pod ctest-pod-66196454] Cmd: wget http://10.109.194.90:80/ -O - -T 5 -t 1 , Output: ctest-pod-32123737 Connecting to 10.109.194.90:80 (10.109.194.90:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-02-10 04:05:16,958 - DEBUG - [Pod Pod ctest-pod-66196454] Cmd wget http://10.109.194.90:80/ -O - -T 5 -t 1 passed 2026-02-10 04:05:16,958 - DEBUG - [Pod Pod ctest-pod-66196454] Cmd output: ctest-pod-32123737 Connecting to 10.109.194.90:80 (10.109.194.90:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-02-10 04:05:16,958 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-32123737': 1, 'ctest-pod-94493639': 1} 2026-02-10 04:05:16,962 - DEBUG - Network policy ingress-on-service not present 2026-02-10 04:05:16,962 - INFO - Creating Network Policy ingress-on-service 2026-02-10 04:05:16,979 - INFO - Network policy found in k8s 2026-02-10 04:05:16,979 - DEBUG - Requesting: http://10.0.0.242:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=9841e52f-838e-45c1-b409-94e73b78bcbb 2026-02-10 04:05:16,986 - INFO - Network Policy ingress-on-service with uuid 9841e52f-838e-45c1-b409-94e73b78bcbb found in kube manager 2026-02-10 04:05:17,016 - INFO - Network policy with name default-policy-management:k8s-allowall found in agent 2026-02-10 04:05:17,017 - INFO - Network policy with name default-policy-management:k8s-allowall associated with default ks8 2026-02-10 04:05:17,017 - DEBUG - Requesting: http://10.0.0.242:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=9841e52f-838e-45c1-b409-94e73b78bcbb 2026-02-10 04:05:17,048 - WARNING - Network policy with name ingress-on-service not found in agent 2026-02-10 04:05:19,049 - DEBUG - Requesting: http://10.0.0.242:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=9841e52f-838e-45c1-b409-94e73b78bcbb 2026-02-10 04:05:19,083 - INFO - Network policy with name ingress-on-service found in agent 2026-02-10 04:05:19,083 - INFO - Network Policy ingress-on-service verification passed 2026-02-10 04:05:24,147 - DEBUG - [Pod ctest-pod-57887898] Cmd: wget http://10.109.194.90:80/ -O - -T 5 -t 1 , Output: Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:05:24,147 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd wget http://10.109.194.90:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:05:24,147 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd output: Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:05:29,203 - DEBUG - [Pod ctest-pod-57887898] Cmd: wget http://10.109.194.90:80/ -O - -T 5 -t 1 , Output: Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:05:29,203 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd wget http://10.109.194.90:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:05:29,203 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd output: Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:05:34,262 - DEBUG - [Pod ctest-pod-57887898] Cmd: wget http://10.109.194.90:80/ -O - -T 5 -t 1 , Output: Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:05:34,262 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd wget http://10.109.194.90:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:05:34,262 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd output: Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:05:39,323 - DEBUG - [Pod ctest-pod-57887898] Cmd: wget http://10.109.194.90:80/ -O - -T 5 -t 1 , Output: Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:05:39,323 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd wget http://10.109.194.90:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:05:39,323 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd output: Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:05:44,384 - DEBUG - [Pod ctest-pod-57887898] Cmd: wget http://10.109.194.90:80/ -O - -T 5 -t 1 , Output: Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:05:44,384 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd wget http://10.109.194.90:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:05:44,384 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd output: Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:05:49,434 - DEBUG - [Pod ctest-pod-57887898] Cmd: wget http://10.109.194.90:80/ -O - -T 5 -t 1 , Output: Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:05:49,435 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd wget http://10.109.194.90:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:05:49,435 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd output: Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:05:54,503 - DEBUG - [Pod ctest-pod-57887898] Cmd: wget http://10.109.194.90:80/ -O - -T 5 -t 1 , Output: Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:05:54,503 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd wget http://10.109.194.90:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:05:54,504 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd output: Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:05:59,558 - DEBUG - [Pod ctest-pod-57887898] Cmd: wget http://10.109.194.90:80/ -O - -T 5 -t 1 , Output: Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:05:59,559 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd wget http://10.109.194.90:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:05:59,559 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd output: Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:06:04,617 - DEBUG - [Pod ctest-pod-57887898] Cmd: wget http://10.109.194.90:80/ -O - -T 5 -t 1 , Output: Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:06:04,618 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd wget http://10.109.194.90:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:06:04,618 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd output: Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:06:09,678 - DEBUG - [Pod ctest-pod-57887898] Cmd: wget http://10.109.194.90:80/ -O - -T 5 -t 1 , Output: Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:06:09,678 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd wget http://10.109.194.90:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:06:09,678 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd output: Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:06:09,678 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-32123737': 0, 'ctest-pod-94493639': 0} 2026-02-10 04:06:09,748 - DEBUG - [Pod ctest-pod-58728610] Cmd: wget http://10.109.194.90:80/ -O - -T 5 -t 1 , Output: ctest-pod-32123737 Connecting to 10.109.194.90:80 (10.109.194.90:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-02-10 04:06:09,748 - DEBUG - [Pod Pod ctest-pod-58728610] Cmd wget http://10.109.194.90:80/ -O - -T 5 -t 1 passed 2026-02-10 04:06:09,748 - DEBUG - [Pod Pod ctest-pod-58728610] Cmd output: ctest-pod-32123737 Connecting to 10.109.194.90:80 (10.109.194.90:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-02-10 04:06:09,812 - DEBUG - [Pod ctest-pod-58728610] Cmd: wget http://10.109.194.90:80/ -O - -T 5 -t 1 , Output: ctest-pod-32123737 Connecting to 10.109.194.90:80 (10.109.194.90:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-02-10 04:06:09,812 - DEBUG - [Pod Pod ctest-pod-58728610] Cmd wget http://10.109.194.90:80/ -O - -T 5 -t 1 passed 2026-02-10 04:06:09,812 - DEBUG - [Pod Pod ctest-pod-58728610] Cmd output: ctest-pod-32123737 Connecting to 10.109.194.90:80 (10.109.194.90:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-02-10 04:06:09,871 - DEBUG - [Pod ctest-pod-58728610] Cmd: wget http://10.109.194.90:80/ -O - -T 5 -t 1 , Output: ctest-pod-32123737 Connecting to 10.109.194.90:80 (10.109.194.90:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-02-10 04:06:09,871 - DEBUG - [Pod Pod ctest-pod-58728610] Cmd wget http://10.109.194.90:80/ -O - -T 5 -t 1 passed 2026-02-10 04:06:09,871 - DEBUG - [Pod Pod ctest-pod-58728610] Cmd output: ctest-pod-32123737 Connecting to 10.109.194.90:80 (10.109.194.90:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-02-10 04:06:09,937 - DEBUG - [Pod ctest-pod-58728610] Cmd: wget http://10.109.194.90:80/ -O - -T 5 -t 1 , Output: ctest-pod-94493639 Connecting to 10.109.194.90:80 (10.109.194.90:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-02-10 04:06:09,938 - DEBUG - [Pod Pod ctest-pod-58728610] Cmd wget http://10.109.194.90:80/ -O - -T 5 -t 1 passed 2026-02-10 04:06:09,938 - DEBUG - [Pod Pod ctest-pod-58728610] Cmd output: ctest-pod-94493639 Connecting to 10.109.194.90:80 (10.109.194.90:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2026-02-10 04:06:09,938 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-32123737': 3, 'ctest-pod-94493639': 1} 2026-02-10 04:06:15,002 - DEBUG - [Pod ctest-pod-66196454] Cmd: wget http://10.109.194.90:80/ -O - -T 5 -t 1 , Output: Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:06:15,002 - DEBUG - [Pod Pod ctest-pod-66196454] Cmd wget http://10.109.194.90:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:06:15,002 - DEBUG - [Pod Pod ctest-pod-66196454] Cmd output: Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:06:20,055 - DEBUG - [Pod ctest-pod-66196454] Cmd: wget http://10.109.194.90:80/ -O - -T 5 -t 1 , Output: Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:06:20,055 - DEBUG - [Pod Pod ctest-pod-66196454] Cmd wget http://10.109.194.90:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:06:20,055 - DEBUG - [Pod Pod ctest-pod-66196454] Cmd output: Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:06:25,110 - DEBUG - [Pod ctest-pod-66196454] Cmd: wget http://10.109.194.90:80/ -O - -T 5 -t 1 , Output: Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:06:25,110 - DEBUG - [Pod Pod ctest-pod-66196454] Cmd wget http://10.109.194.90:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:06:25,110 - DEBUG - [Pod Pod ctest-pod-66196454] Cmd output: Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:06:30,166 - DEBUG - [Pod ctest-pod-66196454] Cmd: wget http://10.109.194.90:80/ -O - -T 5 -t 1 , Output: Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:06:30,166 - DEBUG - [Pod Pod ctest-pod-66196454] Cmd wget http://10.109.194.90:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:06:30,166 - DEBUG - [Pod Pod ctest-pod-66196454] Cmd output: Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:06:35,222 - DEBUG - [Pod ctest-pod-66196454] Cmd: wget http://10.109.194.90:80/ -O - -T 5 -t 1 , Output: Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:06:35,222 - DEBUG - [Pod Pod ctest-pod-66196454] Cmd wget http://10.109.194.90:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:06:35,222 - DEBUG - [Pod Pod ctest-pod-66196454] Cmd output: Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:06:40,281 - DEBUG - [Pod ctest-pod-66196454] Cmd: wget http://10.109.194.90:80/ -O - -T 5 -t 1 , Output: Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:06:40,282 - DEBUG - [Pod Pod ctest-pod-66196454] Cmd wget http://10.109.194.90:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:06:40,282 - DEBUG - [Pod Pod ctest-pod-66196454] Cmd output: Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:06:45,340 - DEBUG - [Pod ctest-pod-66196454] Cmd: wget http://10.109.194.90:80/ -O - -T 5 -t 1 , Output: Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:06:45,340 - DEBUG - [Pod Pod ctest-pod-66196454] Cmd wget http://10.109.194.90:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:06:45,340 - DEBUG - [Pod Pod ctest-pod-66196454] Cmd output: Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:06:50,403 - DEBUG - [Pod ctest-pod-66196454] Cmd: wget http://10.109.194.90:80/ -O - -T 5 -t 1 , Output: Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:06:50,403 - DEBUG - [Pod Pod ctest-pod-66196454] Cmd wget http://10.109.194.90:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:06:50,404 - DEBUG - [Pod Pod ctest-pod-66196454] Cmd output: Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:06:55,467 - DEBUG - [Pod ctest-pod-66196454] Cmd: wget http://10.109.194.90:80/ -O - -T 5 -t 1 , Output: Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:06:55,467 - DEBUG - [Pod Pod ctest-pod-66196454] Cmd wget http://10.109.194.90:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:06:55,467 - DEBUG - [Pod Pod ctest-pod-66196454] Cmd output: Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:07:00,523 - DEBUG - [Pod ctest-pod-66196454] Cmd: wget http://10.109.194.90:80/ -O - -T 5 -t 1 , Output: Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:07:00,523 - DEBUG - [Pod Pod ctest-pod-66196454] Cmd wget http://10.109.194.90:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:07:00,523 - DEBUG - [Pod Pod ctest-pod-66196454] Cmd output: Connecting to 10.109.194.90:80 (10.109.194.90:80) wget: download timed out 2026-02-10 04:07:00,523 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-32123737': 0, 'ctest-pod-94493639': 0} 2026-02-10 04:07:02,583 - DEBUG - [Pod ctest-pod-58728610] Cmd: ping -c 3 10.47.255.247, Output: PING 10.47.255.247 (10.47.255.247): 56 data bytes 64 bytes from 10.47.255.247: seq=0 ttl=63 time=1.639 ms 64 bytes from 10.47.255.247: seq=1 ttl=63 time=0.082 ms 64 bytes from 10.47.255.247: seq=2 ttl=63 time=0.075 ms --- 10.47.255.247 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.075/0.598/1.639 ms 2026-02-10 04:07:02,583 - INFO - Ping check to IP 10.47.255.247 from pod ctest-pod-58728610 with expectation True passed 2026-02-10 04:07:04,647 - DEBUG - [Pod ctest-pod-57887898] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=0.877 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.080 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.080 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.080/0.345/0.877 ms 2026-02-10 04:07:04,648 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-57887898 with expectation True passed 2026-02-10 04:07:06,704 - DEBUG - [Pod ctest-pod-66196454] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=0.987 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.078 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.107 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.078/0.390/0.987 ms 2026-02-10 04:07:06,705 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-66196454 with expectation True passed 2026-02-10 04:07:06,770 - DEBUG - [Pod ctest-pod-58728610] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-10 04:07:06,771 - DEBUG - [Pod Pod ctest-pod-58728610] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2026-02-10 04:07:06,771 - DEBUG - [Pod Pod ctest-pod-58728610] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-10 04:07:06,771 - INFO - wget check of of http://10.47.255.251 from ctest-pod-58728610 passed 2026-02-10 04:07:06,842 - DEBUG - [Pod ctest-pod-58728610] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-10 04:07:06,842 - DEBUG - [Pod Pod ctest-pod-58728610] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2026-02-10 04:07:06,842 - DEBUG - [Pod Pod ctest-pod-58728610] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-10 04:07:06,842 - INFO - wget check of of http://10.47.255.250 from ctest-pod-58728610 passed 2026-02-10 04:07:06,907 - DEBUG - [Pod ctest-pod-57887898] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-10 04:07:06,907 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2026-02-10 04:07:06,907 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-10 04:07:06,907 - WARNING - wget check of of http://10.47.255.250 from ctest-pod-57887898 failed 2026-02-10 04:07:08,972 - DEBUG - [Pod ctest-pod-57887898] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-10 04:07:08,972 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2026-02-10 04:07:08,972 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-10 04:07:08,972 - WARNING - wget check of of http://10.47.255.250 from ctest-pod-57887898 failed 2026-02-10 04:07:11,050 - DEBUG - [Pod ctest-pod-57887898] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-10 04:07:11,050 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2026-02-10 04:07:11,050 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-10 04:07:11,050 - WARNING - wget check of of http://10.47.255.250 from ctest-pod-57887898 failed 2026-02-10 04:07:13,107 - DEBUG - [Pod ctest-pod-57887898] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-10 04:07:13,107 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2026-02-10 04:07:13,107 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-10 04:07:13,107 - WARNING - wget check of of http://10.47.255.250 from ctest-pod-57887898 failed 2026-02-10 04:07:15,171 - DEBUG - [Pod ctest-pod-57887898] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-10 04:07:15,171 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2026-02-10 04:07:15,171 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-10 04:07:15,171 - WARNING - wget check of of http://10.47.255.250 from ctest-pod-57887898 failed 2026-02-10 04:07:17,235 - DEBUG - [Pod ctest-pod-57887898] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-10 04:07:17,235 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2026-02-10 04:07:17,236 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-10 04:07:17,236 - WARNING - wget check of of http://10.47.255.250 from ctest-pod-57887898 failed 2026-02-10 04:07:19,297 - DEBUG - [Pod ctest-pod-57887898] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-10 04:07:19,297 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2026-02-10 04:07:19,297 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-10 04:07:19,297 - WARNING - wget check of of http://10.47.255.250 from ctest-pod-57887898 failed 2026-02-10 04:07:21,367 - DEBUG - [Pod ctest-pod-57887898] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-10 04:07:21,367 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2026-02-10 04:07:21,367 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-10 04:07:21,367 - WARNING - wget check of of http://10.47.255.250 from ctest-pod-57887898 failed 2026-02-10 04:07:23,424 - DEBUG - [Pod ctest-pod-57887898] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-10 04:07:23,424 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2026-02-10 04:07:23,424 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-10 04:07:23,424 - WARNING - wget check of of http://10.47.255.250 from ctest-pod-57887898 failed 2026-02-10 04:07:25,486 - DEBUG - [Pod ctest-pod-57887898] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-10 04:07:25,486 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2026-02-10 04:07:25,486 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-10 04:07:25,487 - WARNING - wget check of of http://10.47.255.250 from ctest-pod-57887898 failed 2026-02-10 04:07:27,561 - DEBUG - [Pod ctest-pod-57887898] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-10 04:07:27,561 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2026-02-10 04:07:27,561 - DEBUG - [Pod Pod ctest-pod-57887898] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2026-02-10 04:07:27,561 - WARNING - wget check of of http://10.47.255.250 from ctest-pod-57887898 failed 2026-02-10 04:07:27,567 - INFO - Deleting Network Policy : ingress-on-service 2026-02-10 04:07:27,568 - INFO - Deleting service : ctest-nginx-svc-32827184 2026-02-10 04:07:27,808 - ERROR - AssertionError Python 3.9.21: /usr/bin/python3 Tue Feb 10 04:07:27 2026 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={}) 78 log.info('Initial checks done. Running the testcase now') 79 log.info('') 80 result = function(self, *args, **kwargs) 81 if self.inputs.upgrade: 82 pid = os.getpid() result = None function = self = args = () kwargs = {} /contrail-test/scripts/k8s_scripts/test_policy.py in test_ingress_rule_on_pod_with_service(self=) 2859 assert self.validate_wget(self.client2_pod_ns1, url1) 2860 assert self.validate_wget(self.client2_pod_ns1, url2) 2861 assert self.validate_wget(self.client1_pod_ns1, url2, expectation=False) 2862 assert self.validate_wget(self.client2_pod_ns2, url1, expectation=False) 2863 # Updating the policy and adding the namespace selector self = self.validate_wget = .deco_retry....ice[ci_k8s_sanity,k8s_sanity] id=0x7f73f0d48af0>> self.client1_pod_ns1 = url2 = 'http://10.47.255.250' expectation undefined AssertionError: __cause__ = None __class__ = __context__ = None __delattr__ = __dict__ = {} __dir__ = __doc__ = 'Assertion failed.' __eq__ = __format__ = __ge__ = __getattribute__ = __gt__ = __hash__ = __init__ = __init_subclass__ = __le__ = __lt__ = __ne__ = __new__ = __reduce__ = __reduce_ex__ = __repr__ = __setattr__ = __setstate__ = __sizeof__ = __str__ = __subclasshook__ = __suppress_context__ = False __traceback__ = args = () with_traceback = The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper result = function(self, *args, **kwargs) File "/contrail-test/scripts/k8s_scripts/test_policy.py", line 2861, in test_ingress_rule_on_pod_with_service assert self.validate_wget(self.client1_pod_ns1, url2, expectation=False) AssertionError 2026-02-10 04:07:27,817 - DEBUG - No XMPP flaps were noticed during the test 2026-02-10 04:07:27,817 - INFO -
2026-02-10 04:07:27,817 - INFO - END TEST : test_ingress_rule_on_pod_with_service : FAILED[0:02:13]
2026-02-10 04:07:27,817 - INFO - -------------------------------------------------------------------------------- 2026-02-10 04:07:27,823 - INFO - Deleting pod ctest-new-default-83327604:ctest-pod-32123737 2026-02-10 04:07:27,823 - INFO - Deleting pod ctest-new-default-83327604:ctest-pod-94493639 2026-02-10 04:07:27,824 - INFO - Deleting pod ctest-non-default-33960794:ctest-pod-30974977 2026-02-10 04:07:27,824 - INFO - Deleting pod ctest-non-default-33960794:ctest-pod-25237338 2026-02-10 04:07:27,825 - INFO - Deleting pod ctest-new-default-83327604:ctest-pod-57887898 2026-02-10 04:07:27,825 - INFO - Deleting pod ctest-new-default-83327604:ctest-pod-58728610 2026-02-10 04:07:27,825 - INFO - Deleting pod ctest-non-default-33960794:ctest-pod-66196454 2026-02-10 04:07:27,826 - INFO - Deleting pod ctest-non-default-33960794:ctest-pod-85178032 2026-02-10 04:07:28,047 - WARNING - Pod uuid f4fe607b-529f-4c65-b4af-e3502eb59623 is still seen in agent 10.0.0.23 VM list 2026-02-10 04:07:28,052 - WARNING - Pod uuid e3987d54-1bd3-4c18-8b66-3968afad7a6b is still seen in agent 10.0.0.23 VM list 2026-02-10 04:07:28,053 - WARNING - Pod uuid 109b7879-1a6d-4836-b84b-28394237a805 is still seen in agent 10.0.0.23 VM list 2026-02-10 04:07:28,062 - WARNING - Pod uuid 8f1448a2-8f7a-49d2-a015-234f5a5ffe53 is still seen in agent 10.0.0.23 VM list 2026-02-10 04:07:28,064 - WARNING - Pod uuid cb1caaa1-7663-4685-a32c-ef842ba3b4ea is still seen in agent 10.0.0.23 VM list 2026-02-10 04:07:28,064 - WARNING - Pod uuid bc6402d5-235d-4c3e-a45e-1f7a24e2fbe0 is still seen in agent 10.0.0.23 VM list 2026-02-10 04:07:28,072 - WARNING - Pod uuid d2d23d92-231d-41c4-bb97-437a795dc1fc is still seen in agent 10.0.0.23 VM list 2026-02-10 04:07:28,072 - WARNING - Pod uuid 5ad221b4-abb7-4f4e-9bfe-12a3eeb0cad1 is still seen in agent 10.0.0.23 VM list 2026-02-10 04:07:33,061 - WARNING - Pod uuid f4fe607b-529f-4c65-b4af-e3502eb59623 is still seen in agent 10.0.0.23 VM list 2026-02-10 04:07:33,077 - WARNING - Pod uuid 109b7879-1a6d-4836-b84b-28394237a805 is still seen in agent 10.0.0.23 VM list 2026-02-10 04:07:33,078 - WARNING - Pod uuid e3987d54-1bd3-4c18-8b66-3968afad7a6b is still seen in agent 10.0.0.23 VM list 2026-02-10 04:07:33,096 - DEBUG - Pod cb1caaa1-7663-4685-a32c-ef842ba3b4ea is not in agent 10.0.0.23 VM list 2026-02-10 04:07:33,098 - DEBUG - Pod bc6402d5-235d-4c3e-a45e-1f7a24e2fbe0 is not in agent 10.0.0.23 VM list 2026-02-10 04:07:33,099 - DEBUG - Pod 8f1448a2-8f7a-49d2-a015-234f5a5ffe53 is not in agent 10.0.0.23 VM list 2026-02-10 04:07:33,102 - DEBUG - Pod 5ad221b4-abb7-4f4e-9bfe-12a3eeb0cad1 is not in agent 10.0.0.23 VM list 2026-02-10 04:07:33,104 - WARNING - Pod uuid d2d23d92-231d-41c4-bb97-437a795dc1fc is still seen in agent 10.0.0.23 VM list 2026-02-10 04:07:33,115 - DEBUG - VMI a844b642-0635-11f1-8b30-fa163e9732a7 is removed from agent 10.0.0.23 2026-02-10 04:07:33,115 - INFO - Verified that pod ctest-pod-94493639 is removed in agent 2026-02-10 04:07:33,118 - DEBUG - VMI a7cd184e-0635-11f1-8b30-fa163e9732a7 is removed from agent 10.0.0.23 2026-02-10 04:07:33,118 - INFO - Verified that pod ctest-pod-32123737 is removed in agent 2026-02-10 04:07:33,119 - DEBUG - VMI a8ec5a82-0635-11f1-8b30-fa163e9732a7 is removed from agent 10.0.0.23 2026-02-10 04:07:33,119 - INFO - Verified that pod ctest-pod-30974977 is removed in agent 2026-02-10 04:07:33,119 - DEBUG - VMI a96df4ac-0635-11f1-8b30-fa163e9732a7 is removed from agent 10.0.0.23 2026-02-10 04:07:33,119 - INFO - Verified that pod ctest-pod-25237338 is removed in agent 2026-02-10 04:07:38,071 - WARNING - Pod uuid f4fe607b-529f-4c65-b4af-e3502eb59623 is still seen in agent 10.0.0.23 VM list 2026-02-10 04:07:38,087 - WARNING - Pod uuid e3987d54-1bd3-4c18-8b66-3968afad7a6b is still seen in agent 10.0.0.23 VM list 2026-02-10 04:07:38,088 - WARNING - Pod uuid 109b7879-1a6d-4836-b84b-28394237a805 is still seen in agent 10.0.0.23 VM list 2026-02-10 04:07:38,114 - WARNING - Pod uuid d2d23d92-231d-41c4-bb97-437a795dc1fc is still seen in agent 10.0.0.23 VM list 2026-02-10 04:07:43,080 - WARNING - Pod uuid f4fe607b-529f-4c65-b4af-e3502eb59623 is still seen in agent 10.0.0.23 VM list 2026-02-10 04:07:43,100 - WARNING - Pod uuid e3987d54-1bd3-4c18-8b66-3968afad7a6b is still seen in agent 10.0.0.23 VM list 2026-02-10 04:07:43,101 - WARNING - Pod uuid 109b7879-1a6d-4836-b84b-28394237a805 is still seen in agent 10.0.0.23 VM list 2026-02-10 04:07:43,123 - WARNING - Pod uuid d2d23d92-231d-41c4-bb97-437a795dc1fc is still seen in agent 10.0.0.23 VM list 2026-02-10 04:07:48,089 - WARNING - Pod uuid f4fe607b-529f-4c65-b4af-e3502eb59623 is still seen in agent 10.0.0.23 VM list 2026-02-10 04:07:48,111 - WARNING - Pod uuid e3987d54-1bd3-4c18-8b66-3968afad7a6b is still seen in agent 10.0.0.23 VM list 2026-02-10 04:07:48,112 - WARNING - Pod uuid 109b7879-1a6d-4836-b84b-28394237a805 is still seen in agent 10.0.0.23 VM list 2026-02-10 04:07:48,132 - WARNING - Pod uuid d2d23d92-231d-41c4-bb97-437a795dc1fc is still seen in agent 10.0.0.23 VM list 2026-02-10 04:07:53,097 - WARNING - Pod uuid f4fe607b-529f-4c65-b4af-e3502eb59623 is still seen in agent 10.0.0.23 VM list 2026-02-10 04:07:53,124 - WARNING - Pod uuid 109b7879-1a6d-4836-b84b-28394237a805 is still seen in agent 10.0.0.23 VM list 2026-02-10 04:07:53,125 - WARNING - Pod uuid e3987d54-1bd3-4c18-8b66-3968afad7a6b is still seen in agent 10.0.0.23 VM list 2026-02-10 04:07:53,140 - WARNING - Pod uuid d2d23d92-231d-41c4-bb97-437a795dc1fc is still seen in agent 10.0.0.23 VM list 2026-02-10 04:07:58,172 - WARNING - Pod uuid e3987d54-1bd3-4c18-8b66-3968afad7a6b is still seen in agent 10.0.0.23 VM list 2026-02-10 04:07:58,173 - WARNING - Pod uuid f4fe607b-529f-4c65-b4af-e3502eb59623 is still seen in agent 10.0.0.23 VM list 2026-02-10 04:07:58,175 - WARNING - Pod uuid d2d23d92-231d-41c4-bb97-437a795dc1fc is still seen in agent 10.0.0.23 VM list 2026-02-10 04:07:58,175 - WARNING - Pod uuid 109b7879-1a6d-4836-b84b-28394237a805 is still seen in agent 10.0.0.23 VM list 2026-02-10 04:08:03,191 - DEBUG - Pod e3987d54-1bd3-4c18-8b66-3968afad7a6b is not in agent 10.0.0.23 VM list 2026-02-10 04:08:03,194 - DEBUG - Pod 109b7879-1a6d-4836-b84b-28394237a805 is not in agent 10.0.0.23 VM list 2026-02-10 04:08:03,195 - DEBUG - Pod f4fe607b-529f-4c65-b4af-e3502eb59623 is not in agent 10.0.0.23 VM list 2026-02-10 04:08:03,196 - DEBUG - Pod d2d23d92-231d-41c4-bb97-437a795dc1fc is not in agent 10.0.0.23 VM list 2026-02-10 04:08:03,209 - DEBUG - VMI a9e74280-0635-11f1-8b30-fa163e9732a7 is removed from agent 10.0.0.23 2026-02-10 04:08:03,210 - INFO - Verified that pod ctest-pod-57887898 is removed in agent 2026-02-10 04:08:03,210 - DEBUG - VMI aa74e6da-0635-11f1-8b30-fa163e9732a7 is removed from agent 10.0.0.23 2026-02-10 04:08:03,210 - INFO - Verified that pod ctest-pod-58728610 is removed in agent 2026-02-10 04:08:03,212 - DEBUG - VMI ab7235ba-0635-11f1-8b30-fa163e9732a7 is removed from agent 10.0.0.23 2026-02-10 04:08:03,212 - INFO - Verified that pod ctest-pod-85178032 is removed in agent 2026-02-10 04:08:03,213 - DEBUG - VMI aaf2b2c2-0635-11f1-8b30-fa163e9732a7 is removed from agent 10.0.0.23 2026-02-10 04:08:03,213 - INFO - Verified that pod ctest-pod-66196454 is removed in agent 2026-02-10 04:08:03,213 - INFO - Deleting namespace ctest-non-default-33960794 2026-02-10 04:08:03,232 - DEBUG - Namespace ctest-non-default-33960794 still in k8s 2026-02-10 04:08:05,239 - DEBUG - Namespace ctest-non-default-33960794 still in k8s 2026-02-10 04:08:07,245 - DEBUG - Namespace ctest-non-default-33960794 still in k8s 2026-02-10 04:08:09,250 - DEBUG - Namespace ctest-non-default-33960794 is not in k8s 2026-02-10 04:08:09,250 - DEBUG - No need to check project deletion in contrail-api 2026-02-10 04:08:09,250 - INFO - Deleting namespace ctest-new-default-83327604 2026-02-10 04:08:09,259 - DEBUG - Namespace ctest-new-default-83327604 still in k8s 2026-02-10 04:08:11,266 - DEBUG - Namespace ctest-new-default-83327604 still in k8s 2026-02-10 04:08:13,272 - DEBUG - Namespace ctest-new-default-83327604 still in k8s 2026-02-10 04:08:15,277 - DEBUG - Namespace ctest-new-default-83327604 is not in k8s 2026-02-10 04:08:15,277 - DEBUG - No need to check project deletion in contrail-api