2024-09-18 05:29:05,653 - INFO - Using existing project ['default-domain', 'k8s-default'](b308abee-2524-40b0-914c-f389782945c8) 2024-09-18 05:29:05,679 - DEBUG - Namespace ctest-new-default-03627883 not present 2024-09-18 05:29:05,696 - INFO - Created namespace ctest-new-default-03627883 2024-09-18 05:29:08,701 - DEBUG - Namespace ctest-non-default-77035363 not present 2024-09-18 05:29:08,722 - INFO - Created namespace ctest-non-default-77035363 2024-09-18 05:29:11,749 - DEBUG - Pod ctest-pod-40801591 not present 2024-09-18 05:29:11,750 - INFO - Creating Pod ctest-pod-40801591 2024-09-18 05:29:11,781 - DEBUG - Pod : ctest-pod-40801591 UUID is 2b2a5d8b-814b-48e6-96fd-042ccb881686 2024-09-18 05:29:11,784 - DEBUG - Pod ctest-pod-90225391 not present 2024-09-18 05:29:11,784 - INFO - Creating Pod ctest-pod-90225391 2024-09-18 05:29:11,825 - DEBUG - Pod : ctest-pod-90225391 UUID is 704a6f5e-f6b0-4fbb-9d51-1c48ffc5b550 2024-09-18 05:29:11,829 - DEBUG - Pod ctest-pod-33895030 not present 2024-09-18 05:29:11,830 - INFO - Creating Pod ctest-pod-33895030 2024-09-18 05:29:11,893 - DEBUG - Pod : ctest-pod-33895030 UUID is 06c593fd-aaac-4cd7-a40d-cda801d30baf 2024-09-18 05:29:11,897 - DEBUG - Pod ctest-pod-14670403 not present 2024-09-18 05:29:11,898 - INFO - Creating Pod ctest-pod-14670403 2024-09-18 05:29:11,933 - DEBUG - Pod : ctest-pod-14670403 UUID is c57aa02b-d1da-41f3-a610-03b5f8af58d9 2024-09-18 05:29:11,936 - DEBUG - Pod ctest-pod-07271080 not present 2024-09-18 05:29:11,937 - INFO - Creating Pod ctest-pod-07271080 2024-09-18 05:29:11,962 - DEBUG - Pod : ctest-pod-07271080 UUID is 28091988-3ff1-480b-957b-e5266bd9bef0 2024-09-18 05:29:11,969 - DEBUG - Pod ctest-pod-25764333 not present 2024-09-18 05:29:11,970 - INFO - Creating Pod ctest-pod-25764333 2024-09-18 05:29:12,000 - DEBUG - Pod : ctest-pod-25764333 UUID is ebf1bad8-947d-4d3c-93ba-e831b6182b11 2024-09-18 05:29:12,003 - DEBUG - Pod ctest-pod-65709397 not present 2024-09-18 05:29:12,004 - INFO - Creating Pod ctest-pod-65709397 2024-09-18 05:29:12,046 - DEBUG - Pod : ctest-pod-65709397 UUID is 518b0655-fc4d-46fb-aa92-5a9df7d93830 2024-09-18 05:29:12,055 - DEBUG - Pod ctest-pod-70339917 not present 2024-09-18 05:29:12,055 - INFO - Creating Pod ctest-pod-70339917 2024-09-18 05:29:12,089 - DEBUG - Pod : ctest-pod-70339917 UUID is ce2460b8-dfe3-46f8-a310-eff8b08674b0 2024-09-18 05:29:12,280 - INFO - Project default-domain:k8s-ctest-new-default-03627883 for Namespace ctest-new-default-03627883 is seen in contrail-api 2024-09-18 05:29:12,280 - DEBUG - Project uuid in Contrail is 41bd8455-e1ba-48b0-8567-df0ec78f4a47 2024-09-18 05:29:12,449 - DEBUG - Requesting: http://10.0.0.42:8108/Snh_NamespaceDatabaseList?namespace_uuid=efab5e28-3d50-442e-bed4-b3f94e358fe8 2024-09-18 05:29:12,464 - INFO - Namespace ctest-new-default-03627883 with uuid efab5e28-3d50-442e-bed4-b3f94e358fe8 found in kube manager 2024-09-18 05:29:12,464 - INFO - Namespace ctest-new-default-03627883 verification passed 2024-09-18 05:29:12,698 - INFO - Project default-domain:k8s-ctest-non-default-77035363 for Namespace ctest-non-default-77035363 is seen in contrail-api 2024-09-18 05:29:12,698 - DEBUG - Project uuid in Contrail is 375447eb-ca84-4035-9610-0b856c3a5cdc 2024-09-18 05:29:12,698 - DEBUG - Requesting: http://10.0.0.42:8108/Snh_NamespaceDatabaseList?namespace_uuid=2e84c72e-c4bc-4d07-bfd0-82b1e3407b58 2024-09-18 05:29:12,702 - INFO - Namespace ctest-non-default-77035363 with uuid 2e84c72e-c4bc-4d07-bfd0-82b1e3407b58 found in kube manager 2024-09-18 05:29:12,703 - INFO - Namespace ctest-non-default-77035363 verification passed 2024-09-18 05:29:12,708 - DEBUG - Pod ctest-pod-40801591 not in running state.Currently in Pending 2024-09-18 05:29:17,714 - DEBUG - Pod ctest-pod-40801591 not in running state.Currently in Pending 2024-09-18 05:29:22,723 - DEBUG - Pod ctest-pod-40801591 not in running state.Currently in Pending 2024-09-18 05:29:27,732 - INFO - Pod ctest-pod-40801591 is in running state.Got IP 10.47.255.251 2024-09-18 05:29:27,797 - DEBUG - Pod ctest-pod-40801591 has vmi f0d7cfd2-757e-11ef-9f1a-fa163ef07116 2024-09-18 05:29:27,797 - INFO - Verified pod ctest-pod-40801591 in contrail-api 2024-09-18 05:29:27,815 - DEBUG - VMI f0d7cfd2-757e-11ef-9f1a-fa163ef07116 is active in agent 10.0.0.33 2024-09-18 05:29:27,815 - INFO - Verified Pod ctest-pod-40801591 in agent 10.0.0.33 2024-09-18 05:29:27,816 - DEBUG - Requesting: http://10.0.0.42:8108/Snh_VirtualMachineDatabaseList?x=2b2a5d8b-814b-48e6-96fd-042ccb881686 2024-09-18 05:29:27,821 - INFO - Pod ctest-pod-40801591 with uuid 2b2a5d8b-814b-48e6-96fd-042ccb881686 found in kube manager 2024-09-18 05:29:27,821 - INFO - Pod ctest-pod-40801591 verification passed 2024-09-18 05:29:27,827 - INFO - Pod ctest-pod-90225391 is in running state.Got IP 10.47.255.250 2024-09-18 05:29:27,851 - DEBUG - Pod ctest-pod-90225391 has vmi f16136f0-757e-11ef-9f1a-fa163ef07116 2024-09-18 05:29:27,852 - INFO - Verified pod ctest-pod-90225391 in contrail-api 2024-09-18 05:29:27,865 - DEBUG - VMI f16136f0-757e-11ef-9f1a-fa163ef07116 is active in agent 10.0.0.33 2024-09-18 05:29:27,866 - INFO - Verified Pod ctest-pod-90225391 in agent 10.0.0.33 2024-09-18 05:29:27,866 - DEBUG - Requesting: http://10.0.0.42:8108/Snh_VirtualMachineDatabaseList?x=704a6f5e-f6b0-4fbb-9d51-1c48ffc5b550 2024-09-18 05:29:27,871 - INFO - Pod ctest-pod-90225391 with uuid 704a6f5e-f6b0-4fbb-9d51-1c48ffc5b550 found in kube manager 2024-09-18 05:29:27,871 - INFO - Pod ctest-pod-90225391 verification passed 2024-09-18 05:29:27,877 - INFO - Pod ctest-pod-33895030 is in running state.Got IP 10.47.255.249 2024-09-18 05:29:27,938 - DEBUG - Pod ctest-pod-33895030 has vmi f225e39c-757e-11ef-9f1a-fa163ef07116 2024-09-18 05:29:27,938 - INFO - Verified pod ctest-pod-33895030 in contrail-api 2024-09-18 05:29:27,954 - DEBUG - VMI f225e39c-757e-11ef-9f1a-fa163ef07116 is active in agent 10.0.0.33 2024-09-18 05:29:27,954 - INFO - Verified Pod ctest-pod-33895030 in agent 10.0.0.33 2024-09-18 05:29:27,955 - DEBUG - Requesting: http://10.0.0.42:8108/Snh_VirtualMachineDatabaseList?x=06c593fd-aaac-4cd7-a40d-cda801d30baf 2024-09-18 05:29:27,963 - INFO - Pod ctest-pod-33895030 with uuid 06c593fd-aaac-4cd7-a40d-cda801d30baf found in kube manager 2024-09-18 05:29:27,964 - INFO - Pod ctest-pod-33895030 verification passed 2024-09-18 05:29:27,972 - DEBUG - Pod ctest-pod-14670403 not in running state.Currently in Pending 2024-09-18 05:29:32,979 - INFO - Pod ctest-pod-14670403 is in running state.Got IP 10.47.255.248 2024-09-18 05:29:33,000 - DEBUG - Pod ctest-pod-14670403 has vmi f2aead80-757e-11ef-9f1a-fa163ef07116 2024-09-18 05:29:33,000 - INFO - Verified pod ctest-pod-14670403 in contrail-api 2024-09-18 05:29:33,014 - DEBUG - VMI f2aead80-757e-11ef-9f1a-fa163ef07116 is active in agent 10.0.0.33 2024-09-18 05:29:33,014 - INFO - Verified Pod ctest-pod-14670403 in agent 10.0.0.33 2024-09-18 05:29:33,015 - DEBUG - Requesting: http://10.0.0.42:8108/Snh_VirtualMachineDatabaseList?x=c57aa02b-d1da-41f3-a610-03b5f8af58d9 2024-09-18 05:29:33,020 - INFO - Pod ctest-pod-14670403 with uuid c57aa02b-d1da-41f3-a610-03b5f8af58d9 found in kube manager 2024-09-18 05:29:33,020 - INFO - Pod ctest-pod-14670403 verification passed 2024-09-18 05:29:33,025 - INFO - Pod ctest-pod-07271080 is in running state.Got IP 10.47.255.247 2024-09-18 05:29:33,089 - DEBUG - Pod ctest-pod-07271080 has vmi f3412dd6-757e-11ef-9f1a-fa163ef07116 2024-09-18 05:29:33,089 - INFO - Verified pod ctest-pod-07271080 in contrail-api 2024-09-18 05:29:33,103 - DEBUG - VMI f3412dd6-757e-11ef-9f1a-fa163ef07116 is active in agent 10.0.0.33 2024-09-18 05:29:33,103 - INFO - Verified Pod ctest-pod-07271080 in agent 10.0.0.33 2024-09-18 05:29:33,103 - DEBUG - Requesting: http://10.0.0.42:8108/Snh_VirtualMachineDatabaseList?x=28091988-3ff1-480b-957b-e5266bd9bef0 2024-09-18 05:29:33,108 - INFO - Pod ctest-pod-07271080 with uuid 28091988-3ff1-480b-957b-e5266bd9bef0 found in kube manager 2024-09-18 05:29:33,108 - INFO - Pod ctest-pod-07271080 verification passed 2024-09-18 05:29:33,114 - INFO - Pod ctest-pod-65709397 is in running state.Got IP 10.47.255.245 2024-09-18 05:29:33,177 - DEBUG - Pod ctest-pod-65709397 has vmi f44fab30-757e-11ef-9f1a-fa163ef07116 2024-09-18 05:29:33,177 - INFO - Verified pod ctest-pod-65709397 in contrail-api 2024-09-18 05:29:33,190 - DEBUG - VMI f44fab30-757e-11ef-9f1a-fa163ef07116 is active in agent 10.0.0.33 2024-09-18 05:29:33,190 - INFO - Verified Pod ctest-pod-65709397 in agent 10.0.0.33 2024-09-18 05:29:33,191 - DEBUG - Requesting: http://10.0.0.42:8108/Snh_VirtualMachineDatabaseList?x=518b0655-fc4d-46fb-aa92-5a9df7d93830 2024-09-18 05:29:33,195 - INFO - Pod ctest-pod-65709397 with uuid 518b0655-fc4d-46fb-aa92-5a9df7d93830 found in kube manager 2024-09-18 05:29:33,196 - INFO - Pod ctest-pod-65709397 verification passed 2024-09-18 05:29:33,201 - INFO - Pod ctest-pod-25764333 is in running state.Got IP 10.47.255.246 2024-09-18 05:29:33,220 - DEBUG - Pod ctest-pod-25764333 has vmi f3bf2b1e-757e-11ef-9f1a-fa163ef07116 2024-09-18 05:29:33,220 - INFO - Verified pod ctest-pod-25764333 in contrail-api 2024-09-18 05:29:33,233 - DEBUG - VMI f3bf2b1e-757e-11ef-9f1a-fa163ef07116 is active in agent 10.0.0.33 2024-09-18 05:29:33,233 - INFO - Verified Pod ctest-pod-25764333 in agent 10.0.0.33 2024-09-18 05:29:33,234 - DEBUG - Requesting: http://10.0.0.42:8108/Snh_VirtualMachineDatabaseList?x=ebf1bad8-947d-4d3c-93ba-e831b6182b11 2024-09-18 05:29:33,238 - INFO - Pod ctest-pod-25764333 with uuid ebf1bad8-947d-4d3c-93ba-e831b6182b11 found in kube manager 2024-09-18 05:29:33,239 - INFO - Pod ctest-pod-25764333 verification passed 2024-09-18 05:29:33,244 - INFO - Pod ctest-pod-70339917 is in running state.Got IP 10.47.255.244 2024-09-18 05:29:33,306 - DEBUG - Pod ctest-pod-70339917 has vmi f4e3af6a-757e-11ef-9f1a-fa163ef07116 2024-09-18 05:29:33,306 - INFO - Verified pod ctest-pod-70339917 in contrail-api 2024-09-18 05:29:33,320 - DEBUG - VMI f4e3af6a-757e-11ef-9f1a-fa163ef07116 is active in agent 10.0.0.33 2024-09-18 05:29:33,320 - INFO - Verified Pod ctest-pod-70339917 in agent 10.0.0.33 2024-09-18 05:29:33,320 - DEBUG - Requesting: http://10.0.0.42:8108/Snh_VirtualMachineDatabaseList?x=ce2460b8-dfe3-46f8-a310-eff8b08674b0 2024-09-18 05:29:33,325 - INFO - Pod ctest-pod-70339917 with uuid ce2460b8-dfe3-46f8-a310-eff8b08674b0 found in kube manager 2024-09-18 05:29:33,325 - INFO - Pod ctest-pod-70339917 verification passed 2024-09-18 05:29:33,325 - INFO - ================================================================================ 2024-09-18 05:29:33,325 - INFO - STARTING TEST : test_ingress_rule_on_pod_with_service 2024-09-18 05:29:33,325 - INFO - TEST DESCRIPTION : Verify that ingress rules on a pod work fine when pods are accessed via a service Steps: 1. Create a service in namespace 1 over a pod which are endpoints of a service 2. Create a network policy to allow 1 pod of namespace 1 as ingress. 3. Verify that service respond correctly to the specified pod in the policy and service do not respond to anyone else. 4. Extend the policy and add the namespaceselector as namespace 2. 5. Verify that service become accessible to the pods of namespace 2. 2024-09-18 05:29:33,566 - DEBUG - Nothing to compare xmpp stats {'10.0.0.33': {'10.0.0.42': '0'}} with 2024-09-18 05:29:33,566 - INFO - Initial checks done. Running the testcase now 2024-09-18 05:29:33,566 - INFO - 2024-09-18 05:29:33,631 - DEBUG - [Pod ctest-pod-40801591] Cmd: echo ctest-pod-40801591 > /usr/share/nginx/html/index.html, Output: 2024-09-18 05:29:33,702 - DEBUG - [Pod ctest-pod-90225391] Cmd: echo ctest-pod-90225391 > /usr/share/nginx/html/index.html, Output: 2024-09-18 05:29:33,765 - DEBUG - [Pod ctest-pod-33895030] Cmd: echo ctest-pod-33895030 > /usr/share/nginx/html/index.html, Output: 2024-09-18 05:29:33,832 - DEBUG - [Pod ctest-pod-14670403] Cmd: echo ctest-pod-14670403 > /usr/share/nginx/html/index.html, Output: 2024-09-18 05:29:33,836 - DEBUG - Service ctest-nginx-svc-15224361 not present 2024-09-18 05:29:33,837 - INFO - Creating service ctest-nginx-svc-15224361 2024-09-18 05:29:33,874 - INFO - Created Service ctest-nginx-svc-15224361 2024-09-18 05:29:33,883 - WARNING - Service UUID 12c4989c-371f-4f6d-9533-e220fbb7def1 not yet found in contrail-api 2024-09-18 05:29:34,890 - INFO - Validated that Service ctest-nginx-svc-15224361 is seen in contrail-api 2024-09-18 05:29:34,890 - DEBUG - Requesting: http://10.0.0.42:8108/Snh_LoadbalancerDatabaseList?x=12c4989c-371f-4f6d-9533-e220fbb7def1 2024-09-18 05:29:34,895 - INFO - Service ctest-nginx-svc-15224361 with uuid 12c4989c-371f-4f6d-9533-e220fbb7def1 found in kube manager 2024-09-18 05:29:34,895 - INFO - Service ctest-nginx-svc-15224361 verification passed 2024-09-18 05:29:35,975 - DEBUG - [Pod ctest-pod-07271080] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: Connecting to 10.108.122.255:80 (10.108.122.255:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-40801591 2024-09-18 05:29:35,976 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 passed 2024-09-18 05:29:35,976 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd output: Connecting to 10.108.122.255:80 (10.108.122.255:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-40801591 2024-09-18 05:29:36,055 - DEBUG - [Pod ctest-pod-07271080] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: ctest-pod-90225391 Connecting to 10.108.122.255:80 (10.108.122.255:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-18 05:29:36,055 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 passed 2024-09-18 05:29:36,055 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd output: ctest-pod-90225391 Connecting to 10.108.122.255:80 (10.108.122.255:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-18 05:29:36,056 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-40801591': 1, 'ctest-pod-90225391': 1} 2024-09-18 05:29:36,127 - DEBUG - [Pod ctest-pod-65709397] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: Connecting to 10.108.122.255:80 (10.108.122.255:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-40801591 2024-09-18 05:29:36,128 - DEBUG - [Pod Pod ctest-pod-65709397] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 passed 2024-09-18 05:29:36,128 - DEBUG - [Pod Pod ctest-pod-65709397] Cmd output: Connecting to 10.108.122.255:80 (10.108.122.255:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-40801591 2024-09-18 05:29:36,186 - DEBUG - [Pod ctest-pod-65709397] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: ctest-pod-40801591 Connecting to 10.108.122.255:80 (10.108.122.255:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-18 05:29:36,186 - DEBUG - [Pod Pod ctest-pod-65709397] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 passed 2024-09-18 05:29:36,186 - DEBUG - [Pod Pod ctest-pod-65709397] Cmd output: ctest-pod-40801591 Connecting to 10.108.122.255:80 (10.108.122.255:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-18 05:29:36,247 - DEBUG - [Pod ctest-pod-65709397] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: ctest-pod-90225391 Connecting to 10.108.122.255:80 (10.108.122.255:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-18 05:29:36,247 - DEBUG - [Pod Pod ctest-pod-65709397] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 passed 2024-09-18 05:29:36,247 - DEBUG - [Pod Pod ctest-pod-65709397] Cmd output: ctest-pod-90225391 Connecting to 10.108.122.255:80 (10.108.122.255:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-18 05:29:36,247 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-40801591': 2, 'ctest-pod-90225391': 1} 2024-09-18 05:29:36,250 - DEBUG - Network policy ingress-on-service not present 2024-09-18 05:29:36,251 - INFO - Creating Network Policy ingress-on-service 2024-09-18 05:29:36,263 - INFO - Network policy found in k8s 2024-09-18 05:29:36,263 - DEBUG - Requesting: http://10.0.0.42:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=3ed79e20-48bd-46a6-9e03-62f5f1b8a1fe 2024-09-18 05:29:36,277 - INFO - Network Policy ingress-on-service with uuid 3ed79e20-48bd-46a6-9e03-62f5f1b8a1fe found in kube manager 2024-09-18 05:29:36,320 - INFO - Network policy with name default-policy-management:k8s-allowall found in agent 2024-09-18 05:29:36,320 - INFO - Network policy with name default-policy-management:k8s-allowall associated with default ks8 2024-09-18 05:29:36,320 - DEBUG - Requesting: http://10.0.0.42:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=3ed79e20-48bd-46a6-9e03-62f5f1b8a1fe 2024-09-18 05:29:36,366 - WARNING - Network policy with name ingress-on-service not found in agent 2024-09-18 05:29:38,367 - DEBUG - Requesting: http://10.0.0.42:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=3ed79e20-48bd-46a6-9e03-62f5f1b8a1fe 2024-09-18 05:29:38,400 - INFO - Network policy with name ingress-on-service found in agent 2024-09-18 05:29:38,400 - INFO - Network Policy ingress-on-service verification passed 2024-09-18 05:29:43,464 - DEBUG - [Pod ctest-pod-07271080] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:29:43,464 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:29:43,464 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:29:48,545 - DEBUG - [Pod ctest-pod-07271080] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:29:48,545 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:29:48,545 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:29:53,600 - DEBUG - [Pod ctest-pod-07271080] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:29:53,600 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:29:53,600 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:29:58,651 - DEBUG - [Pod ctest-pod-07271080] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:29:58,651 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:29:58,651 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:30:03,703 - DEBUG - [Pod ctest-pod-07271080] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:30:03,703 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:30:03,703 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:30:08,757 - DEBUG - [Pod ctest-pod-07271080] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:30:08,758 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:30:08,758 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:30:13,820 - DEBUG - [Pod ctest-pod-07271080] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:30:13,820 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:30:13,820 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:30:18,872 - DEBUG - [Pod ctest-pod-07271080] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:30:18,872 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:30:18,872 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:30:23,940 - DEBUG - [Pod ctest-pod-07271080] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:30:23,940 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:30:23,940 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:30:28,994 - DEBUG - [Pod ctest-pod-07271080] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:30:28,994 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:30:28,994 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:30:28,995 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-40801591': 0, 'ctest-pod-90225391': 0} 2024-09-18 05:30:29,065 - DEBUG - [Pod ctest-pod-25764333] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: ctest-pod-40801591 Connecting to 10.108.122.255:80 (10.108.122.255:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-18 05:30:29,065 - DEBUG - [Pod Pod ctest-pod-25764333] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 passed 2024-09-18 05:30:29,065 - DEBUG - [Pod Pod ctest-pod-25764333] Cmd output: ctest-pod-40801591 Connecting to 10.108.122.255:80 (10.108.122.255:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-18 05:30:29,129 - DEBUG - [Pod ctest-pod-25764333] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: ctest-pod-40801591 Connecting to 10.108.122.255:80 (10.108.122.255:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-18 05:30:29,129 - DEBUG - [Pod Pod ctest-pod-25764333] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 passed 2024-09-18 05:30:29,129 - DEBUG - [Pod Pod ctest-pod-25764333] Cmd output: ctest-pod-40801591 Connecting to 10.108.122.255:80 (10.108.122.255:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-18 05:30:29,201 - DEBUG - [Pod ctest-pod-25764333] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: ctest-pod-90225391 Connecting to 10.108.122.255:80 (10.108.122.255:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-18 05:30:29,201 - DEBUG - [Pod Pod ctest-pod-25764333] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 passed 2024-09-18 05:30:29,201 - DEBUG - [Pod Pod ctest-pod-25764333] Cmd output: ctest-pod-90225391 Connecting to 10.108.122.255:80 (10.108.122.255:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-18 05:30:29,201 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-40801591': 2, 'ctest-pod-90225391': 1} 2024-09-18 05:30:34,256 - DEBUG - [Pod ctest-pod-65709397] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:30:34,256 - DEBUG - [Pod Pod ctest-pod-65709397] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:30:34,256 - DEBUG - [Pod Pod ctest-pod-65709397] Cmd output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:30:39,327 - DEBUG - [Pod ctest-pod-65709397] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:30:39,328 - DEBUG - [Pod Pod ctest-pod-65709397] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:30:39,328 - DEBUG - [Pod Pod ctest-pod-65709397] Cmd output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:30:44,383 - DEBUG - [Pod ctest-pod-65709397] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:30:44,384 - DEBUG - [Pod Pod ctest-pod-65709397] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:30:44,384 - DEBUG - [Pod Pod ctest-pod-65709397] Cmd output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:30:49,443 - DEBUG - [Pod ctest-pod-65709397] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:30:49,443 - DEBUG - [Pod Pod ctest-pod-65709397] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:30:49,443 - DEBUG - [Pod Pod ctest-pod-65709397] Cmd output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:30:54,504 - DEBUG - [Pod ctest-pod-65709397] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:30:54,504 - DEBUG - [Pod Pod ctest-pod-65709397] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:30:54,504 - DEBUG - [Pod Pod ctest-pod-65709397] Cmd output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:30:59,575 - DEBUG - [Pod ctest-pod-65709397] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:30:59,576 - DEBUG - [Pod Pod ctest-pod-65709397] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:30:59,576 - DEBUG - [Pod Pod ctest-pod-65709397] Cmd output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:31:04,640 - DEBUG - [Pod ctest-pod-65709397] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:31:04,640 - DEBUG - [Pod Pod ctest-pod-65709397] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:31:04,640 - DEBUG - [Pod Pod ctest-pod-65709397] Cmd output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:31:09,702 - DEBUG - [Pod ctest-pod-65709397] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:31:09,702 - DEBUG - [Pod Pod ctest-pod-65709397] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:31:09,702 - DEBUG - [Pod Pod ctest-pod-65709397] Cmd output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:31:14,783 - DEBUG - [Pod ctest-pod-65709397] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:31:14,783 - DEBUG - [Pod Pod ctest-pod-65709397] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:31:14,783 - DEBUG - [Pod Pod ctest-pod-65709397] Cmd output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:31:19,857 - DEBUG - [Pod ctest-pod-65709397] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:31:19,857 - DEBUG - [Pod Pod ctest-pod-65709397] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:31:19,857 - DEBUG - [Pod Pod ctest-pod-65709397] Cmd output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:31:19,857 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-40801591': 0, 'ctest-pod-90225391': 0} 2024-09-18 05:31:21,919 - DEBUG - [Pod ctest-pod-25764333] Cmd: ping -c 3 10.47.255.247, Output: PING 10.47.255.247 (10.47.255.247): 56 data bytes 64 bytes from 10.47.255.247: seq=0 ttl=63 time=0.582 ms 64 bytes from 10.47.255.247: seq=1 ttl=63 time=0.091 ms 64 bytes from 10.47.255.247: seq=2 ttl=63 time=0.077 ms --- 10.47.255.247 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.077/0.250/0.582 ms 2024-09-18 05:31:21,919 - INFO - Ping check to IP 10.47.255.247 from pod ctest-pod-25764333 with expectation True passed 2024-09-18 05:31:23,992 - DEBUG - [Pod ctest-pod-07271080] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=1.225 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.085 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.110 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.085/0.473/1.225 ms 2024-09-18 05:31:23,992 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-07271080 with expectation True passed 2024-09-18 05:31:26,052 - DEBUG - [Pod ctest-pod-65709397] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=0.843 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.072 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.087 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.072/0.334/0.843 ms 2024-09-18 05:31:26,052 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-65709397 with expectation True passed 2024-09-18 05:31:26,123 - DEBUG - [Pod ctest-pod-25764333] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-09-18 05:31:26,123 - DEBUG - [Pod Pod ctest-pod-25764333] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2024-09-18 05:31:26,123 - DEBUG - [Pod Pod ctest-pod-25764333] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-09-18 05:31:26,123 - INFO - wget check of of http://10.47.255.251 from ctest-pod-25764333 passed 2024-09-18 05:31:26,182 - DEBUG - [Pod ctest-pod-25764333] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-09-18 05:31:26,182 - DEBUG - [Pod Pod ctest-pod-25764333] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2024-09-18 05:31:26,182 - DEBUG - [Pod Pod ctest-pod-25764333] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-09-18 05:31:26,182 - INFO - wget check of of http://10.47.255.250 from ctest-pod-25764333 passed 2024-09-18 05:31:31,262 - DEBUG - [Pod ctest-pod-07271080] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-09-18 05:31:31,262 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-09-18 05:31:31,262 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-09-18 05:31:31,262 - INFO - wget check of of http://10.47.255.250 from ctest-pod-07271080 passed 2024-09-18 05:31:36,326 - DEBUG - [Pod ctest-pod-70339917] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) wget: download timed out 2024-09-18 05:31:36,326 - DEBUG - [Pod Pod ctest-pod-70339917] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.251 (10.47.255.251:80) wget: download timed out 2024-09-18 05:31:36,326 - DEBUG - [Pod Pod ctest-pod-70339917] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) wget: download timed out 2024-09-18 05:31:36,326 - INFO - wget check of of http://10.47.255.251 from ctest-pod-70339917 passed 2024-09-18 05:31:36,350 - INFO - Updating Network Policy ingress-on-service 2024-09-18 05:31:41,456 - DEBUG - [Pod ctest-pod-07271080] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:31:41,456 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:31:41,456 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:31:46,508 - DEBUG - [Pod ctest-pod-07271080] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:31:46,508 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:31:46,508 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:31:51,558 - DEBUG - [Pod ctest-pod-07271080] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:31:51,558 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:31:51,558 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:31:56,620 - DEBUG - [Pod ctest-pod-07271080] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:31:56,621 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:31:56,621 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:32:01,685 - DEBUG - [Pod ctest-pod-07271080] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:32:01,685 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:32:01,685 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:32:06,744 - DEBUG - [Pod ctest-pod-07271080] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:32:06,744 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:32:06,744 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:32:11,796 - DEBUG - [Pod ctest-pod-07271080] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:32:11,796 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:32:11,796 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:32:16,856 - DEBUG - [Pod ctest-pod-07271080] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:32:16,856 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:32:16,856 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:32:21,917 - DEBUG - [Pod ctest-pod-07271080] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:32:21,917 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:32:21,917 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:32:26,968 - DEBUG - [Pod ctest-pod-07271080] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:32:26,968 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:32:26,968 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd output: Connecting to 10.108.122.255:80 (10.108.122.255:80) wget: download timed out 2024-09-18 05:32:26,968 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-40801591': 0, 'ctest-pod-90225391': 0} 2024-09-18 05:32:27,021 - DEBUG - [Pod ctest-pod-25764333] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: ctest-pod-90225391 Connecting to 10.108.122.255:80 (10.108.122.255:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-18 05:32:27,021 - DEBUG - [Pod Pod ctest-pod-25764333] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 passed 2024-09-18 05:32:27,021 - DEBUG - [Pod Pod ctest-pod-25764333] Cmd output: ctest-pod-90225391 Connecting to 10.108.122.255:80 (10.108.122.255:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-18 05:32:27,087 - DEBUG - [Pod ctest-pod-25764333] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: ctest-pod-40801591 Connecting to 10.108.122.255:80 (10.108.122.255:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-18 05:32:27,087 - DEBUG - [Pod Pod ctest-pod-25764333] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 passed 2024-09-18 05:32:27,087 - DEBUG - [Pod Pod ctest-pod-25764333] Cmd output: ctest-pod-40801591 Connecting to 10.108.122.255:80 (10.108.122.255:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-18 05:32:27,087 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-40801591': 1, 'ctest-pod-90225391': 1} 2024-09-18 05:32:27,135 - DEBUG - [Pod ctest-pod-65709397] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: ctest-pod-40801591 Connecting to 10.108.122.255:80 (10.108.122.255:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-18 05:32:27,135 - DEBUG - [Pod Pod ctest-pod-65709397] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 passed 2024-09-18 05:32:27,135 - DEBUG - [Pod Pod ctest-pod-65709397] Cmd output: ctest-pod-40801591 Connecting to 10.108.122.255:80 (10.108.122.255:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-18 05:32:27,191 - DEBUG - [Pod ctest-pod-65709397] Cmd: wget http://10.108.122.255:80/ -O - -T 5 -t 1 , Output: ctest-pod-90225391 Connecting to 10.108.122.255:80 (10.108.122.255:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-18 05:32:27,191 - DEBUG - [Pod Pod ctest-pod-65709397] Cmd wget http://10.108.122.255:80/ -O - -T 5 -t 1 passed 2024-09-18 05:32:27,191 - DEBUG - [Pod Pod ctest-pod-65709397] Cmd output: ctest-pod-90225391 Connecting to 10.108.122.255:80 (10.108.122.255:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-18 05:32:27,191 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-40801591': 1, 'ctest-pod-90225391': 1} 2024-09-18 05:32:29,261 - DEBUG - [Pod ctest-pod-25764333] Cmd: ping -c 3 10.47.255.247, Output: PING 10.47.255.247 (10.47.255.247): 56 data bytes 64 bytes from 10.47.255.247: seq=0 ttl=63 time=0.853 ms 64 bytes from 10.47.255.247: seq=1 ttl=63 time=0.076 ms 64 bytes from 10.47.255.247: seq=2 ttl=63 time=0.090 ms --- 10.47.255.247 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.076/0.339/0.853 ms 2024-09-18 05:32:29,261 - INFO - Ping check to IP 10.47.255.247 from pod ctest-pod-25764333 with expectation True passed 2024-09-18 05:32:31,317 - DEBUG - [Pod ctest-pod-07271080] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=0.397 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.074 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.114 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.074/0.195/0.397 ms 2024-09-18 05:32:31,317 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-07271080 with expectation True passed 2024-09-18 05:32:33,370 - DEBUG - [Pod ctest-pod-65709397] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=1.265 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.088 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.098 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.088/0.483/1.265 ms 2024-09-18 05:32:33,371 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-65709397 with expectation True passed 2024-09-18 05:32:33,426 - DEBUG - [Pod ctest-pod-25764333] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-09-18 05:32:33,426 - DEBUG - [Pod Pod ctest-pod-25764333] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2024-09-18 05:32:33,426 - DEBUG - [Pod Pod ctest-pod-25764333] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-09-18 05:32:33,426 - INFO - wget check of of http://10.47.255.251 from ctest-pod-25764333 passed 2024-09-18 05:32:33,491 - DEBUG - [Pod ctest-pod-25764333] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-09-18 05:32:33,491 - DEBUG - [Pod Pod ctest-pod-25764333] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2024-09-18 05:32:33,491 - DEBUG - [Pod Pod ctest-pod-25764333] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-09-18 05:32:33,491 - INFO - wget check of of http://10.47.255.250 from ctest-pod-25764333 passed 2024-09-18 05:32:38,565 - DEBUG - [Pod ctest-pod-07271080] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-09-18 05:32:38,565 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-09-18 05:32:38,565 - DEBUG - [Pod Pod ctest-pod-07271080] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-09-18 05:32:38,566 - INFO - wget check of of http://10.47.255.250 from ctest-pod-07271080 passed 2024-09-18 05:32:38,640 - DEBUG - [Pod ctest-pod-70339917] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-09-18 05:32:38,640 - DEBUG - [Pod Pod ctest-pod-70339917] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2024-09-18 05:32:38,640 - DEBUG - [Pod Pod ctest-pod-70339917] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-09-18 05:32:38,640 - INFO - wget check of of http://10.47.255.251 from ctest-pod-70339917 passed 2024-09-18 05:32:38,641 - INFO - Deleting Network Policy : ingress-on-service 2024-09-18 05:32:38,641 - INFO - Deleting service : ctest-nginx-svc-15224361 2024-09-18 05:32:38,906 - DEBUG - No XMPP flaps were noticed during the test 2024-09-18 05:32:38,906 - INFO - END TEST : test_ingress_rule_on_pod_with_service : PASSED[0:03:05] 2024-09-18 05:32:38,906 - INFO - -------------------------------------------------------------------------------- 2024-09-18 05:32:38,912 - INFO - Deleting pod ctest-new-default-03627883:ctest-pod-40801591 2024-09-18 05:32:38,912 - INFO - Deleting pod ctest-new-default-03627883:ctest-pod-90225391 2024-09-18 05:32:38,913 - INFO - Deleting pod ctest-non-default-77035363:ctest-pod-33895030 2024-09-18 05:32:38,913 - INFO - Deleting pod ctest-non-default-77035363:ctest-pod-14670403 2024-09-18 05:32:38,914 - INFO - Deleting pod ctest-new-default-03627883:ctest-pod-07271080 2024-09-18 05:32:38,914 - INFO - Deleting pod ctest-new-default-03627883:ctest-pod-25764333 2024-09-18 05:32:38,916 - INFO - Deleting pod ctest-non-default-77035363:ctest-pod-65709397 2024-09-18 05:32:38,916 - INFO - Deleting pod ctest-non-default-77035363:ctest-pod-70339917 2024-09-18 05:32:38,998 - WARNING - Pod uuid 2b2a5d8b-814b-48e6-96fd-042ccb881686 is still seen in agent 10.0.0.33 VM list 2024-09-18 05:32:39,007 - WARNING - Pod uuid 704a6f5e-f6b0-4fbb-9d51-1c48ffc5b550 is still seen in agent 10.0.0.33 VM list 2024-09-18 05:32:39,047 - WARNING - Pod uuid ebf1bad8-947d-4d3c-93ba-e831b6182b11 is still seen in agent 10.0.0.33 VM list 2024-09-18 05:32:39,122 - WARNING - Pod uuid c57aa02b-d1da-41f3-a610-03b5f8af58d9 is still seen in agent 10.0.0.33 VM list 2024-09-18 05:32:39,122 - WARNING - Pod uuid 28091988-3ff1-480b-957b-e5266bd9bef0 is still seen in agent 10.0.0.33 VM list 2024-09-18 05:32:39,123 - WARNING - Pod uuid 06c593fd-aaac-4cd7-a40d-cda801d30baf is still seen in agent 10.0.0.33 VM list 2024-09-18 05:32:39,123 - WARNING - Pod uuid ce2460b8-dfe3-46f8-a310-eff8b08674b0 is still seen in agent 10.0.0.33 VM list 2024-09-18 05:32:39,124 - WARNING - Pod uuid 518b0655-fc4d-46fb-aa92-5a9df7d93830 is still seen in agent 10.0.0.33 VM list 2024-09-18 05:32:44,016 - DEBUG - Pod 2b2a5d8b-814b-48e6-96fd-042ccb881686 is not in agent 10.0.0.33 VM list 2024-09-18 05:32:44,019 - DEBUG - Pod 704a6f5e-f6b0-4fbb-9d51-1c48ffc5b550 is not in agent 10.0.0.33 VM list 2024-09-18 05:32:44,035 - DEBUG - VMI f0d7cfd2-757e-11ef-9f1a-fa163ef07116 is removed from agent 10.0.0.33 2024-09-18 05:32:44,035 - INFO - Verified that pod ctest-pod-40801591 is removed in agent 2024-09-18 05:32:44,036 - DEBUG - VMI f16136f0-757e-11ef-9f1a-fa163ef07116 is removed from agent 10.0.0.33 2024-09-18 05:32:44,036 - INFO - Verified that pod ctest-pod-90225391 is removed in agent 2024-09-18 05:32:44,054 - WARNING - Pod uuid ebf1bad8-947d-4d3c-93ba-e831b6182b11 is still seen in agent 10.0.0.33 VM list 2024-09-18 05:32:44,147 - WARNING - Pod uuid 518b0655-fc4d-46fb-aa92-5a9df7d93830 is still seen in agent 10.0.0.33 VM list 2024-09-18 05:32:44,147 - WARNING - Pod uuid 28091988-3ff1-480b-957b-e5266bd9bef0 is still seen in agent 10.0.0.33 VM list 2024-09-18 05:32:44,149 - DEBUG - Pod c57aa02b-d1da-41f3-a610-03b5f8af58d9 is not in agent 10.0.0.33 VM list 2024-09-18 05:32:44,151 - DEBUG - Pod 06c593fd-aaac-4cd7-a40d-cda801d30baf is not in agent 10.0.0.33 VM list 2024-09-18 05:32:44,152 - WARNING - Pod uuid ce2460b8-dfe3-46f8-a310-eff8b08674b0 is still seen in agent 10.0.0.33 VM list 2024-09-18 05:32:44,161 - DEBUG - VMI f225e39c-757e-11ef-9f1a-fa163ef07116 is removed from agent 10.0.0.33 2024-09-18 05:32:44,161 - INFO - Verified that pod ctest-pod-33895030 is removed in agent 2024-09-18 05:32:44,162 - DEBUG - VMI f2aead80-757e-11ef-9f1a-fa163ef07116 is removed from agent 10.0.0.33 2024-09-18 05:32:44,162 - INFO - Verified that pod ctest-pod-14670403 is removed in agent 2024-09-18 05:32:49,063 - WARNING - Pod uuid ebf1bad8-947d-4d3c-93ba-e831b6182b11 is still seen in agent 10.0.0.33 VM list 2024-09-18 05:32:49,161 - WARNING - Pod uuid 518b0655-fc4d-46fb-aa92-5a9df7d93830 is still seen in agent 10.0.0.33 VM list 2024-09-18 05:32:49,163 - WARNING - Pod uuid ce2460b8-dfe3-46f8-a310-eff8b08674b0 is still seen in agent 10.0.0.33 VM list 2024-09-18 05:32:49,164 - WARNING - Pod uuid 28091988-3ff1-480b-957b-e5266bd9bef0 is still seen in agent 10.0.0.33 VM list 2024-09-18 05:32:54,072 - WARNING - Pod uuid ebf1bad8-947d-4d3c-93ba-e831b6182b11 is still seen in agent 10.0.0.33 VM list 2024-09-18 05:32:54,179 - WARNING - Pod uuid ce2460b8-dfe3-46f8-a310-eff8b08674b0 is still seen in agent 10.0.0.33 VM list 2024-09-18 05:32:54,179 - WARNING - Pod uuid 518b0655-fc4d-46fb-aa92-5a9df7d93830 is still seen in agent 10.0.0.33 VM list 2024-09-18 05:32:54,181 - WARNING - Pod uuid 28091988-3ff1-480b-957b-e5266bd9bef0 is still seen in agent 10.0.0.33 VM list 2024-09-18 05:32:59,080 - WARNING - Pod uuid ebf1bad8-947d-4d3c-93ba-e831b6182b11 is still seen in agent 10.0.0.33 VM list 2024-09-18 05:32:59,194 - WARNING - Pod uuid 518b0655-fc4d-46fb-aa92-5a9df7d93830 is still seen in agent 10.0.0.33 VM list 2024-09-18 05:32:59,196 - WARNING - Pod uuid ce2460b8-dfe3-46f8-a310-eff8b08674b0 is still seen in agent 10.0.0.33 VM list 2024-09-18 05:32:59,196 - WARNING - Pod uuid 28091988-3ff1-480b-957b-e5266bd9bef0 is still seen in agent 10.0.0.33 VM list 2024-09-18 05:33:04,090 - WARNING - Pod uuid ebf1bad8-947d-4d3c-93ba-e831b6182b11 is still seen in agent 10.0.0.33 VM list 2024-09-18 05:33:04,211 - WARNING - Pod uuid 518b0655-fc4d-46fb-aa92-5a9df7d93830 is still seen in agent 10.0.0.33 VM list 2024-09-18 05:33:04,212 - WARNING - Pod uuid ce2460b8-dfe3-46f8-a310-eff8b08674b0 is still seen in agent 10.0.0.33 VM list 2024-09-18 05:33:04,213 - WARNING - Pod uuid 28091988-3ff1-480b-957b-e5266bd9bef0 is still seen in agent 10.0.0.33 VM list 2024-09-18 05:33:09,098 - WARNING - Pod uuid ebf1bad8-947d-4d3c-93ba-e831b6182b11 is still seen in agent 10.0.0.33 VM list 2024-09-18 05:33:09,227 - WARNING - Pod uuid 518b0655-fc4d-46fb-aa92-5a9df7d93830 is still seen in agent 10.0.0.33 VM list 2024-09-18 05:33:09,229 - WARNING - Pod uuid ce2460b8-dfe3-46f8-a310-eff8b08674b0 is still seen in agent 10.0.0.33 VM list 2024-09-18 05:33:09,229 - WARNING - Pod uuid 28091988-3ff1-480b-957b-e5266bd9bef0 is still seen in agent 10.0.0.33 VM list 2024-09-18 05:33:14,106 - DEBUG - Pod ebf1bad8-947d-4d3c-93ba-e831b6182b11 is not in agent 10.0.0.33 VM list 2024-09-18 05:33:14,113 - DEBUG - VMI f3bf2b1e-757e-11ef-9f1a-fa163ef07116 is removed from agent 10.0.0.33 2024-09-18 05:33:14,113 - INFO - Verified that pod ctest-pod-25764333 is removed in agent 2024-09-18 05:33:14,244 - DEBUG - Pod 518b0655-fc4d-46fb-aa92-5a9df7d93830 is not in agent 10.0.0.33 VM list 2024-09-18 05:33:14,247 - DEBUG - Pod ce2460b8-dfe3-46f8-a310-eff8b08674b0 is not in agent 10.0.0.33 VM list 2024-09-18 05:33:14,249 - DEBUG - Pod 28091988-3ff1-480b-957b-e5266bd9bef0 is not in agent 10.0.0.33 VM list 2024-09-18 05:33:14,260 - DEBUG - VMI f44fab30-757e-11ef-9f1a-fa163ef07116 is removed from agent 10.0.0.33 2024-09-18 05:33:14,260 - INFO - Verified that pod ctest-pod-65709397 is removed in agent 2024-09-18 05:33:14,262 - DEBUG - VMI f4e3af6a-757e-11ef-9f1a-fa163ef07116 is removed from agent 10.0.0.33 2024-09-18 05:33:14,262 - INFO - Verified that pod ctest-pod-70339917 is removed in agent 2024-09-18 05:33:14,262 - DEBUG - VMI f3412dd6-757e-11ef-9f1a-fa163ef07116 is removed from agent 10.0.0.33 2024-09-18 05:33:14,262 - INFO - Verified that pod ctest-pod-07271080 is removed in agent 2024-09-18 05:33:14,262 - INFO - Deleting namespace ctest-non-default-77035363 2024-09-18 05:33:14,282 - DEBUG - Namespace ctest-non-default-77035363 still in k8s 2024-09-18 05:33:16,286 - DEBUG - Namespace ctest-non-default-77035363 still in k8s 2024-09-18 05:33:18,291 - DEBUG - Namespace ctest-non-default-77035363 still in k8s 2024-09-18 05:33:20,296 - DEBUG - Namespace ctest-non-default-77035363 is not in k8s 2024-09-18 05:33:20,296 - DEBUG - No need to check project deletion in contrail-api 2024-09-18 05:33:20,296 - INFO - Deleting namespace ctest-new-default-03627883 2024-09-18 05:33:20,306 - DEBUG - Namespace ctest-new-default-03627883 still in k8s 2024-09-18 05:33:22,310 - DEBUG - Namespace ctest-new-default-03627883 still in k8s 2024-09-18 05:33:24,315 - DEBUG - Namespace ctest-new-default-03627883 still in k8s 2024-09-18 05:33:26,320 - DEBUG - Namespace ctest-new-default-03627883 is not in k8s 2024-09-18 05:33:26,320 - DEBUG - No need to check project deletion in contrail-api