2024-08-30 02:01:32,820 - INFO - Using existing project ['default-domain', 'k8s-default'](9fd622a1-f57e-4789-94d9-5750d48555cf) 2024-08-30 02:01:32,837 - DEBUG - Namespace ctest-new-default-93885893 not present 2024-08-30 02:01:32,853 - INFO - Created namespace ctest-new-default-93885893 2024-08-30 02:01:35,857 - DEBUG - Namespace ctest-non-default-91187833 not present 2024-08-30 02:01:35,871 - INFO - Created namespace ctest-non-default-91187833 2024-08-30 02:01:38,890 - DEBUG - Pod ctest-pod-50396244 not present 2024-08-30 02:01:38,891 - INFO - Creating Pod ctest-pod-50396244 2024-08-30 02:01:38,945 - DEBUG - Pod : ctest-pod-50396244 UUID is b20c1d45-7b84-43ba-beee-0b50bad38058 2024-08-30 02:01:38,948 - DEBUG - Pod ctest-pod-30237662 not present 2024-08-30 02:01:38,948 - INFO - Creating Pod ctest-pod-30237662 2024-08-30 02:01:38,977 - DEBUG - Pod : ctest-pod-30237662 UUID is 927868a4-8166-40fc-a6f3-8265808d4f93 2024-08-30 02:01:38,982 - DEBUG - Pod ctest-pod-23425777 not present 2024-08-30 02:01:38,982 - INFO - Creating Pod ctest-pod-23425777 2024-08-30 02:01:39,019 - DEBUG - Pod : ctest-pod-23425777 UUID is 32b2db15-50b9-475d-a973-3ceb69afaf16 2024-08-30 02:01:39,021 - DEBUG - Pod ctest-pod-16572061 not present 2024-08-30 02:01:39,021 - INFO - Creating Pod ctest-pod-16572061 2024-08-30 02:01:39,038 - DEBUG - Pod : ctest-pod-16572061 UUID is 392a5083-66b5-4146-b686-8cf5e79821da 2024-08-30 02:01:39,040 - DEBUG - Pod ctest-pod-16886868 not present 2024-08-30 02:01:39,040 - INFO - Creating Pod ctest-pod-16886868 2024-08-30 02:01:39,056 - DEBUG - Pod : ctest-pod-16886868 UUID is 44bddf59-4891-4da1-8d9c-d909ea37ff14 2024-08-30 02:01:39,059 - DEBUG - Pod ctest-pod-45140463 not present 2024-08-30 02:01:39,059 - INFO - Creating Pod ctest-pod-45140463 2024-08-30 02:01:39,073 - DEBUG - Pod : ctest-pod-45140463 UUID is ab9397a6-bab3-4abf-9d2c-7863286e8810 2024-08-30 02:01:39,076 - DEBUG - Pod ctest-pod-48042648 not present 2024-08-30 02:01:39,076 - INFO - Creating Pod ctest-pod-48042648 2024-08-30 02:01:39,094 - DEBUG - Pod : ctest-pod-48042648 UUID is 7ad1ad09-8099-462c-9fed-e59c07c8dd09 2024-08-30 02:01:39,096 - DEBUG - Pod ctest-pod-53278191 not present 2024-08-30 02:01:39,096 - INFO - Creating Pod ctest-pod-53278191 2024-08-30 02:01:39,111 - DEBUG - Pod : ctest-pod-53278191 UUID is 46b082ce-8664-4dee-b042-41f1748673d0 2024-08-30 02:01:39,272 - INFO - Project default-domain:k8s-ctest-new-default-93885893 for Namespace ctest-new-default-93885893 is seen in contrail-api 2024-08-30 02:01:39,273 - DEBUG - Project uuid in Contrail is b9d9c443-0573-4a19-bff1-c5f683bce9d1 2024-08-30 02:01:39,425 - DEBUG - Requesting: http://10.0.0.56:8108/Snh_NamespaceDatabaseList?namespace_uuid=bca30297-687b-4808-8999-0f8c116d6fbe 2024-08-30 02:01:39,431 - INFO - Namespace ctest-new-default-93885893 with uuid bca30297-687b-4808-8999-0f8c116d6fbe found in kube manager 2024-08-30 02:01:39,431 - INFO - Namespace ctest-new-default-93885893 verification passed 2024-08-30 02:01:39,605 - INFO - Project default-domain:k8s-ctest-non-default-91187833 for Namespace ctest-non-default-91187833 is seen in contrail-api 2024-08-30 02:01:39,605 - DEBUG - Project uuid in Contrail is 09d0997c-0de9-4f88-952d-27e29fff508b 2024-08-30 02:01:39,605 - DEBUG - Requesting: http://10.0.0.56:8108/Snh_NamespaceDatabaseList?namespace_uuid=1391df67-7f14-4387-8e4a-f4543a169b4f 2024-08-30 02:01:39,611 - INFO - Namespace ctest-non-default-91187833 with uuid 1391df67-7f14-4387-8e4a-f4543a169b4f found in kube manager 2024-08-30 02:01:39,611 - INFO - Namespace ctest-non-default-91187833 verification passed 2024-08-30 02:01:39,616 - DEBUG - Pod ctest-pod-50396244 not in running state.Currently in Pending 2024-08-30 02:01:44,623 - DEBUG - Pod ctest-pod-50396244 not in running state.Currently in Pending 2024-08-30 02:01:49,635 - DEBUG - Pod ctest-pod-50396244 not in running state.Currently in Pending 2024-08-30 02:01:55,125 - INFO - Pod ctest-pod-50396244 is in running state.Got IP 10.47.255.251 2024-08-30 02:01:55,144 - DEBUG - Pod ctest-pod-50396244 has vmi cd0b81b2-6673-11ef-b0d4-fa163eb2fd2f 2024-08-30 02:01:55,144 - INFO - Verified pod ctest-pod-50396244 in contrail-api 2024-08-30 02:01:55,159 - DEBUG - VMI cd0b81b2-6673-11ef-b0d4-fa163eb2fd2f is active in agent 10.0.0.45 2024-08-30 02:01:55,159 - INFO - Verified Pod ctest-pod-50396244 in agent 10.0.0.45 2024-08-30 02:01:55,159 - DEBUG - Requesting: http://10.0.0.56:8108/Snh_VirtualMachineDatabaseList?x=b20c1d45-7b84-43ba-beee-0b50bad38058 2024-08-30 02:01:55,164 - INFO - Pod ctest-pod-50396244 with uuid b20c1d45-7b84-43ba-beee-0b50bad38058 found in kube manager 2024-08-30 02:01:55,164 - INFO - Pod ctest-pod-50396244 verification passed 2024-08-30 02:01:55,171 - INFO - Pod ctest-pod-30237662 is in running state.Got IP 10.47.255.250 2024-08-30 02:01:55,232 - DEBUG - Pod ctest-pod-30237662 has vmi cd751e74-6673-11ef-b0d4-fa163eb2fd2f 2024-08-30 02:01:55,232 - INFO - Verified pod ctest-pod-30237662 in contrail-api 2024-08-30 02:01:55,245 - DEBUG - VMI cd751e74-6673-11ef-b0d4-fa163eb2fd2f is active in agent 10.0.0.45 2024-08-30 02:01:55,245 - INFO - Verified Pod ctest-pod-30237662 in agent 10.0.0.45 2024-08-30 02:01:55,246 - DEBUG - Requesting: http://10.0.0.56:8108/Snh_VirtualMachineDatabaseList?x=927868a4-8166-40fc-a6f3-8265808d4f93 2024-08-30 02:01:55,252 - INFO - Pod ctest-pod-30237662 with uuid 927868a4-8166-40fc-a6f3-8265808d4f93 found in kube manager 2024-08-30 02:01:55,252 - INFO - Pod ctest-pod-30237662 verification passed 2024-08-30 02:01:55,263 - INFO - Pod ctest-pod-23425777 is in running state.Got IP 10.47.255.249 2024-08-30 02:01:55,279 - DEBUG - Pod ctest-pod-23425777 has vmi ce05dd24-6673-11ef-b0d4-fa163eb2fd2f 2024-08-30 02:01:55,279 - INFO - Verified pod ctest-pod-23425777 in contrail-api 2024-08-30 02:01:55,291 - DEBUG - VMI ce05dd24-6673-11ef-b0d4-fa163eb2fd2f is active in agent 10.0.0.45 2024-08-30 02:01:55,291 - INFO - Verified Pod ctest-pod-23425777 in agent 10.0.0.45 2024-08-30 02:01:55,291 - DEBUG - Requesting: http://10.0.0.56:8108/Snh_VirtualMachineDatabaseList?x=32b2db15-50b9-475d-a973-3ceb69afaf16 2024-08-30 02:01:55,297 - INFO - Pod ctest-pod-23425777 with uuid 32b2db15-50b9-475d-a973-3ceb69afaf16 found in kube manager 2024-08-30 02:01:55,297 - INFO - Pod ctest-pod-23425777 verification passed 2024-08-30 02:01:55,304 - DEBUG - Pod ctest-pod-16572061 not in running state.Currently in Pending 2024-08-30 02:02:00,310 - INFO - Pod ctest-pod-16572061 is in running state.Got IP 10.47.255.248 2024-08-30 02:02:00,322 - DEBUG - Pod ctest-pod-16572061 has vmi ce702436-6673-11ef-b0d4-fa163eb2fd2f 2024-08-30 02:02:00,322 - INFO - Verified pod ctest-pod-16572061 in contrail-api 2024-08-30 02:02:00,338 - DEBUG - VMI ce702436-6673-11ef-b0d4-fa163eb2fd2f is active in agent 10.0.0.45 2024-08-30 02:02:00,338 - INFO - Verified Pod ctest-pod-16572061 in agent 10.0.0.45 2024-08-30 02:02:00,338 - DEBUG - Requesting: http://10.0.0.56:8108/Snh_VirtualMachineDatabaseList?x=392a5083-66b5-4146-b686-8cf5e79821da 2024-08-30 02:02:00,344 - INFO - Pod ctest-pod-16572061 with uuid 392a5083-66b5-4146-b686-8cf5e79821da found in kube manager 2024-08-30 02:02:00,344 - INFO - Pod ctest-pod-16572061 verification passed 2024-08-30 02:02:00,351 - INFO - Pod ctest-pod-16886868 is in running state.Got IP 10.47.255.247 2024-08-30 02:02:00,405 - DEBUG - Pod ctest-pod-16886868 has vmi cef2909c-6673-11ef-b0d4-fa163eb2fd2f 2024-08-30 02:02:00,405 - INFO - Verified pod ctest-pod-16886868 in contrail-api 2024-08-30 02:02:00,417 - DEBUG - VMI cef2909c-6673-11ef-b0d4-fa163eb2fd2f is active in agent 10.0.0.45 2024-08-30 02:02:00,418 - INFO - Verified Pod ctest-pod-16886868 in agent 10.0.0.45 2024-08-30 02:02:00,418 - DEBUG - Requesting: http://10.0.0.56:8108/Snh_VirtualMachineDatabaseList?x=44bddf59-4891-4da1-8d9c-d909ea37ff14 2024-08-30 02:02:00,422 - INFO - Pod ctest-pod-16886868 with uuid 44bddf59-4891-4da1-8d9c-d909ea37ff14 found in kube manager 2024-08-30 02:02:00,422 - INFO - Pod ctest-pod-16886868 verification passed 2024-08-30 02:02:00,427 - INFO - Pod ctest-pod-48042648 is in running state.Got IP 10.47.255.245 2024-08-30 02:02:00,480 - DEBUG - Pod ctest-pod-48042648 has vmi cfe66532-6673-11ef-b0d4-fa163eb2fd2f 2024-08-30 02:02:00,480 - INFO - Verified pod ctest-pod-48042648 in contrail-api 2024-08-30 02:02:00,500 - DEBUG - VMI cfe66532-6673-11ef-b0d4-fa163eb2fd2f is active in agent 10.0.0.45 2024-08-30 02:02:00,500 - INFO - Verified Pod ctest-pod-48042648 in agent 10.0.0.45 2024-08-30 02:02:00,500 - DEBUG - Requesting: http://10.0.0.56:8108/Snh_VirtualMachineDatabaseList?x=7ad1ad09-8099-462c-9fed-e59c07c8dd09 2024-08-30 02:02:00,504 - INFO - Pod ctest-pod-48042648 with uuid 7ad1ad09-8099-462c-9fed-e59c07c8dd09 found in kube manager 2024-08-30 02:02:00,505 - INFO - Pod ctest-pod-48042648 verification passed 2024-08-30 02:02:00,509 - INFO - Pod ctest-pod-45140463 is in running state.Got IP 10.47.255.246 2024-08-30 02:02:00,519 - DEBUG - Pod ctest-pod-45140463 has vmi cf6967d0-6673-11ef-b0d4-fa163eb2fd2f 2024-08-30 02:02:00,519 - INFO - Verified pod ctest-pod-45140463 in contrail-api 2024-08-30 02:02:00,533 - DEBUG - VMI cf6967d0-6673-11ef-b0d4-fa163eb2fd2f is active in agent 10.0.0.45 2024-08-30 02:02:00,533 - INFO - Verified Pod ctest-pod-45140463 in agent 10.0.0.45 2024-08-30 02:02:00,533 - DEBUG - Requesting: http://10.0.0.56:8108/Snh_VirtualMachineDatabaseList?x=ab9397a6-bab3-4abf-9d2c-7863286e8810 2024-08-30 02:02:00,537 - INFO - Pod ctest-pod-45140463 with uuid ab9397a6-bab3-4abf-9d2c-7863286e8810 found in kube manager 2024-08-30 02:02:00,538 - INFO - Pod ctest-pod-45140463 verification passed 2024-08-30 02:02:00,542 - INFO - Pod ctest-pod-53278191 is in running state.Got IP 10.47.255.244 2024-08-30 02:02:00,551 - DEBUG - Pod ctest-pod-53278191 has vmi d06e549c-6673-11ef-b0d4-fa163eb2fd2f 2024-08-30 02:02:00,551 - INFO - Verified pod ctest-pod-53278191 in contrail-api 2024-08-30 02:02:00,564 - DEBUG - VMI d06e549c-6673-11ef-b0d4-fa163eb2fd2f is active in agent 10.0.0.45 2024-08-30 02:02:00,564 - INFO - Verified Pod ctest-pod-53278191 in agent 10.0.0.45 2024-08-30 02:02:00,565 - DEBUG - Requesting: http://10.0.0.56:8108/Snh_VirtualMachineDatabaseList?x=46b082ce-8664-4dee-b042-41f1748673d0 2024-08-30 02:02:00,569 - INFO - Pod ctest-pod-53278191 with uuid 46b082ce-8664-4dee-b042-41f1748673d0 found in kube manager 2024-08-30 02:02:00,569 - INFO - Pod ctest-pod-53278191 verification passed 2024-08-30 02:02:00,569 - INFO - ================================================================================ 2024-08-30 02:02:00,569 - INFO - STARTING TEST : test_ingress_rule_on_pod_with_service 2024-08-30 02:02:00,569 - INFO - TEST DESCRIPTION : Verify that ingress rules on a pod work fine when pods are accessed via a service Steps: 1. Create a service in namespace 1 over a pod which are endpoints of a service 2. Create a network policy to allow 1 pod of namespace 1 as ingress. 3. Verify that service respond correctly to the specified pod in the policy and service do not respond to anyone else. 4. Extend the policy and add the namespaceselector as namespace 2. 5. Verify that service become accessible to the pods of namespace 2. 2024-08-30 02:02:04,674 - DEBUG - Nothing to compare xmpp stats {'10.0.0.45': {'10.0.0.56': '0'}} with 2024-08-30 02:02:04,674 - INFO - Initial checks done. Running the testcase now 2024-08-30 02:02:04,674 - INFO - 2024-08-30 02:02:04,734 - DEBUG - [Pod ctest-pod-50396244] Cmd: echo ctest-pod-50396244 > /usr/share/nginx/html/index.html, Output: 2024-08-30 02:02:04,801 - DEBUG - [Pod ctest-pod-30237662] Cmd: echo ctest-pod-30237662 > /usr/share/nginx/html/index.html, Output: 2024-08-30 02:02:04,882 - DEBUG - [Pod ctest-pod-23425777] Cmd: echo ctest-pod-23425777 > /usr/share/nginx/html/index.html, Output: 2024-08-30 02:02:04,946 - DEBUG - [Pod ctest-pod-16572061] Cmd: echo ctest-pod-16572061 > /usr/share/nginx/html/index.html, Output: 2024-08-30 02:02:04,951 - DEBUG - Service ctest-nginx-svc-70591820 not present 2024-08-30 02:02:04,951 - INFO - Creating service ctest-nginx-svc-70591820 2024-08-30 02:02:04,971 - INFO - Created Service ctest-nginx-svc-70591820 2024-08-30 02:02:04,985 - WARNING - Service UUID 16916347-9836-45ad-acb8-74e2652a8673 not yet found in contrail-api 2024-08-30 02:02:05,999 - INFO - Validated that Service ctest-nginx-svc-70591820 is seen in contrail-api 2024-08-30 02:02:05,999 - DEBUG - Requesting: http://10.0.0.56:8108/Snh_LoadbalancerDatabaseList?x=16916347-9836-45ad-acb8-74e2652a8673 2024-08-30 02:02:06,005 - INFO - Service ctest-nginx-svc-70591820 with uuid 16916347-9836-45ad-acb8-74e2652a8673 found in kube manager 2024-08-30 02:02:06,006 - INFO - Service ctest-nginx-svc-70591820 verification passed 2024-08-30 02:02:06,071 - DEBUG - [Pod ctest-pod-16886868] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: ctest-pod-50396244 Connecting to 10.107.224.188:80 (10.107.224.188:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-30 02:02:06,072 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 passed 2024-08-30 02:02:06,072 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd output: ctest-pod-50396244 Connecting to 10.107.224.188:80 (10.107.224.188:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-30 02:02:06,138 - DEBUG - [Pod ctest-pod-16886868] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: ctest-pod-50396244 Connecting to 10.107.224.188:80 (10.107.224.188:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-30 02:02:06,138 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 passed 2024-08-30 02:02:06,138 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd output: ctest-pod-50396244 Connecting to 10.107.224.188:80 (10.107.224.188:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-30 02:02:06,191 - DEBUG - [Pod ctest-pod-16886868] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.224.188:80 (10.107.224.188:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-50396244 2024-08-30 02:02:06,191 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 passed 2024-08-30 02:02:06,191 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd output: Connecting to 10.107.224.188:80 (10.107.224.188:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-50396244 2024-08-30 02:02:06,262 - DEBUG - [Pod ctest-pod-16886868] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: ctest-pod-30237662 Connecting to 10.107.224.188:80 (10.107.224.188:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-30 02:02:06,262 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 passed 2024-08-30 02:02:06,262 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd output: ctest-pod-30237662 Connecting to 10.107.224.188:80 (10.107.224.188:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-30 02:02:06,263 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-50396244': 3, 'ctest-pod-30237662': 1} 2024-08-30 02:02:06,333 - DEBUG - [Pod ctest-pod-48042648] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: ctest-pod-50396244 Connecting to 10.107.224.188:80 (10.107.224.188:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-30 02:02:06,333 - DEBUG - [Pod Pod ctest-pod-48042648] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 passed 2024-08-30 02:02:06,333 - DEBUG - [Pod Pod ctest-pod-48042648] Cmd output: ctest-pod-50396244 Connecting to 10.107.224.188:80 (10.107.224.188:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-30 02:02:08,917 - DEBUG - [Pod ctest-pod-48042648] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: ctest-pod-30237662 Connecting to 10.107.224.188:80 (10.107.224.188:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-30 02:02:08,917 - DEBUG - [Pod Pod ctest-pod-48042648] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 passed 2024-08-30 02:02:08,917 - DEBUG - [Pod Pod ctest-pod-48042648] Cmd output: ctest-pod-30237662 Connecting to 10.107.224.188:80 (10.107.224.188:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-30 02:02:08,917 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-50396244': 1, 'ctest-pod-30237662': 1} 2024-08-30 02:02:08,921 - DEBUG - Network policy ingress-on-service not present 2024-08-30 02:02:08,922 - INFO - Creating Network Policy ingress-on-service 2024-08-30 02:02:09,105 - INFO - Network policy found in k8s 2024-08-30 02:02:09,105 - DEBUG - Requesting: http://10.0.0.56:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=6e3c9314-2831-43c7-83e3-e42e1c1abc56 2024-08-30 02:02:09,109 - INFO - Network Policy ingress-on-service with uuid 6e3c9314-2831-43c7-83e3-e42e1c1abc56 found in kube manager 2024-08-30 02:02:09,143 - INFO - Network policy with name default-policy-management:k8s-allowall found in agent 2024-08-30 02:02:09,143 - INFO - Network policy with name default-policy-management:k8s-allowall associated with default ks8 2024-08-30 02:02:09,143 - DEBUG - Requesting: http://10.0.0.56:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=6e3c9314-2831-43c7-83e3-e42e1c1abc56 2024-08-30 02:02:09,179 - WARNING - Network policy with name ingress-on-service not found in agent 2024-08-30 02:02:11,180 - DEBUG - Requesting: http://10.0.0.56:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=6e3c9314-2831-43c7-83e3-e42e1c1abc56 2024-08-30 02:02:11,210 - INFO - Network policy with name ingress-on-service found in agent 2024-08-30 02:02:11,210 - INFO - Network Policy ingress-on-service verification passed 2024-08-30 02:02:16,259 - DEBUG - [Pod ctest-pod-16886868] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:02:16,259 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:02:16,259 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:02:21,311 - DEBUG - [Pod ctest-pod-16886868] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:02:21,311 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:02:21,311 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:02:26,367 - DEBUG - [Pod ctest-pod-16886868] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:02:26,367 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:02:26,367 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:02:31,418 - DEBUG - [Pod ctest-pod-16886868] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:02:31,418 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:02:31,418 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:02:36,474 - DEBUG - [Pod ctest-pod-16886868] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:02:36,474 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:02:36,474 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:02:41,530 - DEBUG - [Pod ctest-pod-16886868] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:02:41,530 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:02:41,530 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:02:47,131 - DEBUG - [Pod ctest-pod-16886868] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:02:47,131 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:02:47,131 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:02:52,203 - DEBUG - [Pod ctest-pod-16886868] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:02:52,203 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:02:52,203 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:02:57,259 - DEBUG - [Pod ctest-pod-16886868] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:02:57,259 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:02:57,259 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:03:03,694 - DEBUG - [Pod ctest-pod-16886868] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:03:03,694 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:03:03,694 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:03:03,694 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-50396244': 0, 'ctest-pod-30237662': 0} 2024-08-30 02:03:03,755 - DEBUG - [Pod ctest-pod-45140463] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.224.188:80 (10.107.224.188:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-50396244 2024-08-30 02:03:03,755 - DEBUG - [Pod Pod ctest-pod-45140463] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 passed 2024-08-30 02:03:03,755 - DEBUG - [Pod Pod ctest-pod-45140463] Cmd output: Connecting to 10.107.224.188:80 (10.107.224.188:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-50396244 2024-08-30 02:03:03,823 - DEBUG - [Pod ctest-pod-45140463] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: ctest-pod-50396244 Connecting to 10.107.224.188:80 (10.107.224.188:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-30 02:03:03,823 - DEBUG - [Pod Pod ctest-pod-45140463] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 passed 2024-08-30 02:03:03,823 - DEBUG - [Pod Pod ctest-pod-45140463] Cmd output: ctest-pod-50396244 Connecting to 10.107.224.188:80 (10.107.224.188:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-30 02:03:03,865 - DEBUG - [Pod ctest-pod-45140463] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: ctest-pod-50396244 Connecting to 10.107.224.188:80 (10.107.224.188:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-30 02:03:03,865 - DEBUG - [Pod Pod ctest-pod-45140463] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 passed 2024-08-30 02:03:03,865 - DEBUG - [Pod Pod ctest-pod-45140463] Cmd output: ctest-pod-50396244 Connecting to 10.107.224.188:80 (10.107.224.188:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-30 02:03:03,917 - DEBUG - [Pod ctest-pod-45140463] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: ctest-pod-50396244 Connecting to 10.107.224.188:80 (10.107.224.188:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-30 02:03:03,917 - DEBUG - [Pod Pod ctest-pod-45140463] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 passed 2024-08-30 02:03:03,917 - DEBUG - [Pod Pod ctest-pod-45140463] Cmd output: ctest-pod-50396244 Connecting to 10.107.224.188:80 (10.107.224.188:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-30 02:03:03,960 - DEBUG - [Pod ctest-pod-45140463] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: ctest-pod-50396244 Connecting to 10.107.224.188:80 (10.107.224.188:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-30 02:03:03,960 - DEBUG - [Pod Pod ctest-pod-45140463] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 passed 2024-08-30 02:03:03,960 - DEBUG - [Pod Pod ctest-pod-45140463] Cmd output: ctest-pod-50396244 Connecting to 10.107.224.188:80 (10.107.224.188:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-30 02:03:04,011 - DEBUG - [Pod ctest-pod-45140463] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: ctest-pod-50396244 Connecting to 10.107.224.188:80 (10.107.224.188:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-30 02:03:04,011 - DEBUG - [Pod Pod ctest-pod-45140463] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 passed 2024-08-30 02:03:04,011 - DEBUG - [Pod Pod ctest-pod-45140463] Cmd output: ctest-pod-50396244 Connecting to 10.107.224.188:80 (10.107.224.188:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-30 02:03:04,061 - DEBUG - [Pod ctest-pod-45140463] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: ctest-pod-30237662 Connecting to 10.107.224.188:80 (10.107.224.188:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-30 02:03:04,061 - DEBUG - [Pod Pod ctest-pod-45140463] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 passed 2024-08-30 02:03:04,061 - DEBUG - [Pod Pod ctest-pod-45140463] Cmd output: ctest-pod-30237662 Connecting to 10.107.224.188:80 (10.107.224.188:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-30 02:03:04,061 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-50396244': 6, 'ctest-pod-30237662': 1} 2024-08-30 02:03:09,114 - DEBUG - [Pod ctest-pod-48042648] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:03:09,114 - DEBUG - [Pod Pod ctest-pod-48042648] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:03:09,114 - DEBUG - [Pod Pod ctest-pod-48042648] Cmd output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:03:14,170 - DEBUG - [Pod ctest-pod-48042648] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:03:14,170 - DEBUG - [Pod Pod ctest-pod-48042648] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:03:14,170 - DEBUG - [Pod Pod ctest-pod-48042648] Cmd output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:03:19,234 - DEBUG - [Pod ctest-pod-48042648] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:03:19,234 - DEBUG - [Pod Pod ctest-pod-48042648] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:03:19,234 - DEBUG - [Pod Pod ctest-pod-48042648] Cmd output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:03:24,282 - DEBUG - [Pod ctest-pod-48042648] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:03:24,283 - DEBUG - [Pod Pod ctest-pod-48042648] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:03:24,283 - DEBUG - [Pod Pod ctest-pod-48042648] Cmd output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:03:29,331 - DEBUG - [Pod ctest-pod-48042648] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:03:29,331 - DEBUG - [Pod Pod ctest-pod-48042648] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:03:29,331 - DEBUG - [Pod Pod ctest-pod-48042648] Cmd output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:03:34,394 - DEBUG - [Pod ctest-pod-48042648] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:03:34,394 - DEBUG - [Pod Pod ctest-pod-48042648] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:03:34,394 - DEBUG - [Pod Pod ctest-pod-48042648] Cmd output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:03:39,458 - DEBUG - [Pod ctest-pod-48042648] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:03:39,458 - DEBUG - [Pod Pod ctest-pod-48042648] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:03:39,459 - DEBUG - [Pod Pod ctest-pod-48042648] Cmd output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:03:45,972 - DEBUG - [Pod ctest-pod-48042648] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:03:45,973 - DEBUG - [Pod Pod ctest-pod-48042648] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:03:45,973 - DEBUG - [Pod Pod ctest-pod-48042648] Cmd output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:03:51,024 - DEBUG - [Pod ctest-pod-48042648] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:03:51,025 - DEBUG - [Pod Pod ctest-pod-48042648] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:03:51,025 - DEBUG - [Pod Pod ctest-pod-48042648] Cmd output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:03:56,092 - DEBUG - [Pod ctest-pod-48042648] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:03:56,092 - DEBUG - [Pod Pod ctest-pod-48042648] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:03:56,092 - DEBUG - [Pod Pod ctest-pod-48042648] Cmd output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:03:56,093 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-50396244': 0, 'ctest-pod-30237662': 0} 2024-08-30 02:03:58,143 - DEBUG - [Pod ctest-pod-45140463] Cmd: ping -c 3 10.47.255.247, Output: PING 10.47.255.247 (10.47.255.247): 56 data bytes 64 bytes from 10.47.255.247: seq=0 ttl=63 time=1.111 ms 64 bytes from 10.47.255.247: seq=1 ttl=63 time=0.078 ms 64 bytes from 10.47.255.247: seq=2 ttl=63 time=0.089 ms --- 10.47.255.247 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.078/0.426/1.111 ms 2024-08-30 02:03:58,143 - INFO - Ping check to IP 10.47.255.247 from pod ctest-pod-45140463 with expectation True passed 2024-08-30 02:04:00,207 - DEBUG - [Pod ctest-pod-16886868] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=1.256 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.082 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.069 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.069/0.469/1.256 ms 2024-08-30 02:04:00,207 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-16886868 with expectation True passed 2024-08-30 02:04:02,256 - DEBUG - [Pod ctest-pod-48042648] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=1.079 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.081 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.072 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.072/0.410/1.079 ms 2024-08-30 02:04:02,256 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-48042648 with expectation True passed 2024-08-30 02:04:02,320 - DEBUG - [Pod ctest-pod-45140463] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-08-30 02:04:02,320 - DEBUG - [Pod Pod ctest-pod-45140463] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2024-08-30 02:04:02,320 - DEBUG - [Pod Pod ctest-pod-45140463] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-08-30 02:04:02,320 - INFO - wget check of of http://10.47.255.251 from ctest-pod-45140463 passed 2024-08-30 02:04:02,378 - DEBUG - [Pod ctest-pod-45140463] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-08-30 02:04:02,378 - DEBUG - [Pod Pod ctest-pod-45140463] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2024-08-30 02:04:02,378 - DEBUG - [Pod Pod ctest-pod-45140463] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-08-30 02:04:02,378 - INFO - wget check of of http://10.47.255.250 from ctest-pod-45140463 passed 2024-08-30 02:04:09,845 - DEBUG - [Pod ctest-pod-16886868] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-08-30 02:04:09,845 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-08-30 02:04:09,845 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-08-30 02:04:09,845 - INFO - wget check of of http://10.47.255.250 from ctest-pod-16886868 passed 2024-08-30 02:04:14,907 - DEBUG - [Pod ctest-pod-53278191] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) wget: download timed out 2024-08-30 02:04:14,907 - DEBUG - [Pod Pod ctest-pod-53278191] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.251 (10.47.255.251:80) wget: download timed out 2024-08-30 02:04:14,907 - DEBUG - [Pod Pod ctest-pod-53278191] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) wget: download timed out 2024-08-30 02:04:14,908 - INFO - wget check of of http://10.47.255.251 from ctest-pod-53278191 passed 2024-08-30 02:04:14,915 - INFO - Updating Network Policy ingress-on-service 2024-08-30 02:04:19,983 - DEBUG - [Pod ctest-pod-16886868] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:04:19,983 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:04:19,984 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:04:25,050 - DEBUG - [Pod ctest-pod-16886868] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:04:25,050 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:04:25,050 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:04:30,102 - DEBUG - [Pod ctest-pod-16886868] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:04:30,102 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:04:30,102 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:04:35,143 - DEBUG - [Pod ctest-pod-16886868] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:04:35,143 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:04:35,143 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:04:40,195 - DEBUG - [Pod ctest-pod-16886868] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:04:40,195 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:04:40,195 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:04:47,324 - DEBUG - [Pod ctest-pod-16886868] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:04:48,401 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:04:48,401 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:04:53,478 - DEBUG - [Pod ctest-pod-16886868] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:04:53,478 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:04:53,478 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:04:59,638 - DEBUG - [Pod ctest-pod-16886868] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:04:59,638 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:04:59,638 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:05:07,771 - DEBUG - [Pod ctest-pod-16886868] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:05:07,771 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:05:07,771 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:05:13,131 - DEBUG - [Pod ctest-pod-16886868] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:05:13,131 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:05:13,132 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd output: Connecting to 10.107.224.188:80 (10.107.224.188:80) wget: download timed out 2024-08-30 02:05:13,132 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-50396244': 0, 'ctest-pod-30237662': 0} 2024-08-30 02:05:13,940 - DEBUG - [Pod ctest-pod-45140463] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: ctest-pod-50396244 Connecting to 10.107.224.188:80 (10.107.224.188:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-30 02:05:13,940 - DEBUG - [Pod Pod ctest-pod-45140463] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 passed 2024-08-30 02:05:13,940 - DEBUG - [Pod Pod ctest-pod-45140463] Cmd output: ctest-pod-50396244 Connecting to 10.107.224.188:80 (10.107.224.188:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-30 02:05:14,006 - DEBUG - [Pod ctest-pod-45140463] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: ctest-pod-30237662 Connecting to 10.107.224.188:80 (10.107.224.188:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-30 02:05:14,006 - DEBUG - [Pod Pod ctest-pod-45140463] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 passed 2024-08-30 02:05:14,006 - DEBUG - [Pod Pod ctest-pod-45140463] Cmd output: ctest-pod-30237662 Connecting to 10.107.224.188:80 (10.107.224.188:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-30 02:05:14,006 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-50396244': 1, 'ctest-pod-30237662': 1} 2024-08-30 02:05:14,084 - DEBUG - [Pod ctest-pod-48042648] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: ctest-pod-50396244 Connecting to 10.107.224.188:80 (10.107.224.188:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-30 02:05:14,084 - DEBUG - [Pod Pod ctest-pod-48042648] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 passed 2024-08-30 02:05:14,084 - DEBUG - [Pod Pod ctest-pod-48042648] Cmd output: ctest-pod-50396244 Connecting to 10.107.224.188:80 (10.107.224.188:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-30 02:05:14,147 - DEBUG - [Pod ctest-pod-48042648] Cmd: wget http://10.107.224.188:80/ -O - -T 5 -t 1 , Output: ctest-pod-30237662 Connecting to 10.107.224.188:80 (10.107.224.188:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-30 02:05:14,148 - DEBUG - [Pod Pod ctest-pod-48042648] Cmd wget http://10.107.224.188:80/ -O - -T 5 -t 1 passed 2024-08-30 02:05:14,148 - DEBUG - [Pod Pod ctest-pod-48042648] Cmd output: ctest-pod-30237662 Connecting to 10.107.224.188:80 (10.107.224.188:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-08-30 02:05:14,148 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-50396244': 1, 'ctest-pod-30237662': 1} 2024-08-30 02:05:16,195 - DEBUG - [Pod ctest-pod-45140463] Cmd: ping -c 3 10.47.255.247, Output: PING 10.47.255.247 (10.47.255.247): 56 data bytes 64 bytes from 10.47.255.247: seq=0 ttl=63 time=0.835 ms 64 bytes from 10.47.255.247: seq=1 ttl=63 time=0.080 ms 64 bytes from 10.47.255.247: seq=2 ttl=63 time=0.071 ms --- 10.47.255.247 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.071/0.328/0.835 ms 2024-08-30 02:05:16,212 - INFO - Ping check to IP 10.47.255.247 from pod ctest-pod-45140463 with expectation True passed 2024-08-30 02:05:18,278 - DEBUG - [Pod ctest-pod-16886868] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=0.860 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.068 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.075 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.068/0.334/0.860 ms 2024-08-30 02:05:18,279 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-16886868 with expectation True passed 2024-08-30 02:05:20,334 - DEBUG - [Pod ctest-pod-48042648] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=0.603 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.072 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.092 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.072/0.255/0.603 ms 2024-08-30 02:05:20,334 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-48042648 with expectation True passed 2024-08-30 02:05:20,397 - DEBUG - [Pod ctest-pod-45140463] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-08-30 02:05:20,397 - DEBUG - [Pod Pod ctest-pod-45140463] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2024-08-30 02:05:20,397 - DEBUG - [Pod Pod ctest-pod-45140463] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-08-30 02:05:20,397 - INFO - wget check of of http://10.47.255.251 from ctest-pod-45140463 passed 2024-08-30 02:05:20,461 - DEBUG - [Pod ctest-pod-45140463] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-08-30 02:05:20,461 - DEBUG - [Pod Pod ctest-pod-45140463] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2024-08-30 02:05:20,461 - DEBUG - [Pod Pod ctest-pod-45140463] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-08-30 02:05:20,461 - INFO - wget check of of http://10.47.255.250 from ctest-pod-45140463 passed 2024-08-30 02:05:25,520 - DEBUG - [Pod ctest-pod-16886868] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-08-30 02:05:25,520 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-08-30 02:05:25,520 - DEBUG - [Pod Pod ctest-pod-16886868] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-08-30 02:05:25,520 - INFO - wget check of of http://10.47.255.250 from ctest-pod-16886868 passed 2024-08-30 02:05:27,638 - DEBUG - [Pod ctest-pod-53278191] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-08-30 02:05:27,638 - DEBUG - [Pod Pod ctest-pod-53278191] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2024-08-30 02:05:27,638 - DEBUG - [Pod Pod ctest-pod-53278191] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-08-30 02:05:27,638 - INFO - wget check of of http://10.47.255.251 from ctest-pod-53278191 passed 2024-08-30 02:05:27,639 - INFO - Deleting Network Policy : ingress-on-service 2024-08-30 02:05:27,639 - INFO - Deleting service : ctest-nginx-svc-70591820 2024-08-30 02:05:27,832 - DEBUG - No XMPP flaps were noticed during the test
2024-08-30 02:05:27,832 - INFO - END TEST : test_ingress_rule_on_pod_with_service : PASSED[0:03:27]
2024-08-30 02:05:27,832 - INFO - -------------------------------------------------------------------------------- 2024-08-30 02:05:27,837 - INFO - Deleting pod ctest-new-default-93885893:ctest-pod-50396244 2024-08-30 02:05:27,838 - INFO - Deleting pod ctest-new-default-93885893:ctest-pod-30237662 2024-08-30 02:05:27,838 - INFO - Deleting pod ctest-non-default-91187833:ctest-pod-23425777 2024-08-30 02:05:27,839 - INFO - Deleting pod ctest-non-default-91187833:ctest-pod-16572061 2024-08-30 02:05:27,839 - INFO - Deleting pod ctest-new-default-93885893:ctest-pod-16886868 2024-08-30 02:05:27,839 - INFO - Deleting pod ctest-new-default-93885893:ctest-pod-45140463 2024-08-30 02:05:27,840 - INFO - Deleting pod ctest-non-default-91187833:ctest-pod-48042648 2024-08-30 02:05:27,840 - INFO - Deleting pod ctest-non-default-91187833:ctest-pod-53278191 2024-08-30 02:05:28,002 - WARNING - Pod uuid b20c1d45-7b84-43ba-beee-0b50bad38058 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:05:28,003 - WARNING - Pod uuid 927868a4-8166-40fc-a6f3-8265808d4f93 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:05:28,004 - WARNING - Pod uuid ab9397a6-bab3-4abf-9d2c-7863286e8810 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:05:28,007 - WARNING - Pod uuid 46b082ce-8664-4dee-b042-41f1748673d0 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:05:28,008 - WARNING - Pod uuid 32b2db15-50b9-475d-a973-3ceb69afaf16 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:05:28,008 - WARNING - Pod uuid 392a5083-66b5-4146-b686-8cf5e79821da is still seen in agent 10.0.0.45 VM list 2024-08-30 02:05:28,009 - WARNING - Pod uuid 7ad1ad09-8099-462c-9fed-e59c07c8dd09 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:05:28,010 - WARNING - Pod uuid 44bddf59-4891-4da1-8d9c-d909ea37ff14 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:05:33,035 - WARNING - Pod uuid 44bddf59-4891-4da1-8d9c-d909ea37ff14 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:05:33,036 - WARNING - Pod uuid 46b082ce-8664-4dee-b042-41f1748673d0 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:05:33,036 - WARNING - Pod uuid 927868a4-8166-40fc-a6f3-8265808d4f93 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:05:33,037 - WARNING - Pod uuid ab9397a6-bab3-4abf-9d2c-7863286e8810 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:05:33,037 - WARNING - Pod uuid b20c1d45-7b84-43ba-beee-0b50bad38058 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:05:33,038 - WARNING - Pod uuid 32b2db15-50b9-475d-a973-3ceb69afaf16 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:05:33,039 - WARNING - Pod uuid 7ad1ad09-8099-462c-9fed-e59c07c8dd09 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:05:33,039 - WARNING - Pod uuid 392a5083-66b5-4146-b686-8cf5e79821da is still seen in agent 10.0.0.45 VM list 2024-08-30 02:05:38,064 - WARNING - Pod uuid 927868a4-8166-40fc-a6f3-8265808d4f93 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:05:38,067 - WARNING - Pod uuid b20c1d45-7b84-43ba-beee-0b50bad38058 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:05:38,067 - WARNING - Pod uuid 32b2db15-50b9-475d-a973-3ceb69afaf16 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:05:38,067 - WARNING - Pod uuid 46b082ce-8664-4dee-b042-41f1748673d0 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:05:38,068 - WARNING - Pod uuid ab9397a6-bab3-4abf-9d2c-7863286e8810 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:05:38,068 - DEBUG - Pod 392a5083-66b5-4146-b686-8cf5e79821da is not in agent 10.0.0.45 VM list 2024-08-30 02:05:38,069 - WARNING - Pod uuid 44bddf59-4891-4da1-8d9c-d909ea37ff14 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:05:38,074 - DEBUG - VMI ce702436-6673-11ef-b0d4-fa163eb2fd2f is removed from agent 10.0.0.45 2024-08-30 02:05:38,074 - INFO - Verified that pod ctest-pod-16572061 is removed in agent 2024-08-30 02:05:39,063 - WARNING - Pod uuid 7ad1ad09-8099-462c-9fed-e59c07c8dd09 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:05:43,076 - DEBUG - Pod 927868a4-8166-40fc-a6f3-8265808d4f93 is not in agent 10.0.0.45 VM list 2024-08-30 02:05:43,092 - WARNING - Pod uuid 46b082ce-8664-4dee-b042-41f1748673d0 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:05:43,092 - DEBUG - Pod 32b2db15-50b9-475d-a973-3ceb69afaf16 is not in agent 10.0.0.45 VM list 2024-08-30 02:05:43,096 - WARNING - Pod uuid 44bddf59-4891-4da1-8d9c-d909ea37ff14 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:05:43,096 - DEBUG - VMI cd751e74-6673-11ef-b0d4-fa163eb2fd2f is removed from agent 10.0.0.45 2024-08-30 02:05:43,096 - INFO - Verified that pod ctest-pod-30237662 is removed in agent 2024-08-30 02:05:43,097 - DEBUG - Pod b20c1d45-7b84-43ba-beee-0b50bad38058 is not in agent 10.0.0.45 VM list 2024-08-30 02:05:43,098 - WARNING - Pod uuid ab9397a6-bab3-4abf-9d2c-7863286e8810 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:05:43,105 - DEBUG - VMI cd0b81b2-6673-11ef-b0d4-fa163eb2fd2f is removed from agent 10.0.0.45 2024-08-30 02:05:43,105 - INFO - Verified that pod ctest-pod-50396244 is removed in agent 2024-08-30 02:05:43,106 - DEBUG - VMI ce05dd24-6673-11ef-b0d4-fa163eb2fd2f is removed from agent 10.0.0.45 2024-08-30 02:05:43,106 - INFO - Verified that pod ctest-pod-23425777 is removed in agent 2024-08-30 02:05:44,071 - WARNING - Pod uuid 7ad1ad09-8099-462c-9fed-e59c07c8dd09 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:05:48,101 - WARNING - Pod uuid 46b082ce-8664-4dee-b042-41f1748673d0 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:05:48,108 - WARNING - Pod uuid ab9397a6-bab3-4abf-9d2c-7863286e8810 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:05:48,109 - WARNING - Pod uuid 44bddf59-4891-4da1-8d9c-d909ea37ff14 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:05:49,079 - WARNING - Pod uuid 7ad1ad09-8099-462c-9fed-e59c07c8dd09 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:05:53,111 - WARNING - Pod uuid 46b082ce-8664-4dee-b042-41f1748673d0 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:05:53,320 - WARNING - Pod uuid ab9397a6-bab3-4abf-9d2c-7863286e8810 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:05:53,321 - WARNING - Pod uuid 44bddf59-4891-4da1-8d9c-d909ea37ff14 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:05:54,114 - WARNING - Pod uuid 7ad1ad09-8099-462c-9fed-e59c07c8dd09 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:06:00,815 - WARNING - Pod uuid ab9397a6-bab3-4abf-9d2c-7863286e8810 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:06:00,817 - WARNING - Pod uuid 44bddf59-4891-4da1-8d9c-d909ea37ff14 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:06:00,818 - WARNING - Pod uuid 7ad1ad09-8099-462c-9fed-e59c07c8dd09 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:06:00,883 - WARNING - Pod uuid 46b082ce-8664-4dee-b042-41f1748673d0 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:06:05,830 - WARNING - Pod uuid ab9397a6-bab3-4abf-9d2c-7863286e8810 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:06:05,832 - WARNING - Pod uuid 7ad1ad09-8099-462c-9fed-e59c07c8dd09 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:06:05,832 - WARNING - Pod uuid 44bddf59-4891-4da1-8d9c-d909ea37ff14 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:06:06,188 - WARNING - Pod uuid 46b082ce-8664-4dee-b042-41f1748673d0 is still seen in agent 10.0.0.45 VM list 2024-08-30 02:06:10,845 - DEBUG - Pod ab9397a6-bab3-4abf-9d2c-7863286e8810 is not in agent 10.0.0.45 VM list 2024-08-30 02:06:10,848 - DEBUG - Pod 44bddf59-4891-4da1-8d9c-d909ea37ff14 is not in agent 10.0.0.45 VM list 2024-08-30 02:06:10,850 - DEBUG - Pod 7ad1ad09-8099-462c-9fed-e59c07c8dd09 is not in agent 10.0.0.45 VM list 2024-08-30 02:06:10,860 - DEBUG - VMI cf6967d0-6673-11ef-b0d4-fa163eb2fd2f is removed from agent 10.0.0.45 2024-08-30 02:06:10,860 - INFO - Verified that pod ctest-pod-45140463 is removed in agent 2024-08-30 02:06:10,861 - DEBUG - VMI cef2909c-6673-11ef-b0d4-fa163eb2fd2f is removed from agent 10.0.0.45 2024-08-30 02:06:10,861 - INFO - Verified that pod ctest-pod-16886868 is removed in agent 2024-08-30 02:06:10,862 - DEBUG - VMI cfe66532-6673-11ef-b0d4-fa163eb2fd2f is removed from agent 10.0.0.45 2024-08-30 02:06:10,862 - INFO - Verified that pod ctest-pod-48042648 is removed in agent 2024-08-30 02:06:11,196 - DEBUG - Pod 46b082ce-8664-4dee-b042-41f1748673d0 is not in agent 10.0.0.45 VM list 2024-08-30 02:06:12,203 - DEBUG - VMI d06e549c-6673-11ef-b0d4-fa163eb2fd2f is removed from agent 10.0.0.45 2024-08-30 02:06:12,203 - INFO - Verified that pod ctest-pod-53278191 is removed in agent 2024-08-30 02:06:12,204 - INFO - Deleting namespace ctest-non-default-91187833 2024-08-30 02:06:12,213 - DEBUG - Namespace ctest-non-default-91187833 still in k8s 2024-08-30 02:06:14,219 - DEBUG - Namespace ctest-non-default-91187833 still in k8s 2024-08-30 02:06:16,222 - DEBUG - Namespace ctest-non-default-91187833 still in k8s 2024-08-30 02:06:19,808 - DEBUG - Namespace ctest-non-default-91187833 is not in k8s 2024-08-30 02:06:19,808 - DEBUG - No need to check project deletion in contrail-api 2024-08-30 02:06:19,808 - INFO - Deleting namespace ctest-new-default-93885893 2024-08-30 02:06:19,821 - DEBUG - Namespace ctest-new-default-93885893 still in k8s 2024-08-30 02:06:21,826 - DEBUG - Namespace ctest-new-default-93885893 still in k8s 2024-08-30 02:06:23,831 - DEBUG - Namespace ctest-new-default-93885893 still in k8s 2024-08-30 02:06:25,834 - DEBUG - Namespace ctest-new-default-93885893 is not in k8s 2024-08-30 02:06:25,834 - DEBUG - No need to check project deletion in contrail-api