2024-11-04 03:28:54,659 - INFO - Using existing project ['default-domain', 'k8s-default'](7a2e9f33-d28a-4806-bfcb-cc0cbfd970db) 2024-11-04 03:28:54,680 - DEBUG - Namespace ctest-new-default-58657994 not present 2024-11-04 03:28:54,688 - INFO - Created namespace ctest-new-default-58657994 2024-11-04 03:28:57,692 - DEBUG - Namespace ctest-non-default-94017682 not present 2024-11-04 03:28:57,707 - INFO - Created namespace ctest-non-default-94017682 2024-11-04 03:29:00,731 - DEBUG - Pod ctest-pod-41922036 not present 2024-11-04 03:29:00,732 - INFO - Creating Pod ctest-pod-41922036 2024-11-04 03:29:00,758 - DEBUG - Pod : ctest-pod-41922036 UUID is e0db9ae1-0a03-4cf8-8a6b-6e9c7168f428 2024-11-04 03:29:00,761 - DEBUG - Pod ctest-pod-13443166 not present 2024-11-04 03:29:00,762 - INFO - Creating Pod ctest-pod-13443166 2024-11-04 03:29:00,790 - DEBUG - Pod : ctest-pod-13443166 UUID is d5459880-3fdc-43f7-8519-a54d0e2f9e7f 2024-11-04 03:29:00,795 - DEBUG - Pod ctest-pod-71835920 not present 2024-11-04 03:29:00,796 - INFO - Creating Pod ctest-pod-71835920 2024-11-04 03:29:00,845 - DEBUG - Pod : ctest-pod-71835920 UUID is 5a226b82-7523-4f5c-8194-2ee805791967 2024-11-04 03:29:00,849 - DEBUG - Pod ctest-pod-83732116 not present 2024-11-04 03:29:00,849 - INFO - Creating Pod ctest-pod-83732116 2024-11-04 03:29:00,879 - DEBUG - Pod : ctest-pod-83732116 UUID is 3f49c63c-832b-46c6-8054-779eaae7e5b4 2024-11-04 03:29:00,881 - DEBUG - Pod ctest-pod-31456432 not present 2024-11-04 03:29:00,881 - INFO - Creating Pod ctest-pod-31456432 2024-11-04 03:29:00,905 - DEBUG - Pod : ctest-pod-31456432 UUID is d3f1c76c-b501-4ac8-b3f2-5ad75ac78d4c 2024-11-04 03:29:00,908 - DEBUG - Pod ctest-pod-34493036 not present 2024-11-04 03:29:00,908 - INFO - Creating Pod ctest-pod-34493036 2024-11-04 03:29:00,925 - DEBUG - Pod : ctest-pod-34493036 UUID is 90d6953b-8679-464c-835c-d8e68c232def 2024-11-04 03:29:00,928 - DEBUG - Pod ctest-pod-70615538 not present 2024-11-04 03:29:00,928 - INFO - Creating Pod ctest-pod-70615538 2024-11-04 03:29:00,951 - DEBUG - Pod : ctest-pod-70615538 UUID is 0c54170e-6329-4a50-8c46-2314943eacfa 2024-11-04 03:29:00,954 - DEBUG - Pod ctest-pod-51035490 not present 2024-11-04 03:29:00,954 - INFO - Creating Pod ctest-pod-51035490 2024-11-04 03:29:00,981 - DEBUG - Pod : ctest-pod-51035490 UUID is c2758868-14ee-4dfb-8802-fad17c225ee7 2024-11-04 03:29:01,171 - INFO - Project default-domain:k8s-ctest-new-default-58657994 for Namespace ctest-new-default-58657994 is seen in contrail-api 2024-11-04 03:29:01,171 - DEBUG - Project uuid in Contrail is babc76d8-0193-447c-94a8-5d4c92f80b6f 2024-11-04 03:29:01,363 - DEBUG - Requesting: http://10.0.0.28:8108/Snh_NamespaceDatabaseList?namespace_uuid=d22a8bbf-e5ab-4731-a1b0-5206a4b6bdec 2024-11-04 03:29:01,368 - INFO - Namespace ctest-new-default-58657994 with uuid d22a8bbf-e5ab-4731-a1b0-5206a4b6bdec found in kube manager 2024-11-04 03:29:01,368 - INFO - Namespace ctest-new-default-58657994 verification passed 2024-11-04 03:29:01,625 - INFO - Project default-domain:k8s-ctest-non-default-94017682 for Namespace ctest-non-default-94017682 is seen in contrail-api 2024-11-04 03:29:01,625 - DEBUG - Project uuid in Contrail is 394a3232-c30c-4205-b790-877ad04cdc0e 2024-11-04 03:29:01,625 - DEBUG - Requesting: http://10.0.0.28:8108/Snh_NamespaceDatabaseList?namespace_uuid=688675fc-a554-4ced-a10d-1f2a6e212ba1 2024-11-04 03:29:01,630 - INFO - Namespace ctest-non-default-94017682 with uuid 688675fc-a554-4ced-a10d-1f2a6e212ba1 found in kube manager 2024-11-04 03:29:01,630 - INFO - Namespace ctest-non-default-94017682 verification passed 2024-11-04 03:29:01,637 - DEBUG - Pod ctest-pod-41922036 not in running state.Currently in Pending 2024-11-04 03:29:06,650 - DEBUG - Pod ctest-pod-41922036 not in running state.Currently in Pending 2024-11-04 03:29:11,656 - DEBUG - Pod ctest-pod-41922036 not in running state.Currently in Pending 2024-11-04 03:29:16,664 - INFO - Pod ctest-pod-41922036 is in running state.Got IP 10.47.255.251 2024-11-04 03:29:16,684 - DEBUG - Pod ctest-pod-41922036 has vmi f021ba46-9a5c-11ef-b331-fa163e954dcf 2024-11-04 03:29:16,684 - INFO - Verified pod ctest-pod-41922036 in contrail-api 2024-11-04 03:29:16,717 - DEBUG - VMI f021ba46-9a5c-11ef-b331-fa163e954dcf is active in agent 10.0.0.31 2024-11-04 03:29:16,717 - INFO - Verified Pod ctest-pod-41922036 in agent 10.0.0.31 2024-11-04 03:29:16,717 - DEBUG - Requesting: http://10.0.0.28:8108/Snh_VirtualMachineDatabaseList?x=e0db9ae1-0a03-4cf8-8a6b-6e9c7168f428 2024-11-04 03:29:16,726 - INFO - Pod ctest-pod-41922036 with uuid e0db9ae1-0a03-4cf8-8a6b-6e9c7168f428 found in kube manager 2024-11-04 03:29:16,726 - INFO - Pod ctest-pod-41922036 verification passed 2024-11-04 03:29:16,731 - INFO - Pod ctest-pod-13443166 is in running state.Got IP 10.47.255.250 2024-11-04 03:29:16,790 - DEBUG - Pod ctest-pod-13443166 has vmi f0ae4128-9a5c-11ef-b331-fa163e954dcf 2024-11-04 03:29:16,790 - INFO - Verified pod ctest-pod-13443166 in contrail-api 2024-11-04 03:29:16,807 - DEBUG - VMI f0ae4128-9a5c-11ef-b331-fa163e954dcf is active in agent 10.0.0.31 2024-11-04 03:29:16,809 - INFO - Verified Pod ctest-pod-13443166 in agent 10.0.0.31 2024-11-04 03:29:16,809 - DEBUG - Requesting: http://10.0.0.28:8108/Snh_VirtualMachineDatabaseList?x=d5459880-3fdc-43f7-8519-a54d0e2f9e7f 2024-11-04 03:29:16,815 - INFO - Pod ctest-pod-13443166 with uuid d5459880-3fdc-43f7-8519-a54d0e2f9e7f found in kube manager 2024-11-04 03:29:16,815 - INFO - Pod ctest-pod-13443166 verification passed 2024-11-04 03:29:16,823 - INFO - Pod ctest-pod-71835920 is in running state.Got IP 10.47.255.249 2024-11-04 03:29:16,850 - DEBUG - Pod ctest-pod-71835920 has vmi f165d856-9a5c-11ef-b331-fa163e954dcf 2024-11-04 03:29:16,850 - INFO - Verified pod ctest-pod-71835920 in contrail-api 2024-11-04 03:29:16,864 - DEBUG - VMI f165d856-9a5c-11ef-b331-fa163e954dcf is active in agent 10.0.0.31 2024-11-04 03:29:16,864 - INFO - Verified Pod ctest-pod-71835920 in agent 10.0.0.31 2024-11-04 03:29:16,864 - DEBUG - Requesting: http://10.0.0.28:8108/Snh_VirtualMachineDatabaseList?x=5a226b82-7523-4f5c-8194-2ee805791967 2024-11-04 03:29:16,869 - INFO - Pod ctest-pod-71835920 with uuid 5a226b82-7523-4f5c-8194-2ee805791967 found in kube manager 2024-11-04 03:29:16,869 - INFO - Pod ctest-pod-71835920 verification passed 2024-11-04 03:29:16,875 - INFO - Pod ctest-pod-83732116 is in running state.Got IP 10.47.255.248 2024-11-04 03:29:16,932 - DEBUG - Pod ctest-pod-83732116 has vmi f1f5ed74-9a5c-11ef-b331-fa163e954dcf 2024-11-04 03:29:16,933 - INFO - Verified pod ctest-pod-83732116 in contrail-api 2024-11-04 03:29:16,948 - DEBUG - VMI f1f5ed74-9a5c-11ef-b331-fa163e954dcf is active in agent 10.0.0.31 2024-11-04 03:29:16,949 - INFO - Verified Pod ctest-pod-83732116 in agent 10.0.0.31 2024-11-04 03:29:16,949 - DEBUG - Requesting: http://10.0.0.28:8108/Snh_VirtualMachineDatabaseList?x=3f49c63c-832b-46c6-8054-779eaae7e5b4 2024-11-04 03:29:16,954 - INFO - Pod ctest-pod-83732116 with uuid 3f49c63c-832b-46c6-8054-779eaae7e5b4 found in kube manager 2024-11-04 03:29:16,954 - INFO - Pod ctest-pod-83732116 verification passed 2024-11-04 03:29:16,960 - DEBUG - Pod ctest-pod-31456432 not in running state.Currently in Pending 2024-11-04 03:29:21,968 - INFO - Pod ctest-pod-31456432 is in running state.Got IP 10.47.255.247 2024-11-04 03:29:21,991 - DEBUG - Pod ctest-pod-31456432 has vmi f28c5ee4-9a5c-11ef-b331-fa163e954dcf 2024-11-04 03:29:21,991 - INFO - Verified pod ctest-pod-31456432 in contrail-api 2024-11-04 03:29:22,005 - DEBUG - VMI f28c5ee4-9a5c-11ef-b331-fa163e954dcf is active in agent 10.0.0.31 2024-11-04 03:29:22,005 - INFO - Verified Pod ctest-pod-31456432 in agent 10.0.0.31 2024-11-04 03:29:22,006 - DEBUG - Requesting: http://10.0.0.28:8108/Snh_VirtualMachineDatabaseList?x=d3f1c76c-b501-4ac8-b3f2-5ad75ac78d4c 2024-11-04 03:29:22,011 - INFO - Pod ctest-pod-31456432 with uuid d3f1c76c-b501-4ac8-b3f2-5ad75ac78d4c found in kube manager 2024-11-04 03:29:22,011 - INFO - Pod ctest-pod-31456432 verification passed 2024-11-04 03:29:22,017 - INFO - Pod ctest-pod-70615538 is in running state.Got IP 10.47.255.245 2024-11-04 03:29:22,086 - DEBUG - Pod ctest-pod-70615538 has vmi f38cffb0-9a5c-11ef-b331-fa163e954dcf 2024-11-04 03:29:22,086 - INFO - Verified pod ctest-pod-70615538 in contrail-api 2024-11-04 03:29:22,107 - DEBUG - VMI f38cffb0-9a5c-11ef-b331-fa163e954dcf is active in agent 10.0.0.31 2024-11-04 03:29:22,108 - INFO - Verified Pod ctest-pod-70615538 in agent 10.0.0.31 2024-11-04 03:29:22,108 - DEBUG - Requesting: http://10.0.0.28:8108/Snh_VirtualMachineDatabaseList?x=0c54170e-6329-4a50-8c46-2314943eacfa 2024-11-04 03:29:22,112 - INFO - Pod ctest-pod-70615538 with uuid 0c54170e-6329-4a50-8c46-2314943eacfa found in kube manager 2024-11-04 03:29:22,112 - INFO - Pod ctest-pod-70615538 verification passed 2024-11-04 03:29:22,118 - INFO - Pod ctest-pod-34493036 is in running state.Got IP 10.47.255.246 2024-11-04 03:29:22,177 - DEBUG - Pod ctest-pod-34493036 has vmi f309e5f8-9a5c-11ef-b331-fa163e954dcf 2024-11-04 03:29:22,177 - INFO - Verified pod ctest-pod-34493036 in contrail-api 2024-11-04 03:29:22,192 - DEBUG - VMI f309e5f8-9a5c-11ef-b331-fa163e954dcf is active in agent 10.0.0.31 2024-11-04 03:29:22,192 - INFO - Verified Pod ctest-pod-34493036 in agent 10.0.0.31 2024-11-04 03:29:22,192 - DEBUG - Requesting: http://10.0.0.28:8108/Snh_VirtualMachineDatabaseList?x=90d6953b-8679-464c-835c-d8e68c232def 2024-11-04 03:29:22,205 - INFO - Pod ctest-pod-34493036 with uuid 90d6953b-8679-464c-835c-d8e68c232def found in kube manager 2024-11-04 03:29:22,205 - INFO - Pod ctest-pod-34493036 verification passed 2024-11-04 03:29:22,215 - INFO - Pod ctest-pod-51035490 is in running state.Got IP 10.47.255.244 2024-11-04 03:29:22,242 - DEBUG - Pod ctest-pod-51035490 has vmi f406748a-9a5c-11ef-b331-fa163e954dcf 2024-11-04 03:29:22,242 - INFO - Verified pod ctest-pod-51035490 in contrail-api 2024-11-04 03:29:22,258 - DEBUG - VMI f406748a-9a5c-11ef-b331-fa163e954dcf is active in agent 10.0.0.31 2024-11-04 03:29:22,258 - INFO - Verified Pod ctest-pod-51035490 in agent 10.0.0.31 2024-11-04 03:29:22,259 - DEBUG - Requesting: http://10.0.0.28:8108/Snh_VirtualMachineDatabaseList?x=c2758868-14ee-4dfb-8802-fad17c225ee7 2024-11-04 03:29:22,265 - INFO - Pod ctest-pod-51035490 with uuid c2758868-14ee-4dfb-8802-fad17c225ee7 found in kube manager 2024-11-04 03:29:22,265 - INFO - Pod ctest-pod-51035490 verification passed 2024-11-04 03:29:22,269 - INFO - ================================================================================ 2024-11-04 03:29:22,269 - INFO - STARTING TEST : test_ingress_rule_on_pod_with_service 2024-11-04 03:29:22,269 - INFO - TEST DESCRIPTION : Verify that ingress rules on a pod work fine when pods are accessed via a service Steps: 1. Create a service in namespace 1 over a pod which are endpoints of a service 2. Create a network policy to allow 1 pod of namespace 1 as ingress. 3. Verify that service respond correctly to the specified pod in the policy and service do not respond to anyone else. 4. Extend the policy and add the namespaceselector as namespace 2. 5. Verify that service become accessible to the pods of namespace 2. 2024-11-04 03:29:22,523 - DEBUG - Nothing to compare xmpp stats {'10.0.0.31': {'10.0.0.28': '0'}} with 2024-11-04 03:29:22,523 - INFO - Initial checks done. Running the testcase now 2024-11-04 03:29:22,523 - INFO - 2024-11-04 03:29:22,594 - DEBUG - [Pod ctest-pod-41922036] Cmd: echo ctest-pod-41922036 > /usr/share/nginx/html/index.html, Output: 2024-11-04 03:29:22,685 - DEBUG - [Pod ctest-pod-13443166] Cmd: echo ctest-pod-13443166 > /usr/share/nginx/html/index.html, Output: 2024-11-04 03:29:22,771 - DEBUG - [Pod ctest-pod-71835920] Cmd: echo ctest-pod-71835920 > /usr/share/nginx/html/index.html, Output: 2024-11-04 03:29:22,837 - DEBUG - [Pod ctest-pod-83732116] Cmd: echo ctest-pod-83732116 > /usr/share/nginx/html/index.html, Output: 2024-11-04 03:29:22,850 - DEBUG - Service ctest-nginx-svc-64180762 not present 2024-11-04 03:29:22,851 - INFO - Creating service ctest-nginx-svc-64180762 2024-11-04 03:29:22,901 - INFO - Created Service ctest-nginx-svc-64180762 2024-11-04 03:29:22,905 - WARNING - Service UUID 14bf7144-70fb-4134-ace1-47b1aa06d726 not yet found in contrail-api 2024-11-04 03:29:23,919 - INFO - Validated that Service ctest-nginx-svc-64180762 is seen in contrail-api 2024-11-04 03:29:23,920 - DEBUG - Requesting: http://10.0.0.28:8108/Snh_LoadbalancerDatabaseList?x=14bf7144-70fb-4134-ace1-47b1aa06d726 2024-11-04 03:29:23,925 - INFO - Service ctest-nginx-svc-64180762 with uuid 14bf7144-70fb-4134-ace1-47b1aa06d726 found in kube manager 2024-11-04 03:29:23,925 - INFO - Service ctest-nginx-svc-64180762 verification passed 2024-11-04 03:29:25,003 - DEBUG - [Pod ctest-pod-31456432] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: Connecting to 10.101.17.108:80 (10.101.17.108:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-13443166 2024-11-04 03:29:25,003 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 passed 2024-11-04 03:29:25,003 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd output: Connecting to 10.101.17.108:80 (10.101.17.108:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-13443166 2024-11-04 03:29:25,070 - DEBUG - [Pod ctest-pod-31456432] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: Connecting to 10.101.17.108:80 (10.101.17.108:80) ctest-pod-41922036 writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-04 03:29:25,070 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 passed 2024-11-04 03:29:25,071 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd output: Connecting to 10.101.17.108:80 (10.101.17.108:80) ctest-pod-41922036 writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-04 03:29:25,071 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-41922036': 1, 'ctest-pod-13443166': 1} 2024-11-04 03:29:25,144 - DEBUG - [Pod ctest-pod-70615538] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: ctest-pod-13443166 Connecting to 10.101.17.108:80 (10.101.17.108:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-04 03:29:25,145 - DEBUG - [Pod Pod ctest-pod-70615538] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 passed 2024-11-04 03:29:25,145 - DEBUG - [Pod Pod ctest-pod-70615538] Cmd output: ctest-pod-13443166 Connecting to 10.101.17.108:80 (10.101.17.108:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-04 03:29:25,210 - DEBUG - [Pod ctest-pod-70615538] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: Connecting to 10.101.17.108:80 (10.101.17.108:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-41922036 2024-11-04 03:29:25,210 - DEBUG - [Pod Pod ctest-pod-70615538] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 passed 2024-11-04 03:29:25,210 - DEBUG - [Pod Pod ctest-pod-70615538] Cmd output: Connecting to 10.101.17.108:80 (10.101.17.108:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-41922036 2024-11-04 03:29:25,210 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-41922036': 1, 'ctest-pod-13443166': 1} 2024-11-04 03:29:25,214 - DEBUG - Network policy ingress-on-service not present 2024-11-04 03:29:25,215 - INFO - Creating Network Policy ingress-on-service 2024-11-04 03:29:25,228 - INFO - Network policy found in k8s 2024-11-04 03:29:25,229 - DEBUG - Requesting: http://10.0.0.28:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=c672f21c-578e-4d48-aeb5-dc7a817aec30 2024-11-04 03:29:25,235 - INFO - Network Policy ingress-on-service with uuid c672f21c-578e-4d48-aeb5-dc7a817aec30 found in kube manager 2024-11-04 03:29:25,284 - INFO - Network policy with name default-policy-management:k8s-allowall found in agent 2024-11-04 03:29:25,284 - INFO - Network policy with name default-policy-management:k8s-allowall associated with default ks8 2024-11-04 03:29:25,284 - DEBUG - Requesting: http://10.0.0.28:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=c672f21c-578e-4d48-aeb5-dc7a817aec30 2024-11-04 03:29:25,338 - WARNING - Network policy with name ingress-on-service not found in agent 2024-11-04 03:29:27,339 - DEBUG - Requesting: http://10.0.0.28:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=c672f21c-578e-4d48-aeb5-dc7a817aec30 2024-11-04 03:29:27,386 - WARNING - Network policy with name ingress-on-service not found in agent 2024-11-04 03:29:29,386 - DEBUG - Requesting: http://10.0.0.28:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=c672f21c-578e-4d48-aeb5-dc7a817aec30 2024-11-04 03:29:29,419 - INFO - Network policy with name ingress-on-service found in agent 2024-11-04 03:29:29,419 - INFO - Network Policy ingress-on-service verification passed 2024-11-04 03:29:34,475 - DEBUG - [Pod ctest-pod-31456432] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:29:34,475 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:29:34,475 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:29:39,528 - DEBUG - [Pod ctest-pod-31456432] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:29:39,528 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:29:39,528 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:29:44,580 - DEBUG - [Pod ctest-pod-31456432] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:29:44,580 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:29:44,580 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:29:49,631 - DEBUG - [Pod ctest-pod-31456432] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:29:49,631 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:29:49,631 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:29:54,701 - DEBUG - [Pod ctest-pod-31456432] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:29:54,702 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:29:54,702 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:29:59,765 - DEBUG - [Pod ctest-pod-31456432] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:29:59,765 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:29:59,765 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:30:04,828 - DEBUG - [Pod ctest-pod-31456432] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:30:04,828 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:30:04,828 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:30:09,882 - DEBUG - [Pod ctest-pod-31456432] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:30:09,882 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:30:09,882 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:30:14,951 - DEBUG - [Pod ctest-pod-31456432] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:30:14,951 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:30:14,951 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:30:20,020 - DEBUG - [Pod ctest-pod-31456432] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:30:20,021 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:30:20,021 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:30:20,021 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-41922036': 0, 'ctest-pod-13443166': 0} 2024-11-04 03:30:20,081 - DEBUG - [Pod ctest-pod-34493036] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: Connecting to 10.101.17.108:80 (10.101.17.108:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-13443166 2024-11-04 03:30:20,081 - DEBUG - [Pod Pod ctest-pod-34493036] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 passed 2024-11-04 03:30:20,081 - DEBUG - [Pod Pod ctest-pod-34493036] Cmd output: Connecting to 10.101.17.108:80 (10.101.17.108:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-13443166 2024-11-04 03:30:20,138 - DEBUG - [Pod ctest-pod-34493036] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: ctest-pod-13443166 Connecting to 10.101.17.108:80 (10.101.17.108:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-04 03:30:20,138 - DEBUG - [Pod Pod ctest-pod-34493036] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 passed 2024-11-04 03:30:20,138 - DEBUG - [Pod Pod ctest-pod-34493036] Cmd output: ctest-pod-13443166 Connecting to 10.101.17.108:80 (10.101.17.108:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-04 03:30:20,205 - DEBUG - [Pod ctest-pod-34493036] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: Connecting to 10.101.17.108:80 (10.101.17.108:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-41922036 2024-11-04 03:30:20,205 - DEBUG - [Pod Pod ctest-pod-34493036] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 passed 2024-11-04 03:30:20,205 - DEBUG - [Pod Pod ctest-pod-34493036] Cmd output: Connecting to 10.101.17.108:80 (10.101.17.108:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-41922036 2024-11-04 03:30:20,205 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-41922036': 1, 'ctest-pod-13443166': 2} 2024-11-04 03:30:25,262 - DEBUG - [Pod ctest-pod-70615538] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:30:25,262 - DEBUG - [Pod Pod ctest-pod-70615538] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:30:25,262 - DEBUG - [Pod Pod ctest-pod-70615538] Cmd output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:30:30,319 - DEBUG - [Pod ctest-pod-70615538] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:30:30,319 - DEBUG - [Pod Pod ctest-pod-70615538] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:30:30,319 - DEBUG - [Pod Pod ctest-pod-70615538] Cmd output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:30:35,371 - DEBUG - [Pod ctest-pod-70615538] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:30:35,372 - DEBUG - [Pod Pod ctest-pod-70615538] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:30:35,372 - DEBUG - [Pod Pod ctest-pod-70615538] Cmd output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:30:40,450 - DEBUG - [Pod ctest-pod-70615538] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:30:40,450 - DEBUG - [Pod Pod ctest-pod-70615538] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:30:40,450 - DEBUG - [Pod Pod ctest-pod-70615538] Cmd output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:30:45,519 - DEBUG - [Pod ctest-pod-70615538] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:30:45,519 - DEBUG - [Pod Pod ctest-pod-70615538] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:30:45,519 - DEBUG - [Pod Pod ctest-pod-70615538] Cmd output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:30:50,619 - DEBUG - [Pod ctest-pod-70615538] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:30:50,619 - DEBUG - [Pod Pod ctest-pod-70615538] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:30:50,619 - DEBUG - [Pod Pod ctest-pod-70615538] Cmd output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:30:55,692 - DEBUG - [Pod ctest-pod-70615538] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:30:55,692 - DEBUG - [Pod Pod ctest-pod-70615538] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:30:55,692 - DEBUG - [Pod Pod ctest-pod-70615538] Cmd output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:30:59,228 - WARNING - ApiException is caught (0) Reason: Connection is already closed. . Retrying... 2024-11-04 03:31:09,318 - DEBUG - [Pod ctest-pod-70615538] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:31:09,318 - DEBUG - [Pod Pod ctest-pod-70615538] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:31:09,318 - DEBUG - [Pod Pod ctest-pod-70615538] Cmd output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:31:14,389 - DEBUG - [Pod ctest-pod-70615538] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:31:14,390 - DEBUG - [Pod Pod ctest-pod-70615538] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:31:14,390 - DEBUG - [Pod Pod ctest-pod-70615538] Cmd output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:31:19,450 - DEBUG - [Pod ctest-pod-70615538] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:31:19,451 - DEBUG - [Pod Pod ctest-pod-70615538] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:31:19,451 - DEBUG - [Pod Pod ctest-pod-70615538] Cmd output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:31:19,451 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-41922036': 0, 'ctest-pod-13443166': 0} 2024-11-04 03:31:21,503 - DEBUG - [Pod ctest-pod-34493036] Cmd: ping -c 3 10.47.255.247, Output: PING 10.47.255.247 (10.47.255.247): 56 data bytes 64 bytes from 10.47.255.247: seq=0 ttl=63 time=1.336 ms 64 bytes from 10.47.255.247: seq=1 ttl=63 time=0.080 ms 64 bytes from 10.47.255.247: seq=2 ttl=63 time=0.095 ms --- 10.47.255.247 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.080/0.503/1.336 ms 2024-11-04 03:31:21,503 - INFO - Ping check to IP 10.47.255.247 from pod ctest-pod-34493036 with expectation True passed 2024-11-04 03:31:23,579 - DEBUG - [Pod ctest-pod-31456432] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=1.380 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.094 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.111 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.094/0.528/1.380 ms 2024-11-04 03:31:23,579 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-31456432 with expectation True passed 2024-11-04 03:31:25,653 - DEBUG - [Pod ctest-pod-70615538] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=0.367 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.096 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.115 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.096/0.192/0.367 ms 2024-11-04 03:31:25,654 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-70615538 with expectation True passed 2024-11-04 03:31:25,725 - DEBUG - [Pod ctest-pod-34493036] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-11-04 03:31:25,725 - DEBUG - [Pod Pod ctest-pod-34493036] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2024-11-04 03:31:25,725 - DEBUG - [Pod Pod ctest-pod-34493036] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-11-04 03:31:25,725 - INFO - wget check of of http://10.47.255.251 from ctest-pod-34493036 passed 2024-11-04 03:31:25,810 - DEBUG - [Pod ctest-pod-34493036] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-11-04 03:31:25,810 - DEBUG - [Pod Pod ctest-pod-34493036] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2024-11-04 03:31:25,810 - DEBUG - [Pod Pod ctest-pod-34493036] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-11-04 03:31:25,810 - INFO - wget check of of http://10.47.255.250 from ctest-pod-34493036 passed 2024-11-04 03:31:30,895 - DEBUG - [Pod ctest-pod-31456432] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-11-04 03:31:30,895 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-11-04 03:31:30,895 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-11-04 03:31:30,895 - INFO - wget check of of http://10.47.255.250 from ctest-pod-31456432 passed 2024-11-04 03:31:35,968 - DEBUG - [Pod ctest-pod-51035490] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) wget: download timed out 2024-11-04 03:31:35,968 - DEBUG - [Pod Pod ctest-pod-51035490] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.251 (10.47.255.251:80) wget: download timed out 2024-11-04 03:31:35,968 - DEBUG - [Pod Pod ctest-pod-51035490] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) wget: download timed out 2024-11-04 03:31:35,968 - INFO - wget check of of http://10.47.255.251 from ctest-pod-51035490 passed 2024-11-04 03:31:35,976 - INFO - Updating Network Policy ingress-on-service 2024-11-04 03:31:41,059 - DEBUG - [Pod ctest-pod-31456432] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:31:41,059 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:31:41,059 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:31:46,116 - DEBUG - [Pod ctest-pod-31456432] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:31:46,116 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:31:46,116 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:31:51,166 - DEBUG - [Pod ctest-pod-31456432] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:31:51,166 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:31:51,166 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:31:56,219 - DEBUG - [Pod ctest-pod-31456432] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:31:56,219 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:31:56,219 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:32:01,272 - DEBUG - [Pod ctest-pod-31456432] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:32:01,272 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:32:01,272 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:32:06,335 - DEBUG - [Pod ctest-pod-31456432] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:32:06,335 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:32:06,335 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:32:11,408 - DEBUG - [Pod ctest-pod-31456432] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:32:11,408 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:32:11,408 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:32:16,463 - DEBUG - [Pod ctest-pod-31456432] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:32:16,463 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:32:16,463 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:32:21,520 - DEBUG - [Pod ctest-pod-31456432] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:32:21,520 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:32:21,520 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:32:26,573 - DEBUG - [Pod ctest-pod-31456432] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:32:26,573 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:32:26,574 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd output: Connecting to 10.101.17.108:80 (10.101.17.108:80) wget: download timed out 2024-11-04 03:32:26,574 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-41922036': 0, 'ctest-pod-13443166': 0} 2024-11-04 03:32:26,638 - DEBUG - [Pod ctest-pod-34493036] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: Connecting to 10.101.17.108:80 (10.101.17.108:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-13443166 2024-11-04 03:32:26,638 - DEBUG - [Pod Pod ctest-pod-34493036] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 passed 2024-11-04 03:32:26,638 - DEBUG - [Pod Pod ctest-pod-34493036] Cmd output: Connecting to 10.101.17.108:80 (10.101.17.108:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-13443166 2024-11-04 03:32:26,693 - DEBUG - [Pod ctest-pod-34493036] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: Connecting to 10.101.17.108:80 (10.101.17.108:80) ctest-pod-13443166 writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-04 03:32:26,693 - DEBUG - [Pod Pod ctest-pod-34493036] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 passed 2024-11-04 03:32:26,693 - DEBUG - [Pod Pod ctest-pod-34493036] Cmd output: Connecting to 10.101.17.108:80 (10.101.17.108:80) ctest-pod-13443166 writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-04 03:32:26,761 - DEBUG - [Pod ctest-pod-34493036] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: ctest-pod-13443166 Connecting to 10.101.17.108:80 (10.101.17.108:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-04 03:32:26,761 - DEBUG - [Pod Pod ctest-pod-34493036] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 passed 2024-11-04 03:32:26,761 - DEBUG - [Pod Pod ctest-pod-34493036] Cmd output: ctest-pod-13443166 Connecting to 10.101.17.108:80 (10.101.17.108:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-04 03:32:26,824 - DEBUG - [Pod ctest-pod-34493036] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: ctest-pod-13443166 Connecting to 10.101.17.108:80 (10.101.17.108:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-04 03:32:26,824 - DEBUG - [Pod Pod ctest-pod-34493036] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 passed 2024-11-04 03:32:26,824 - DEBUG - [Pod Pod ctest-pod-34493036] Cmd output: ctest-pod-13443166 Connecting to 10.101.17.108:80 (10.101.17.108:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-04 03:32:26,890 - DEBUG - [Pod ctest-pod-34493036] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: ctest-pod-41922036 Connecting to 10.101.17.108:80 (10.101.17.108:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-04 03:32:26,890 - DEBUG - [Pod Pod ctest-pod-34493036] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 passed 2024-11-04 03:32:26,890 - DEBUG - [Pod Pod ctest-pod-34493036] Cmd output: ctest-pod-41922036 Connecting to 10.101.17.108:80 (10.101.17.108:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-04 03:32:26,890 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-41922036': 1, 'ctest-pod-13443166': 4} 2024-11-04 03:32:26,941 - DEBUG - [Pod ctest-pod-70615538] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: ctest-pod-13443166 Connecting to 10.101.17.108:80 (10.101.17.108:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-04 03:32:26,941 - DEBUG - [Pod Pod ctest-pod-70615538] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 passed 2024-11-04 03:32:26,941 - DEBUG - [Pod Pod ctest-pod-70615538] Cmd output: ctest-pod-13443166 Connecting to 10.101.17.108:80 (10.101.17.108:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-04 03:32:27,016 - DEBUG - [Pod ctest-pod-70615538] Cmd: wget http://10.101.17.108:80/ -O - -T 5 -t 1 , Output: ctest-pod-41922036 Connecting to 10.101.17.108:80 (10.101.17.108:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-04 03:32:27,016 - DEBUG - [Pod Pod ctest-pod-70615538] Cmd wget http://10.101.17.108:80/ -O - -T 5 -t 1 passed 2024-11-04 03:32:27,016 - DEBUG - [Pod Pod ctest-pod-70615538] Cmd output: ctest-pod-41922036 Connecting to 10.101.17.108:80 (10.101.17.108:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-11-04 03:32:27,016 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-41922036': 1, 'ctest-pod-13443166': 1} 2024-11-04 03:32:29,083 - DEBUG - [Pod ctest-pod-34493036] Cmd: ping -c 3 10.47.255.247, Output: PING 10.47.255.247 (10.47.255.247): 56 data bytes 64 bytes from 10.47.255.247: seq=0 ttl=63 time=0.835 ms 64 bytes from 10.47.255.247: seq=1 ttl=63 time=0.083 ms 64 bytes from 10.47.255.247: seq=2 ttl=63 time=0.103 ms --- 10.47.255.247 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.083/0.340/0.835 ms 2024-11-04 03:32:29,083 - INFO - Ping check to IP 10.47.255.247 from pod ctest-pod-34493036 with expectation True passed 2024-11-04 03:32:31,165 - DEBUG - [Pod ctest-pod-31456432] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=1.114 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.088 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.105 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.088/0.435/1.114 ms 2024-11-04 03:32:31,165 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-31456432 with expectation True passed 2024-11-04 03:32:33,235 - DEBUG - [Pod ctest-pod-70615538] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=0.908 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.099 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.098 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.098/0.368/0.908 ms 2024-11-04 03:32:33,235 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-70615538 with expectation True passed 2024-11-04 03:32:33,307 - DEBUG - [Pod ctest-pod-34493036] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-11-04 03:32:33,307 - DEBUG - [Pod Pod ctest-pod-34493036] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2024-11-04 03:32:33,307 - DEBUG - [Pod Pod ctest-pod-34493036] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-11-04 03:32:33,307 - INFO - wget check of of http://10.47.255.251 from ctest-pod-34493036 passed 2024-11-04 03:32:33,360 - DEBUG - [Pod ctest-pod-34493036] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-11-04 03:32:33,360 - DEBUG - [Pod Pod ctest-pod-34493036] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2024-11-04 03:32:33,360 - DEBUG - [Pod Pod ctest-pod-34493036] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-11-04 03:32:33,360 - INFO - wget check of of http://10.47.255.250 from ctest-pod-34493036 passed 2024-11-04 03:32:38,416 - DEBUG - [Pod ctest-pod-31456432] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-11-04 03:32:38,416 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-11-04 03:32:38,416 - DEBUG - [Pod Pod ctest-pod-31456432] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-11-04 03:32:38,416 - INFO - wget check of of http://10.47.255.250 from ctest-pod-31456432 passed 2024-11-04 03:32:38,479 - DEBUG - [Pod ctest-pod-51035490] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-11-04 03:32:38,480 - DEBUG - [Pod Pod ctest-pod-51035490] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2024-11-04 03:32:38,480 - DEBUG - [Pod Pod ctest-pod-51035490] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-11-04 03:32:38,480 - INFO - wget check of of http://10.47.255.251 from ctest-pod-51035490 passed 2024-11-04 03:32:38,480 - INFO - Deleting Network Policy : ingress-on-service 2024-11-04 03:32:38,480 - INFO - Deleting service : ctest-nginx-svc-64180762 2024-11-04 03:32:38,703 - DEBUG - No XMPP flaps were noticed during the test 2024-11-04 03:32:38,703 - INFO - END TEST : test_ingress_rule_on_pod_with_service : PASSED[0:03:16] 2024-11-04 03:32:38,704 - INFO - -------------------------------------------------------------------------------- 2024-11-04 03:32:38,710 - INFO - Deleting pod ctest-new-default-58657994:ctest-pod-41922036 2024-11-04 03:32:38,710 - INFO - Deleting pod ctest-new-default-58657994:ctest-pod-13443166 2024-11-04 03:32:38,711 - INFO - Deleting pod ctest-non-default-94017682:ctest-pod-71835920 2024-11-04 03:32:38,711 - INFO - Deleting pod ctest-non-default-94017682:ctest-pod-83732116 2024-11-04 03:32:38,712 - INFO - Deleting pod ctest-new-default-58657994:ctest-pod-31456432 2024-11-04 03:32:38,712 - INFO - Deleting pod ctest-new-default-58657994:ctest-pod-34493036 2024-11-04 03:32:38,712 - INFO - Deleting pod ctest-non-default-94017682:ctest-pod-70615538 2024-11-04 03:32:38,713 - INFO - Deleting pod ctest-non-default-94017682:ctest-pod-51035490 2024-11-04 03:32:38,746 - WARNING - Pod uuid e0db9ae1-0a03-4cf8-8a6b-6e9c7168f428 is still seen in agent 10.0.0.31 VM list 2024-11-04 03:32:38,857 - WARNING - Pod uuid 3f49c63c-832b-46c6-8054-779eaae7e5b4 is still seen in agent 10.0.0.31 VM list 2024-11-04 03:32:38,858 - WARNING - Pod uuid 0c54170e-6329-4a50-8c46-2314943eacfa is still seen in agent 10.0.0.31 VM list 2024-11-04 03:32:38,859 - WARNING - Pod uuid c2758868-14ee-4dfb-8802-fad17c225ee7 is still seen in agent 10.0.0.31 VM list 2024-11-04 03:32:38,859 - WARNING - Pod uuid 90d6953b-8679-464c-835c-d8e68c232def is still seen in agent 10.0.0.31 VM list 2024-11-04 03:32:38,859 - WARNING - Pod uuid d5459880-3fdc-43f7-8519-a54d0e2f9e7f is still seen in agent 10.0.0.31 VM list 2024-11-04 03:32:38,860 - WARNING - Pod uuid d3f1c76c-b501-4ac8-b3f2-5ad75ac78d4c is still seen in agent 10.0.0.31 VM list 2024-11-04 03:32:38,860 - WARNING - Pod uuid 5a226b82-7523-4f5c-8194-2ee805791967 is still seen in agent 10.0.0.31 VM list 2024-11-04 03:32:43,758 - DEBUG - Pod e0db9ae1-0a03-4cf8-8a6b-6e9c7168f428 is not in agent 10.0.0.31 VM list 2024-11-04 03:32:43,764 - DEBUG - VMI f021ba46-9a5c-11ef-b331-fa163e954dcf is removed from agent 10.0.0.31 2024-11-04 03:32:43,764 - INFO - Verified that pod ctest-pod-41922036 is removed in agent 2024-11-04 03:32:43,890 - WARNING - Pod uuid 90d6953b-8679-464c-835c-d8e68c232def is still seen in agent 10.0.0.31 VM list 2024-11-04 03:32:43,890 - WARNING - Pod uuid c2758868-14ee-4dfb-8802-fad17c225ee7 is still seen in agent 10.0.0.31 VM list 2024-11-04 03:32:43,892 - DEBUG - Pod 5a226b82-7523-4f5c-8194-2ee805791967 is not in agent 10.0.0.31 VM list 2024-11-04 03:32:43,894 - WARNING - Pod uuid d3f1c76c-b501-4ac8-b3f2-5ad75ac78d4c is still seen in agent 10.0.0.31 VM list 2024-11-04 03:32:43,894 - WARNING - Pod uuid 0c54170e-6329-4a50-8c46-2314943eacfa is still seen in agent 10.0.0.31 VM list 2024-11-04 03:32:43,895 - DEBUG - Pod d5459880-3fdc-43f7-8519-a54d0e2f9e7f is not in agent 10.0.0.31 VM list 2024-11-04 03:32:43,896 - DEBUG - Pod 3f49c63c-832b-46c6-8054-779eaae7e5b4 is not in agent 10.0.0.31 VM list 2024-11-04 03:32:43,909 - DEBUG - VMI f165d856-9a5c-11ef-b331-fa163e954dcf is removed from agent 10.0.0.31 2024-11-04 03:32:43,909 - INFO - Verified that pod ctest-pod-71835920 is removed in agent 2024-11-04 03:32:43,910 - DEBUG - VMI f0ae4128-9a5c-11ef-b331-fa163e954dcf is removed from agent 10.0.0.31 2024-11-04 03:32:43,910 - INFO - Verified that pod ctest-pod-13443166 is removed in agent 2024-11-04 03:32:43,911 - DEBUG - VMI f1f5ed74-9a5c-11ef-b331-fa163e954dcf is removed from agent 10.0.0.31 2024-11-04 03:32:43,911 - INFO - Verified that pod ctest-pod-83732116 is removed in agent 2024-11-04 03:32:48,915 - WARNING - Pod uuid d3f1c76c-b501-4ac8-b3f2-5ad75ac78d4c is still seen in agent 10.0.0.31 VM list 2024-11-04 03:32:48,915 - WARNING - Pod uuid c2758868-14ee-4dfb-8802-fad17c225ee7 is still seen in agent 10.0.0.31 VM list 2024-11-04 03:32:48,917 - WARNING - Pod uuid 0c54170e-6329-4a50-8c46-2314943eacfa is still seen in agent 10.0.0.31 VM list 2024-11-04 03:32:48,917 - WARNING - Pod uuid 90d6953b-8679-464c-835c-d8e68c232def is still seen in agent 10.0.0.31 VM list 2024-11-04 03:32:53,936 - WARNING - Pod uuid d3f1c76c-b501-4ac8-b3f2-5ad75ac78d4c is still seen in agent 10.0.0.31 VM list 2024-11-04 03:32:53,937 - WARNING - Pod uuid c2758868-14ee-4dfb-8802-fad17c225ee7 is still seen in agent 10.0.0.31 VM list 2024-11-04 03:32:53,939 - WARNING - Pod uuid 90d6953b-8679-464c-835c-d8e68c232def is still seen in agent 10.0.0.31 VM list 2024-11-04 03:32:53,939 - WARNING - Pod uuid 0c54170e-6329-4a50-8c46-2314943eacfa is still seen in agent 10.0.0.31 VM list 2024-11-04 03:32:58,957 - WARNING - Pod uuid c2758868-14ee-4dfb-8802-fad17c225ee7 is still seen in agent 10.0.0.31 VM list 2024-11-04 03:32:58,958 - WARNING - Pod uuid d3f1c76c-b501-4ac8-b3f2-5ad75ac78d4c is still seen in agent 10.0.0.31 VM list 2024-11-04 03:32:58,959 - WARNING - Pod uuid 90d6953b-8679-464c-835c-d8e68c232def is still seen in agent 10.0.0.31 VM list 2024-11-04 03:32:58,960 - WARNING - Pod uuid 0c54170e-6329-4a50-8c46-2314943eacfa is still seen in agent 10.0.0.31 VM list 2024-11-04 03:33:03,976 - WARNING - Pod uuid d3f1c76c-b501-4ac8-b3f2-5ad75ac78d4c is still seen in agent 10.0.0.31 VM list 2024-11-04 03:33:03,978 - WARNING - Pod uuid 90d6953b-8679-464c-835c-d8e68c232def is still seen in agent 10.0.0.31 VM list 2024-11-04 03:33:03,978 - WARNING - Pod uuid 0c54170e-6329-4a50-8c46-2314943eacfa is still seen in agent 10.0.0.31 VM list 2024-11-04 03:33:03,979 - WARNING - Pod uuid c2758868-14ee-4dfb-8802-fad17c225ee7 is still seen in agent 10.0.0.31 VM list 2024-11-04 03:33:09,062 - WARNING - Pod uuid 90d6953b-8679-464c-835c-d8e68c232def is still seen in agent 10.0.0.31 VM list 2024-11-04 03:33:09,063 - WARNING - Pod uuid d3f1c76c-b501-4ac8-b3f2-5ad75ac78d4c is still seen in agent 10.0.0.31 VM list 2024-11-04 03:33:09,065 - WARNING - Pod uuid 0c54170e-6329-4a50-8c46-2314943eacfa is still seen in agent 10.0.0.31 VM list 2024-11-04 03:33:09,065 - WARNING - Pod uuid c2758868-14ee-4dfb-8802-fad17c225ee7 is still seen in agent 10.0.0.31 VM list 2024-11-04 03:33:14,090 - DEBUG - Pod 90d6953b-8679-464c-835c-d8e68c232def is not in agent 10.0.0.31 VM list 2024-11-04 03:33:14,093 - DEBUG - Pod 0c54170e-6329-4a50-8c46-2314943eacfa is not in agent 10.0.0.31 VM list 2024-11-04 03:33:14,095 - DEBUG - Pod c2758868-14ee-4dfb-8802-fad17c225ee7 is not in agent 10.0.0.31 VM list 2024-11-04 03:33:14,098 - DEBUG - Pod d3f1c76c-b501-4ac8-b3f2-5ad75ac78d4c is not in agent 10.0.0.31 VM list 2024-11-04 03:33:14,110 - DEBUG - VMI f309e5f8-9a5c-11ef-b331-fa163e954dcf is removed from agent 10.0.0.31 2024-11-04 03:33:14,110 - INFO - Verified that pod ctest-pod-34493036 is removed in agent 2024-11-04 03:33:14,113 - DEBUG - VMI f38cffb0-9a5c-11ef-b331-fa163e954dcf is removed from agent 10.0.0.31 2024-11-04 03:33:14,113 - INFO - Verified that pod ctest-pod-70615538 is removed in agent 2024-11-04 03:33:14,115 - DEBUG - VMI f28c5ee4-9a5c-11ef-b331-fa163e954dcf is removed from agent 10.0.0.31 2024-11-04 03:33:14,115 - INFO - Verified that pod ctest-pod-31456432 is removed in agent 2024-11-04 03:33:14,116 - DEBUG - VMI f406748a-9a5c-11ef-b331-fa163e954dcf is removed from agent 10.0.0.31 2024-11-04 03:33:14,116 - INFO - Verified that pod ctest-pod-51035490 is removed in agent 2024-11-04 03:33:14,116 - INFO - Deleting namespace ctest-non-default-94017682 2024-11-04 03:33:14,126 - DEBUG - Namespace ctest-non-default-94017682 still in k8s 2024-11-04 03:33:16,131 - DEBUG - Namespace ctest-non-default-94017682 still in k8s 2024-11-04 03:33:18,136 - DEBUG - Namespace ctest-non-default-94017682 still in k8s 2024-11-04 03:33:20,140 - DEBUG - Namespace ctest-non-default-94017682 is not in k8s 2024-11-04 03:33:20,140 - DEBUG - No need to check project deletion in contrail-api 2024-11-04 03:33:20,140 - INFO - Deleting namespace ctest-new-default-58657994 2024-11-04 03:33:20,148 - DEBUG - Namespace ctest-new-default-58657994 still in k8s 2024-11-04 03:33:22,153 - DEBUG - Namespace ctest-new-default-58657994 still in k8s 2024-11-04 03:33:24,166 - DEBUG - Namespace ctest-new-default-58657994 still in k8s 2024-11-04 03:33:26,170 - DEBUG - Namespace ctest-new-default-58657994 is not in k8s 2024-11-04 03:33:26,170 - DEBUG - No need to check project deletion in contrail-api