2024-09-11 15:55:52,272 - INFO - Using existing project ['default-domain', 'k8s-default'](bea114bc-386f-4b68-b45c-432501996b7b) 2024-09-11 15:55:52,289 - DEBUG - Namespace ctest-new-default-58999978 not present 2024-09-11 15:55:52,302 - INFO - Created namespace ctest-new-default-58999978 2024-09-11 15:55:55,307 - DEBUG - Namespace ctest-non-default-21463801 not present 2024-09-11 15:55:55,318 - INFO - Created namespace ctest-non-default-21463801 2024-09-11 15:55:58,484 - DEBUG - Pod ctest-pod-91448207 not present 2024-09-11 15:55:58,484 - INFO - Creating Pod ctest-pod-91448207 2024-09-11 15:55:58,551 - DEBUG - Pod : ctest-pod-91448207 UUID is cdd9d9d1-14dd-4148-9cd1-bdd9341c2524 2024-09-11 15:55:58,568 - DEBUG - Pod ctest-pod-28233703 not present 2024-09-11 15:55:58,569 - INFO - Creating Pod ctest-pod-28233703 2024-09-11 15:55:58,637 - DEBUG - Pod : ctest-pod-28233703 UUID is b1c7193e-5809-4ae5-9004-59f9bb1fa90c 2024-09-11 15:55:58,643 - DEBUG - Pod ctest-pod-74662013 not present 2024-09-11 15:55:58,643 - INFO - Creating Pod ctest-pod-74662013 2024-09-11 15:55:58,693 - DEBUG - Pod : ctest-pod-74662013 UUID is 12101a40-3c59-49f2-8af0-e3809c9e2c1a 2024-09-11 15:55:58,700 - DEBUG - Pod ctest-pod-40431095 not present 2024-09-11 15:55:58,708 - INFO - Creating Pod ctest-pod-40431095 2024-09-11 15:55:58,774 - DEBUG - Pod : ctest-pod-40431095 UUID is ccdcc52f-177a-4037-8868-20d89f5bb548 2024-09-11 15:55:58,787 - DEBUG - Pod ctest-pod-74380417 not present 2024-09-11 15:55:58,787 - INFO - Creating Pod ctest-pod-74380417 2024-09-11 15:55:58,850 - DEBUG - Pod : ctest-pod-74380417 UUID is 4cfcb7ec-bf3b-49f0-a09e-92ccec4d8841 2024-09-11 15:55:58,863 - DEBUG - Pod ctest-pod-88373613 not present 2024-09-11 15:55:58,863 - INFO - Creating Pod ctest-pod-88373613 2024-09-11 15:55:58,900 - DEBUG - Pod : ctest-pod-88373613 UUID is f478e0c2-5217-4cb3-9583-80c75189f0a8 2024-09-11 15:55:58,909 - DEBUG - Pod ctest-pod-18271201 not present 2024-09-11 15:55:58,910 - INFO - Creating Pod ctest-pod-18271201 2024-09-11 15:56:01,103 - DEBUG - Pod : ctest-pod-18271201 UUID is dfa84d6d-f1aa-4bea-81be-33b1bee51a81 2024-09-11 15:56:01,107 - DEBUG - Pod ctest-pod-36502319 not present 2024-09-11 15:56:01,107 - INFO - Creating Pod ctest-pod-36502319 2024-09-11 15:56:01,136 - DEBUG - Pod : ctest-pod-36502319 UUID is 1a799bc1-041c-4d14-950b-5c340f6b95f6 2024-09-11 15:56:01,478 - INFO - Project default-domain:k8s-ctest-new-default-58999978 for Namespace ctest-new-default-58999978 is seen in contrail-api 2024-09-11 15:56:01,478 - DEBUG - Project uuid in Contrail is 8153cdae-3ec3-4c45-9e92-abf8fd1214fe 2024-09-11 15:56:01,674 - DEBUG - Requesting: http://10.0.0.34:8108/Snh_NamespaceDatabaseList?namespace_uuid=bcdca132-cc40-467b-b59a-badc647c07b2 2024-09-11 15:56:01,679 - INFO - Namespace ctest-new-default-58999978 with uuid bcdca132-cc40-467b-b59a-badc647c07b2 found in kube manager 2024-09-11 15:56:01,679 - INFO - Namespace ctest-new-default-58999978 verification passed 2024-09-11 15:56:01,955 - INFO - Project default-domain:k8s-ctest-non-default-21463801 for Namespace ctest-non-default-21463801 is seen in contrail-api 2024-09-11 15:56:01,955 - DEBUG - Project uuid in Contrail is ce270ade-c3ef-4c52-96c8-07821114437b 2024-09-11 15:56:01,955 - DEBUG - Requesting: http://10.0.0.34:8108/Snh_NamespaceDatabaseList?namespace_uuid=6215df02-7f91-42b0-b5b3-17eccaab9062 2024-09-11 15:56:01,961 - INFO - Namespace ctest-non-default-21463801 with uuid 6215df02-7f91-42b0-b5b3-17eccaab9062 found in kube manager 2024-09-11 15:56:01,961 - INFO - Namespace ctest-non-default-21463801 verification passed 2024-09-11 15:56:01,966 - DEBUG - Pod ctest-pod-91448207 not in running state.Currently in Pending 2024-09-11 15:56:06,975 - DEBUG - Pod ctest-pod-91448207 not in running state.Currently in Pending 2024-09-11 15:56:11,983 - DEBUG - Pod ctest-pod-91448207 not in running state.Currently in Pending 2024-09-11 15:56:16,989 - INFO - Pod ctest-pod-91448207 is in running state.Got IP 10.47.255.251 2024-09-11 15:56:17,009 - DEBUG - Pod ctest-pod-91448207 has vmi 5a9b8770-7056-11ef-a3d3-fa163e6c300c 2024-09-11 15:56:17,009 - INFO - Verified pod ctest-pod-91448207 in contrail-api 2024-09-11 15:56:17,024 - DEBUG - VMI 5a9b8770-7056-11ef-a3d3-fa163e6c300c is active in agent 10.0.0.38 2024-09-11 15:56:17,024 - INFO - Verified Pod ctest-pod-91448207 in agent 10.0.0.38 2024-09-11 15:56:17,024 - DEBUG - Requesting: http://10.0.0.34:8108/Snh_VirtualMachineDatabaseList?x=cdd9d9d1-14dd-4148-9cd1-bdd9341c2524 2024-09-11 15:56:17,028 - INFO - Pod ctest-pod-91448207 with uuid cdd9d9d1-14dd-4148-9cd1-bdd9341c2524 found in kube manager 2024-09-11 15:56:17,028 - INFO - Pod ctest-pod-91448207 verification passed 2024-09-11 15:56:17,033 - INFO - Pod ctest-pod-28233703 is in running state.Got IP 10.47.255.250 2024-09-11 15:56:17,094 - DEBUG - Pod ctest-pod-28233703 has vmi 5b540fc0-7056-11ef-a3d3-fa163e6c300c 2024-09-11 15:56:17,094 - INFO - Verified pod ctest-pod-28233703 in contrail-api 2024-09-11 15:56:17,109 - DEBUG - VMI 5b540fc0-7056-11ef-a3d3-fa163e6c300c is active in agent 10.0.0.38 2024-09-11 15:56:17,109 - INFO - Verified Pod ctest-pod-28233703 in agent 10.0.0.38 2024-09-11 15:56:17,109 - DEBUG - Requesting: http://10.0.0.34:8108/Snh_VirtualMachineDatabaseList?x=b1c7193e-5809-4ae5-9004-59f9bb1fa90c 2024-09-11 15:56:17,114 - INFO - Pod ctest-pod-28233703 with uuid b1c7193e-5809-4ae5-9004-59f9bb1fa90c found in kube manager 2024-09-11 15:56:17,114 - INFO - Pod ctest-pod-28233703 verification passed 2024-09-11 15:56:17,120 - DEBUG - Pod ctest-pod-74662013 not in running state.Currently in Pending 2024-09-11 15:56:22,126 - INFO - Pod ctest-pod-74662013 is in running state.Got IP 10.47.255.249 2024-09-11 15:56:22,143 - DEBUG - Pod ctest-pod-74662013 has vmi 5ca2632c-7056-11ef-a3d3-fa163e6c300c 2024-09-11 15:56:22,144 - INFO - Verified pod ctest-pod-74662013 in contrail-api 2024-09-11 15:56:22,158 - DEBUG - VMI 5ca2632c-7056-11ef-a3d3-fa163e6c300c is active in agent 10.0.0.38 2024-09-11 15:56:22,159 - INFO - Verified Pod ctest-pod-74662013 in agent 10.0.0.38 2024-09-11 15:56:22,159 - DEBUG - Requesting: http://10.0.0.34:8108/Snh_VirtualMachineDatabaseList?x=12101a40-3c59-49f2-8af0-e3809c9e2c1a 2024-09-11 15:56:22,163 - INFO - Pod ctest-pod-74662013 with uuid 12101a40-3c59-49f2-8af0-e3809c9e2c1a found in kube manager 2024-09-11 15:56:22,163 - INFO - Pod ctest-pod-74662013 verification passed 2024-09-11 15:56:22,169 - INFO - Pod ctest-pod-40431095 is in running state.Got IP 10.47.255.248 2024-09-11 15:56:22,230 - DEBUG - Pod ctest-pod-40431095 has vmi 5d29501c-7056-11ef-a3d3-fa163e6c300c 2024-09-11 15:56:22,230 - INFO - Verified pod ctest-pod-40431095 in contrail-api 2024-09-11 15:56:22,243 - DEBUG - VMI 5d29501c-7056-11ef-a3d3-fa163e6c300c is active in agent 10.0.0.38 2024-09-11 15:56:22,243 - INFO - Verified Pod ctest-pod-40431095 in agent 10.0.0.38 2024-09-11 15:56:22,243 - DEBUG - Requesting: http://10.0.0.34:8108/Snh_VirtualMachineDatabaseList?x=ccdcc52f-177a-4037-8868-20d89f5bb548 2024-09-11 15:56:22,247 - INFO - Pod ctest-pod-40431095 with uuid ccdcc52f-177a-4037-8868-20d89f5bb548 found in kube manager 2024-09-11 15:56:22,248 - INFO - Pod ctest-pod-40431095 verification passed 2024-09-11 15:56:22,252 - INFO - Pod ctest-pod-74380417 is in running state.Got IP 10.47.255.247 2024-09-11 15:56:22,309 - DEBUG - Pod ctest-pod-74380417 has vmi 5dbe2bec-7056-11ef-a3d3-fa163e6c300c 2024-09-11 15:56:22,310 - INFO - Verified pod ctest-pod-74380417 in contrail-api 2024-09-11 15:56:22,323 - DEBUG - VMI 5dbe2bec-7056-11ef-a3d3-fa163e6c300c is active in agent 10.0.0.38 2024-09-11 15:56:22,323 - INFO - Verified Pod ctest-pod-74380417 in agent 10.0.0.38 2024-09-11 15:56:22,323 - DEBUG - Requesting: http://10.0.0.34:8108/Snh_VirtualMachineDatabaseList?x=4cfcb7ec-bf3b-49f0-a09e-92ccec4d8841 2024-09-11 15:56:22,327 - INFO - Pod ctest-pod-74380417 with uuid 4cfcb7ec-bf3b-49f0-a09e-92ccec4d8841 found in kube manager 2024-09-11 15:56:22,327 - INFO - Pod ctest-pod-74380417 verification passed 2024-09-11 15:56:22,332 - INFO - Pod ctest-pod-18271201 is in running state.Got IP 10.47.255.245 2024-09-11 15:56:22,351 - DEBUG - Pod ctest-pod-18271201 has vmi 5f559026-7056-11ef-a3d3-fa163e6c300c 2024-09-11 15:56:22,351 - INFO - Verified pod ctest-pod-18271201 in contrail-api 2024-09-11 15:56:22,365 - DEBUG - VMI 5f559026-7056-11ef-a3d3-fa163e6c300c is active in agent 10.0.0.38 2024-09-11 15:56:22,365 - INFO - Verified Pod ctest-pod-18271201 in agent 10.0.0.38 2024-09-11 15:56:22,365 - DEBUG - Requesting: http://10.0.0.34:8108/Snh_VirtualMachineDatabaseList?x=dfa84d6d-f1aa-4bea-81be-33b1bee51a81 2024-09-11 15:56:22,379 - INFO - Pod ctest-pod-18271201 with uuid dfa84d6d-f1aa-4bea-81be-33b1bee51a81 found in kube manager 2024-09-11 15:56:22,380 - INFO - Pod ctest-pod-18271201 verification passed 2024-09-11 15:56:22,385 - INFO - Pod ctest-pod-88373613 is in running state.Got IP 10.47.255.246 2024-09-11 15:56:22,445 - DEBUG - Pod ctest-pod-88373613 has vmi 5e3fe8c6-7056-11ef-a3d3-fa163e6c300c 2024-09-11 15:56:22,445 - INFO - Verified pod ctest-pod-88373613 in contrail-api 2024-09-11 15:56:22,458 - DEBUG - VMI 5e3fe8c6-7056-11ef-a3d3-fa163e6c300c is active in agent 10.0.0.38 2024-09-11 15:56:22,458 - INFO - Verified Pod ctest-pod-88373613 in agent 10.0.0.38 2024-09-11 15:56:22,458 - DEBUG - Requesting: http://10.0.0.34:8108/Snh_VirtualMachineDatabaseList?x=f478e0c2-5217-4cb3-9583-80c75189f0a8 2024-09-11 15:56:22,463 - INFO - Pod ctest-pod-88373613 with uuid f478e0c2-5217-4cb3-9583-80c75189f0a8 found in kube manager 2024-09-11 15:56:22,463 - INFO - Pod ctest-pod-88373613 verification passed 2024-09-11 15:56:22,467 - INFO - Pod ctest-pod-36502319 is in running state.Got IP 10.47.255.244 2024-09-11 15:56:22,526 - DEBUG - Pod ctest-pod-36502319 has vmi 6002361e-7056-11ef-a3d3-fa163e6c300c 2024-09-11 15:56:22,526 - INFO - Verified pod ctest-pod-36502319 in contrail-api 2024-09-11 15:56:22,539 - DEBUG - VMI 6002361e-7056-11ef-a3d3-fa163e6c300c is active in agent 10.0.0.38 2024-09-11 15:56:22,539 - INFO - Verified Pod ctest-pod-36502319 in agent 10.0.0.38 2024-09-11 15:56:22,539 - DEBUG - Requesting: http://10.0.0.34:8108/Snh_VirtualMachineDatabaseList?x=1a799bc1-041c-4d14-950b-5c340f6b95f6 2024-09-11 15:56:22,543 - INFO - Pod ctest-pod-36502319 with uuid 1a799bc1-041c-4d14-950b-5c340f6b95f6 found in kube manager 2024-09-11 15:56:22,544 - INFO - Pod ctest-pod-36502319 verification passed 2024-09-11 15:56:22,544 - INFO - ================================================================================ 2024-09-11 15:56:22,544 - INFO - STARTING TEST : test_ingress_rule_on_pod_with_service 2024-09-11 15:56:22,544 - INFO - TEST DESCRIPTION : Verify that ingress rules on a pod work fine when pods are accessed via a service Steps: 1. Create a service in namespace 1 over a pod which are endpoints of a service 2. Create a network policy to allow 1 pod of namespace 1 as ingress. 3. Verify that service respond correctly to the specified pod in the policy and service do not respond to anyone else. 4. Extend the policy and add the namespaceselector as namespace 2. 5. Verify that service become accessible to the pods of namespace 2. 2024-09-11 15:56:24,067 - DEBUG - Nothing to compare xmpp stats {'10.0.0.38': {'10.0.0.34': '0'}} with 2024-09-11 15:56:24,067 - INFO - Initial checks done. Running the testcase now 2024-09-11 15:56:24,067 - INFO - 2024-09-11 15:56:24,128 - DEBUG - [Pod ctest-pod-91448207] Cmd: echo ctest-pod-91448207 > /usr/share/nginx/html/index.html, Output: 2024-09-11 15:56:24,191 - DEBUG - [Pod ctest-pod-28233703] Cmd: echo ctest-pod-28233703 > /usr/share/nginx/html/index.html, Output: 2024-09-11 15:56:24,257 - DEBUG - [Pod ctest-pod-74662013] Cmd: echo ctest-pod-74662013 > /usr/share/nginx/html/index.html, Output: 2024-09-11 15:56:24,309 - DEBUG - [Pod ctest-pod-40431095] Cmd: echo ctest-pod-40431095 > /usr/share/nginx/html/index.html, Output: 2024-09-11 15:56:24,312 - DEBUG - Service ctest-nginx-svc-73476262 not present 2024-09-11 15:56:24,312 - INFO - Creating service ctest-nginx-svc-73476262 2024-09-11 15:56:24,329 - INFO - Created Service ctest-nginx-svc-73476262 2024-09-11 15:56:24,335 - WARNING - Service UUID 598b0a20-256f-4539-b364-be56751c2c71 not yet found in contrail-api 2024-09-11 15:56:25,343 - INFO - Validated that Service ctest-nginx-svc-73476262 is seen in contrail-api 2024-09-11 15:56:25,343 - DEBUG - Requesting: http://10.0.0.34:8108/Snh_LoadbalancerDatabaseList?x=598b0a20-256f-4539-b364-be56751c2c71 2024-09-11 15:56:25,349 - INFO - Service ctest-nginx-svc-73476262 with uuid 598b0a20-256f-4539-b364-be56751c2c71 found in kube manager 2024-09-11 15:56:25,349 - INFO - Service ctest-nginx-svc-73476262 verification passed 2024-09-11 15:56:26,435 - DEBUG - [Pod ctest-pod-74380417] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-28233703 2024-09-11 15:56:26,435 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 passed 2024-09-11 15:56:26,435 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd output: Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-28233703 2024-09-11 15:56:26,484 - DEBUG - [Pod ctest-pod-74380417] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: ctest-pod-28233703 Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-11 15:56:26,484 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 passed 2024-09-11 15:56:26,484 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd output: ctest-pod-28233703 Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-11 15:56:26,545 - DEBUG - [Pod ctest-pod-74380417] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-28233703 2024-09-11 15:56:26,545 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 passed 2024-09-11 15:56:26,546 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd output: Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-28233703 2024-09-11 15:56:26,605 - DEBUG - [Pod ctest-pod-74380417] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: ctest-pod-28233703 Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-11 15:56:26,605 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 passed 2024-09-11 15:56:26,605 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd output: ctest-pod-28233703 Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-11 15:56:26,657 - DEBUG - [Pod ctest-pod-74380417] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-91448207 2024-09-11 15:56:26,657 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 passed 2024-09-11 15:56:26,657 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd output: Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-91448207 2024-09-11 15:56:26,657 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-91448207': 1, 'ctest-pod-28233703': 4} 2024-09-11 15:56:26,730 - DEBUG - [Pod ctest-pod-18271201] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: ctest-pod-91448207 Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-11 15:56:26,730 - DEBUG - [Pod Pod ctest-pod-18271201] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 passed 2024-09-11 15:56:26,730 - DEBUG - [Pod Pod ctest-pod-18271201] Cmd output: ctest-pod-91448207 Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-11 15:56:26,792 - DEBUG - [Pod ctest-pod-18271201] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-91448207 2024-09-11 15:56:26,792 - DEBUG - [Pod Pod ctest-pod-18271201] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 passed 2024-09-11 15:56:26,792 - DEBUG - [Pod Pod ctest-pod-18271201] Cmd output: Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-91448207 2024-09-11 15:56:26,836 - DEBUG - [Pod ctest-pod-18271201] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: ctest-pod-28233703 Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-11 15:56:26,836 - DEBUG - [Pod Pod ctest-pod-18271201] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 passed 2024-09-11 15:56:26,836 - DEBUG - [Pod Pod ctest-pod-18271201] Cmd output: ctest-pod-28233703 Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-11 15:56:26,837 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-91448207': 2, 'ctest-pod-28233703': 1} 2024-09-11 15:56:26,840 - DEBUG - Network policy ingress-on-service not present 2024-09-11 15:56:26,840 - INFO - Creating Network Policy ingress-on-service 2024-09-11 15:56:26,849 - INFO - Network policy found in k8s 2024-09-11 15:56:26,849 - DEBUG - Requesting: http://10.0.0.34:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=8936844f-aaca-4813-adb6-dd55419719ee 2024-09-11 15:56:26,854 - INFO - Network Policy ingress-on-service with uuid 8936844f-aaca-4813-adb6-dd55419719ee found in kube manager 2024-09-11 15:56:26,884 - INFO - Network policy with name default-policy-management:k8s-allowall found in agent 2024-09-11 15:56:26,884 - INFO - Network policy with name default-policy-management:k8s-allowall associated with default ks8 2024-09-11 15:56:26,885 - DEBUG - Requesting: http://10.0.0.34:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=8936844f-aaca-4813-adb6-dd55419719ee 2024-09-11 15:56:26,912 - WARNING - Network policy with name ingress-on-service not found in agent 2024-09-11 15:56:28,913 - DEBUG - Requesting: http://10.0.0.34:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=8936844f-aaca-4813-adb6-dd55419719ee 2024-09-11 15:56:28,944 - INFO - Network policy with name ingress-on-service found in agent 2024-09-11 15:56:28,945 - INFO - Network Policy ingress-on-service verification passed 2024-09-11 15:56:34,012 - DEBUG - [Pod ctest-pod-74380417] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:56:34,013 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:56:34,013 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:56:39,275 - DEBUG - [Pod ctest-pod-74380417] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:56:39,275 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:56:39,275 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:56:44,335 - DEBUG - [Pod ctest-pod-74380417] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:56:44,336 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:56:44,336 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:56:49,403 - DEBUG - [Pod ctest-pod-74380417] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:56:49,404 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:56:49,404 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:56:54,555 - DEBUG - [Pod ctest-pod-74380417] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:56:54,555 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:56:54,555 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:56:59,624 - DEBUG - [Pod ctest-pod-74380417] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:56:59,624 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:56:59,624 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:57:04,684 - DEBUG - [Pod ctest-pod-74380417] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:57:04,684 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:57:04,684 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:57:09,760 - DEBUG - [Pod ctest-pod-74380417] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:57:09,760 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:57:09,760 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:57:14,827 - DEBUG - [Pod ctest-pod-74380417] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:57:14,827 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:57:14,827 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:57:19,893 - DEBUG - [Pod ctest-pod-74380417] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:57:19,893 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:57:19,893 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:57:19,893 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-91448207': 0, 'ctest-pod-28233703': 0} 2024-09-11 15:57:19,952 - DEBUG - [Pod ctest-pod-88373613] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-28233703 2024-09-11 15:57:19,952 - DEBUG - [Pod Pod ctest-pod-88373613] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 passed 2024-09-11 15:57:19,952 - DEBUG - [Pod Pod ctest-pod-88373613] Cmd output: Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-28233703 2024-09-11 15:57:20,017 - DEBUG - [Pod ctest-pod-88373613] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: ctest-pod-28233703 Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-11 15:57:20,017 - DEBUG - [Pod Pod ctest-pod-88373613] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 passed 2024-09-11 15:57:20,017 - DEBUG - [Pod Pod ctest-pod-88373613] Cmd output: ctest-pod-28233703 Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-11 15:57:20,069 - DEBUG - [Pod ctest-pod-88373613] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: ctest-pod-28233703 Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-11 15:57:20,069 - DEBUG - [Pod Pod ctest-pod-88373613] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 passed 2024-09-11 15:57:20,069 - DEBUG - [Pod Pod ctest-pod-88373613] Cmd output: ctest-pod-28233703 Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-11 15:57:20,140 - DEBUG - [Pod ctest-pod-88373613] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-28233703 2024-09-11 15:57:20,140 - DEBUG - [Pod Pod ctest-pod-88373613] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 passed 2024-09-11 15:57:20,140 - DEBUG - [Pod Pod ctest-pod-88373613] Cmd output: Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-28233703 2024-09-11 15:57:20,201 - DEBUG - [Pod ctest-pod-88373613] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: ctest-pod-91448207 Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-11 15:57:20,201 - DEBUG - [Pod Pod ctest-pod-88373613] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 passed 2024-09-11 15:57:20,202 - DEBUG - [Pod Pod ctest-pod-88373613] Cmd output: ctest-pod-91448207 Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-11 15:57:20,202 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-91448207': 1, 'ctest-pod-28233703': 4} 2024-09-11 15:57:25,267 - DEBUG - [Pod ctest-pod-18271201] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:57:25,267 - DEBUG - [Pod Pod ctest-pod-18271201] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:57:25,267 - DEBUG - [Pod Pod ctest-pod-18271201] Cmd output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:57:30,715 - DEBUG - [Pod ctest-pod-18271201] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:57:30,715 - DEBUG - [Pod Pod ctest-pod-18271201] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:57:30,715 - DEBUG - [Pod Pod ctest-pod-18271201] Cmd output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:57:35,781 - DEBUG - [Pod ctest-pod-18271201] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:57:35,781 - DEBUG - [Pod Pod ctest-pod-18271201] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:57:35,781 - DEBUG - [Pod Pod ctest-pod-18271201] Cmd output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:57:40,917 - DEBUG - [Pod ctest-pod-18271201] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:57:40,917 - DEBUG - [Pod Pod ctest-pod-18271201] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:57:40,917 - DEBUG - [Pod Pod ctest-pod-18271201] Cmd output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:57:46,268 - DEBUG - [Pod ctest-pod-18271201] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:57:46,268 - DEBUG - [Pod Pod ctest-pod-18271201] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:57:46,268 - DEBUG - [Pod Pod ctest-pod-18271201] Cmd output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:57:51,332 - DEBUG - [Pod ctest-pod-18271201] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:57:51,332 - DEBUG - [Pod Pod ctest-pod-18271201] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:57:51,332 - DEBUG - [Pod Pod ctest-pod-18271201] Cmd output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:57:56,411 - DEBUG - [Pod ctest-pod-18271201] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:57:56,411 - DEBUG - [Pod Pod ctest-pod-18271201] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:57:56,411 - DEBUG - [Pod Pod ctest-pod-18271201] Cmd output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:58:01,467 - DEBUG - [Pod ctest-pod-18271201] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:58:01,467 - DEBUG - [Pod Pod ctest-pod-18271201] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:58:01,467 - DEBUG - [Pod Pod ctest-pod-18271201] Cmd output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:58:06,522 - DEBUG - [Pod ctest-pod-18271201] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:58:06,523 - DEBUG - [Pod Pod ctest-pod-18271201] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:58:06,523 - DEBUG - [Pod Pod ctest-pod-18271201] Cmd output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:58:11,580 - DEBUG - [Pod ctest-pod-18271201] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:58:11,581 - DEBUG - [Pod Pod ctest-pod-18271201] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:58:11,581 - DEBUG - [Pod Pod ctest-pod-18271201] Cmd output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:58:11,581 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-91448207': 0, 'ctest-pod-28233703': 0} 2024-09-11 15:58:13,640 - DEBUG - [Pod ctest-pod-88373613] Cmd: ping -c 3 10.47.255.247, Output: PING 10.47.255.247 (10.47.255.247): 56 data bytes 64 bytes from 10.47.255.247: seq=0 ttl=63 time=0.906 ms 64 bytes from 10.47.255.247: seq=1 ttl=63 time=0.097 ms 64 bytes from 10.47.255.247: seq=2 ttl=63 time=0.115 ms --- 10.47.255.247 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.097/0.372/0.906 ms 2024-09-11 15:58:13,640 - INFO - Ping check to IP 10.47.255.247 from pod ctest-pod-88373613 with expectation True passed 2024-09-11 15:58:15,701 - DEBUG - [Pod ctest-pod-74380417] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=0.405 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.088 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.108 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.088/0.200/0.405 ms 2024-09-11 15:58:15,701 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-74380417 with expectation True passed 2024-09-11 15:58:17,768 - DEBUG - [Pod ctest-pod-18271201] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=0.458 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.071 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.082 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.071/0.203/0.458 ms 2024-09-11 15:58:17,768 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-18271201 with expectation True passed 2024-09-11 15:58:17,837 - DEBUG - [Pod ctest-pod-88373613] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-09-11 15:58:17,837 - DEBUG - [Pod Pod ctest-pod-88373613] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2024-09-11 15:58:17,837 - DEBUG - [Pod Pod ctest-pod-88373613] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-09-11 15:58:17,837 - INFO - wget check of of http://10.47.255.251 from ctest-pod-88373613 passed 2024-09-11 15:58:17,897 - DEBUG - [Pod ctest-pod-88373613] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-09-11 15:58:17,897 - DEBUG - [Pod Pod ctest-pod-88373613] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2024-09-11 15:58:17,897 - DEBUG - [Pod Pod ctest-pod-88373613] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-09-11 15:58:17,897 - INFO - wget check of of http://10.47.255.250 from ctest-pod-88373613 passed 2024-09-11 15:58:22,956 - DEBUG - [Pod ctest-pod-74380417] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-09-11 15:58:22,956 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-09-11 15:58:22,956 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-09-11 15:58:22,956 - INFO - wget check of of http://10.47.255.250 from ctest-pod-74380417 passed 2024-09-11 15:58:28,015 - DEBUG - [Pod ctest-pod-36502319] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) wget: download timed out 2024-09-11 15:58:28,016 - DEBUG - [Pod Pod ctest-pod-36502319] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.251 (10.47.255.251:80) wget: download timed out 2024-09-11 15:58:28,016 - DEBUG - [Pod Pod ctest-pod-36502319] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) wget: download timed out 2024-09-11 15:58:28,016 - INFO - wget check of of http://10.47.255.251 from ctest-pod-36502319 passed 2024-09-11 15:58:28,126 - INFO - Updating Network Policy ingress-on-service 2024-09-11 15:58:33,204 - DEBUG - [Pod ctest-pod-74380417] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:58:33,204 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:58:33,204 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:58:38,260 - DEBUG - [Pod ctest-pod-74380417] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:58:38,260 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:58:38,260 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:58:43,317 - DEBUG - [Pod ctest-pod-74380417] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:58:43,317 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:58:43,317 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:58:48,368 - DEBUG - [Pod ctest-pod-74380417] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:58:48,368 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:58:48,368 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:58:53,439 - DEBUG - [Pod ctest-pod-74380417] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:58:53,439 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:58:53,439 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:58:58,508 - DEBUG - [Pod ctest-pod-74380417] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:58:58,508 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:58:58,508 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:59:03,571 - DEBUG - [Pod ctest-pod-74380417] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:59:03,572 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:59:03,572 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:59:08,847 - DEBUG - [Pod ctest-pod-74380417] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:59:08,848 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:59:08,848 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:59:13,967 - DEBUG - [Pod ctest-pod-74380417] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:59:13,967 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:59:13,967 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:59:19,035 - DEBUG - [Pod ctest-pod-74380417] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:59:19,035 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:59:19,035 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd output: Connecting to 10.106.33.132:80 (10.106.33.132:80) wget: download timed out 2024-09-11 15:59:19,035 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-91448207': 0, 'ctest-pod-28233703': 0} 2024-09-11 15:59:19,101 - DEBUG - [Pod ctest-pod-88373613] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: ctest-pod-91448207 Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-11 15:59:19,101 - DEBUG - [Pod Pod ctest-pod-88373613] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 passed 2024-09-11 15:59:19,101 - DEBUG - [Pod Pod ctest-pod-88373613] Cmd output: ctest-pod-91448207 Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-11 15:59:19,164 - DEBUG - [Pod ctest-pod-88373613] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: ctest-pod-91448207 Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-11 15:59:19,165 - DEBUG - [Pod Pod ctest-pod-88373613] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 passed 2024-09-11 15:59:19,165 - DEBUG - [Pod Pod ctest-pod-88373613] Cmd output: ctest-pod-91448207 Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-11 15:59:19,226 - DEBUG - [Pod ctest-pod-88373613] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: ctest-pod-91448207 Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-11 15:59:19,226 - DEBUG - [Pod Pod ctest-pod-88373613] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 passed 2024-09-11 15:59:19,226 - DEBUG - [Pod Pod ctest-pod-88373613] Cmd output: ctest-pod-91448207 Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-11 15:59:19,284 - DEBUG - [Pod ctest-pod-88373613] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: ctest-pod-91448207 Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-11 15:59:19,284 - DEBUG - [Pod Pod ctest-pod-88373613] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 passed 2024-09-11 15:59:19,284 - DEBUG - [Pod Pod ctest-pod-88373613] Cmd output: ctest-pod-91448207 Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-11 15:59:19,350 - DEBUG - [Pod ctest-pod-88373613] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-28233703 2024-09-11 15:59:19,350 - DEBUG - [Pod Pod ctest-pod-88373613] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 passed 2024-09-11 15:59:19,350 - DEBUG - [Pod Pod ctest-pod-88373613] Cmd output: Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-28233703 2024-09-11 15:59:19,350 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-91448207': 4, 'ctest-pod-28233703': 1} 2024-09-11 15:59:19,408 - DEBUG - [Pod ctest-pod-18271201] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: ctest-pod-91448207 Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-11 15:59:19,409 - DEBUG - [Pod Pod ctest-pod-18271201] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 passed 2024-09-11 15:59:19,409 - DEBUG - [Pod Pod ctest-pod-18271201] Cmd output: ctest-pod-91448207 Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-11 15:59:19,464 - DEBUG - [Pod ctest-pod-18271201] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: ctest-pod-91448207 Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-11 15:59:19,464 - DEBUG - [Pod Pod ctest-pod-18271201] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 passed 2024-09-11 15:59:19,464 - DEBUG - [Pod Pod ctest-pod-18271201] Cmd output: ctest-pod-91448207 Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2024-09-11 15:59:19,527 - DEBUG - [Pod ctest-pod-18271201] Cmd: wget http://10.106.33.132:80/ -O - -T 5 -t 1 , Output: Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-28233703 2024-09-11 15:59:19,527 - DEBUG - [Pod Pod ctest-pod-18271201] Cmd wget http://10.106.33.132:80/ -O - -T 5 -t 1 passed 2024-09-11 15:59:19,527 - DEBUG - [Pod Pod ctest-pod-18271201] Cmd output: Connecting to 10.106.33.132:80 (10.106.33.132:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-28233703 2024-09-11 15:59:19,527 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-91448207': 2, 'ctest-pod-28233703': 1} 2024-09-11 15:59:21,587 - DEBUG - [Pod ctest-pod-88373613] Cmd: ping -c 3 10.47.255.247, Output: PING 10.47.255.247 (10.47.255.247): 56 data bytes 64 bytes from 10.47.255.247: seq=0 ttl=63 time=0.881 ms 64 bytes from 10.47.255.247: seq=1 ttl=63 time=0.077 ms 64 bytes from 10.47.255.247: seq=2 ttl=63 time=0.096 ms --- 10.47.255.247 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.077/0.351/0.881 ms 2024-09-11 15:59:21,587 - INFO - Ping check to IP 10.47.255.247 from pod ctest-pod-88373613 with expectation True passed 2024-09-11 15:59:23,647 - DEBUG - [Pod ctest-pod-74380417] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=2.268 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.079 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.101 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.079/0.816/2.268 ms 2024-09-11 15:59:23,648 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-74380417 with expectation True passed 2024-09-11 15:59:26,148 - DEBUG - [Pod ctest-pod-18271201] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=0.706 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.083 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.091 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.083/0.293/0.706 ms 2024-09-11 15:59:26,148 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-18271201 with expectation True passed 2024-09-11 15:59:26,205 - DEBUG - [Pod ctest-pod-88373613] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-09-11 15:59:26,205 - DEBUG - [Pod Pod ctest-pod-88373613] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2024-09-11 15:59:26,206 - DEBUG - [Pod Pod ctest-pod-88373613] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-09-11 15:59:26,206 - INFO - wget check of of http://10.47.255.251 from ctest-pod-88373613 passed 2024-09-11 15:59:26,270 - DEBUG - [Pod ctest-pod-88373613] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-09-11 15:59:26,270 - DEBUG - [Pod Pod ctest-pod-88373613] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2024-09-11 15:59:26,270 - DEBUG - [Pod Pod ctest-pod-88373613] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-09-11 15:59:26,270 - INFO - wget check of of http://10.47.255.250 from ctest-pod-88373613 passed 2024-09-11 15:59:31,327 - DEBUG - [Pod ctest-pod-74380417] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-09-11 15:59:31,327 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-09-11 15:59:31,327 - DEBUG - [Pod Pod ctest-pod-74380417] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2024-09-11 15:59:31,327 - INFO - wget check of of http://10.47.255.250 from ctest-pod-74380417 passed 2024-09-11 15:59:31,945 - DEBUG - [Pod ctest-pod-36502319] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-09-11 15:59:31,945 - DEBUG - [Pod Pod ctest-pod-36502319] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2024-09-11 15:59:31,945 - DEBUG - [Pod Pod ctest-pod-36502319] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2024-09-11 15:59:31,945 - INFO - wget check of of http://10.47.255.251 from ctest-pod-36502319 passed 2024-09-11 15:59:31,945 - INFO - Deleting Network Policy : ingress-on-service 2024-09-11 15:59:31,946 - INFO - Deleting service : ctest-nginx-svc-73476262 2024-09-11 15:59:32,184 - DEBUG - No XMPP flaps were noticed during the test 2024-09-11 15:59:32,184 - INFO - END TEST : test_ingress_rule_on_pod_with_service : PASSED[0:03:10] 2024-09-11 15:59:32,184 - INFO - -------------------------------------------------------------------------------- 2024-09-11 15:59:32,189 - INFO - Deleting pod ctest-new-default-58999978:ctest-pod-91448207 2024-09-11 15:59:32,190 - INFO - Deleting pod ctest-new-default-58999978:ctest-pod-28233703 2024-09-11 15:59:32,190 - INFO - Deleting pod ctest-non-default-21463801:ctest-pod-74662013 2024-09-11 15:59:32,191 - INFO - Deleting pod ctest-non-default-21463801:ctest-pod-40431095 2024-09-11 15:59:32,191 - INFO - Deleting pod ctest-new-default-58999978:ctest-pod-74380417 2024-09-11 15:59:32,191 - INFO - Deleting pod ctest-new-default-58999978:ctest-pod-88373613 2024-09-11 15:59:32,192 - INFO - Deleting pod ctest-non-default-21463801:ctest-pod-18271201 2024-09-11 15:59:32,192 - INFO - Deleting pod ctest-non-default-21463801:ctest-pod-36502319 2024-09-11 15:59:32,213 - WARNING - Pod uuid cdd9d9d1-14dd-4148-9cd1-bdd9341c2524 is still seen in agent 10.0.0.38 VM list 2024-09-11 15:59:32,339 - WARNING - Pod uuid 4cfcb7ec-bf3b-49f0-a09e-92ccec4d8841 is still seen in agent 10.0.0.38 VM list 2024-09-11 15:59:32,339 - WARNING - Pod uuid b1c7193e-5809-4ae5-9004-59f9bb1fa90c is still seen in agent 10.0.0.38 VM list 2024-09-11 15:59:32,341 - WARNING - Pod uuid 1a799bc1-041c-4d14-950b-5c340f6b95f6 is still seen in agent 10.0.0.38 VM list 2024-09-11 15:59:32,342 - WARNING - Pod uuid dfa84d6d-f1aa-4bea-81be-33b1bee51a81 is still seen in agent 10.0.0.38 VM list 2024-09-11 15:59:32,342 - WARNING - Pod uuid f478e0c2-5217-4cb3-9583-80c75189f0a8 is still seen in agent 10.0.0.38 VM list 2024-09-11 15:59:32,343 - WARNING - Pod uuid ccdcc52f-177a-4037-8868-20d89f5bb548 is still seen in agent 10.0.0.38 VM list 2024-09-11 15:59:32,343 - WARNING - Pod uuid 12101a40-3c59-49f2-8af0-e3809c9e2c1a is still seen in agent 10.0.0.38 VM list 2024-09-11 15:59:37,221 - DEBUG - Pod cdd9d9d1-14dd-4148-9cd1-bdd9341c2524 is not in agent 10.0.0.38 VM list 2024-09-11 15:59:37,228 - DEBUG - VMI 5a9b8770-7056-11ef-a3d3-fa163e6c300c is removed from agent 10.0.0.38 2024-09-11 15:59:37,228 - INFO - Verified that pod ctest-pod-91448207 is removed in agent 2024-09-11 15:59:37,366 - DEBUG - Pod b1c7193e-5809-4ae5-9004-59f9bb1fa90c is not in agent 10.0.0.38 VM list 2024-09-11 15:59:37,368 - WARNING - Pod uuid 1a799bc1-041c-4d14-950b-5c340f6b95f6 is still seen in agent 10.0.0.38 VM list 2024-09-11 15:59:37,368 - DEBUG - Pod ccdcc52f-177a-4037-8868-20d89f5bb548 is not in agent 10.0.0.38 VM list 2024-09-11 15:59:37,372 - WARNING - Pod uuid f478e0c2-5217-4cb3-9583-80c75189f0a8 is still seen in agent 10.0.0.38 VM list 2024-09-11 15:59:37,372 - WARNING - Pod uuid 4cfcb7ec-bf3b-49f0-a09e-92ccec4d8841 is still seen in agent 10.0.0.38 VM list 2024-09-11 15:59:37,373 - WARNING - Pod uuid dfa84d6d-f1aa-4bea-81be-33b1bee51a81 is still seen in agent 10.0.0.38 VM list 2024-09-11 15:59:37,379 - DEBUG - VMI 5d29501c-7056-11ef-a3d3-fa163e6c300c is removed from agent 10.0.0.38 2024-09-11 15:59:37,379 - INFO - Verified that pod ctest-pod-40431095 is removed in agent 2024-09-11 15:59:37,380 - DEBUG - VMI 5b540fc0-7056-11ef-a3d3-fa163e6c300c is removed from agent 10.0.0.38 2024-09-11 15:59:37,380 - INFO - Verified that pod ctest-pod-28233703 is removed in agent 2024-09-11 15:59:39,523 - DEBUG - Pod 12101a40-3c59-49f2-8af0-e3809c9e2c1a is not in agent 10.0.0.38 VM list 2024-09-11 15:59:39,530 - DEBUG - VMI 5ca2632c-7056-11ef-a3d3-fa163e6c300c is removed from agent 10.0.0.38 2024-09-11 15:59:39,530 - INFO - Verified that pod ctest-pod-74662013 is removed in agent 2024-09-11 15:59:42,391 - WARNING - Pod uuid 4cfcb7ec-bf3b-49f0-a09e-92ccec4d8841 is still seen in agent 10.0.0.38 VM list 2024-09-11 15:59:42,392 - WARNING - Pod uuid f478e0c2-5217-4cb3-9583-80c75189f0a8 is still seen in agent 10.0.0.38 VM list 2024-09-11 15:59:42,393 - WARNING - Pod uuid 1a799bc1-041c-4d14-950b-5c340f6b95f6 is still seen in agent 10.0.0.38 VM list 2024-09-11 15:59:42,394 - WARNING - Pod uuid dfa84d6d-f1aa-4bea-81be-33b1bee51a81 is still seen in agent 10.0.0.38 VM list 2024-09-11 15:59:47,411 - WARNING - Pod uuid f478e0c2-5217-4cb3-9583-80c75189f0a8 is still seen in agent 10.0.0.38 VM list 2024-09-11 15:59:47,412 - WARNING - Pod uuid 4cfcb7ec-bf3b-49f0-a09e-92ccec4d8841 is still seen in agent 10.0.0.38 VM list 2024-09-11 15:59:47,413 - WARNING - Pod uuid dfa84d6d-f1aa-4bea-81be-33b1bee51a81 is still seen in agent 10.0.0.38 VM list 2024-09-11 15:59:47,413 - WARNING - Pod uuid 1a799bc1-041c-4d14-950b-5c340f6b95f6 is still seen in agent 10.0.0.38 VM list 2024-09-11 15:59:52,431 - WARNING - Pod uuid 4cfcb7ec-bf3b-49f0-a09e-92ccec4d8841 is still seen in agent 10.0.0.38 VM list 2024-09-11 15:59:52,431 - WARNING - Pod uuid f478e0c2-5217-4cb3-9583-80c75189f0a8 is still seen in agent 10.0.0.38 VM list 2024-09-11 15:59:52,432 - WARNING - Pod uuid dfa84d6d-f1aa-4bea-81be-33b1bee51a81 is still seen in agent 10.0.0.38 VM list 2024-09-11 15:59:52,433 - WARNING - Pod uuid 1a799bc1-041c-4d14-950b-5c340f6b95f6 is still seen in agent 10.0.0.38 VM list 2024-09-11 15:59:57,458 - WARNING - Pod uuid f478e0c2-5217-4cb3-9583-80c75189f0a8 is still seen in agent 10.0.0.38 VM list 2024-09-11 15:59:57,458 - WARNING - Pod uuid 1a799bc1-041c-4d14-950b-5c340f6b95f6 is still seen in agent 10.0.0.38 VM list 2024-09-11 15:59:57,459 - WARNING - Pod uuid dfa84d6d-f1aa-4bea-81be-33b1bee51a81 is still seen in agent 10.0.0.38 VM list 2024-09-11 15:59:57,460 - WARNING - Pod uuid 4cfcb7ec-bf3b-49f0-a09e-92ccec4d8841 is still seen in agent 10.0.0.38 VM list 2024-09-11 16:00:02,487 - WARNING - Pod uuid dfa84d6d-f1aa-4bea-81be-33b1bee51a81 is still seen in agent 10.0.0.38 VM list 2024-09-11 16:00:02,488 - WARNING - Pod uuid f478e0c2-5217-4cb3-9583-80c75189f0a8 is still seen in agent 10.0.0.38 VM list 2024-09-11 16:00:02,488 - WARNING - Pod uuid 1a799bc1-041c-4d14-950b-5c340f6b95f6 is still seen in agent 10.0.0.38 VM list 2024-09-11 16:00:02,488 - WARNING - Pod uuid 4cfcb7ec-bf3b-49f0-a09e-92ccec4d8841 is still seen in agent 10.0.0.38 VM list 2024-09-11 16:00:07,508 - DEBUG - Pod dfa84d6d-f1aa-4bea-81be-33b1bee51a81 is not in agent 10.0.0.38 VM list 2024-09-11 16:00:07,512 - DEBUG - Pod 1a799bc1-041c-4d14-950b-5c340f6b95f6 is not in agent 10.0.0.38 VM list 2024-09-11 16:00:07,513 - DEBUG - Pod 4cfcb7ec-bf3b-49f0-a09e-92ccec4d8841 is not in agent 10.0.0.38 VM list 2024-09-11 16:00:07,514 - DEBUG - Pod f478e0c2-5217-4cb3-9583-80c75189f0a8 is not in agent 10.0.0.38 VM list 2024-09-11 16:00:07,528 - DEBUG - VMI 5e3fe8c6-7056-11ef-a3d3-fa163e6c300c is removed from agent 10.0.0.38 2024-09-11 16:00:07,528 - INFO - Verified that pod ctest-pod-88373613 is removed in agent 2024-09-11 16:00:07,528 - DEBUG - VMI 5f559026-7056-11ef-a3d3-fa163e6c300c is removed from agent 10.0.0.38 2024-09-11 16:00:07,528 - INFO - Verified that pod ctest-pod-18271201 is removed in agent 2024-09-11 16:00:07,530 - DEBUG - VMI 6002361e-7056-11ef-a3d3-fa163e6c300c is removed from agent 10.0.0.38 2024-09-11 16:00:07,530 - INFO - Verified that pod ctest-pod-36502319 is removed in agent 2024-09-11 16:00:07,530 - DEBUG - VMI 5dbe2bec-7056-11ef-a3d3-fa163e6c300c is removed from agent 10.0.0.38 2024-09-11 16:00:07,530 - INFO - Verified that pod ctest-pod-74380417 is removed in agent 2024-09-11 16:00:07,530 - INFO - Deleting namespace ctest-non-default-21463801 2024-09-11 16:00:07,548 - DEBUG - Namespace ctest-non-default-21463801 still in k8s 2024-09-11 16:00:09,557 - DEBUG - Namespace ctest-non-default-21463801 still in k8s 2024-09-11 16:00:12,587 - DEBUG - Namespace ctest-non-default-21463801 still in k8s 2024-09-11 16:00:14,593 - DEBUG - Namespace ctest-non-default-21463801 is not in k8s 2024-09-11 16:00:14,593 - DEBUG - No need to check project deletion in contrail-api 2024-09-11 16:00:14,593 - INFO - Deleting namespace ctest-new-default-58999978 2024-09-11 16:00:15,125 - DEBUG - Namespace ctest-new-default-58999978 still in k8s 2024-09-11 16:00:17,134 - DEBUG - Namespace ctest-new-default-58999978 still in k8s 2024-09-11 16:00:19,145 - DEBUG - Namespace ctest-new-default-58999978 still in k8s 2024-09-11 16:00:21,150 - DEBUG - Namespace ctest-new-default-58999978 is not in k8s 2024-09-11 16:00:21,150 - DEBUG - No need to check project deletion in contrail-api