2025-04-24 13:20:41,346 - INFO - Using existing project ['default-domain', 'k8s-default'](7d9b7429-4201-4d7d-832e-6fc52f1bf69c) 2025-04-24 13:20:41,365 - DEBUG - Namespace ctest-new-default-51845398 not present 2025-04-24 13:20:41,376 - INFO - Created namespace ctest-new-default-51845398 2025-04-24 13:20:44,381 - DEBUG - Namespace ctest-non-default-84573982 not present 2025-04-24 13:20:44,391 - INFO - Created namespace ctest-non-default-84573982 2025-04-24 13:20:47,408 - DEBUG - Pod ctest-pod-01098602 not present 2025-04-24 13:20:47,408 - INFO - Creating Pod ctest-pod-01098602 2025-04-24 13:20:47,434 - DEBUG - Pod : ctest-pod-01098602 UUID is f7ee65a6-04db-4683-9c3d-99070ef96598 2025-04-24 13:20:47,438 - DEBUG - Pod ctest-pod-30357626 not present 2025-04-24 13:20:47,439 - INFO - Creating Pod ctest-pod-30357626 2025-04-24 13:20:47,460 - DEBUG - Pod : ctest-pod-30357626 UUID is beba0172-a3cf-4861-92f7-6ca871efd8fb 2025-04-24 13:20:47,462 - DEBUG - Pod ctest-pod-39501585 not present 2025-04-24 13:20:47,462 - INFO - Creating Pod ctest-pod-39501585 2025-04-24 13:20:47,481 - DEBUG - Pod : ctest-pod-39501585 UUID is 8febe692-2cae-4b7e-a160-574d16543292 2025-04-24 13:20:47,484 - DEBUG - Pod ctest-pod-71544246 not present 2025-04-24 13:20:47,484 - INFO - Creating Pod ctest-pod-71544246 2025-04-24 13:20:47,499 - DEBUG - Pod : ctest-pod-71544246 UUID is 1bdbf034-43e6-4f02-b9d1-b15809652329 2025-04-24 13:20:47,505 - DEBUG - Pod ctest-pod-01650019 not present 2025-04-24 13:20:47,505 - INFO - Creating Pod ctest-pod-01650019 2025-04-24 13:20:47,524 - DEBUG - Pod : ctest-pod-01650019 UUID is 6618b694-24c8-484d-9cb9-de92c37826d9 2025-04-24 13:20:47,527 - DEBUG - Pod ctest-pod-38201478 not present 2025-04-24 13:20:47,528 - INFO - Creating Pod ctest-pod-38201478 2025-04-24 13:20:47,548 - DEBUG - Pod : ctest-pod-38201478 UUID is 839d34c8-23f1-4935-9760-d86e39a0eafe 2025-04-24 13:20:47,550 - DEBUG - Pod ctest-pod-49117006 not present 2025-04-24 13:20:47,550 - INFO - Creating Pod ctest-pod-49117006 2025-04-24 13:20:47,567 - DEBUG - Pod : ctest-pod-49117006 UUID is 1cb961d1-22db-4b68-a695-942b1060d2b9 2025-04-24 13:20:47,568 - DEBUG - Pod ctest-pod-76290634 not present 2025-04-24 13:20:47,569 - INFO - Creating Pod ctest-pod-76290634 2025-04-24 13:20:47,584 - DEBUG - Pod : ctest-pod-76290634 UUID is 3c4ec23f-2b67-4bc3-a84b-7fa09d59489d 2025-04-24 13:20:47,751 - INFO - Project default-domain:k8s-ctest-new-default-51845398 for Namespace ctest-new-default-51845398 is seen in contrail-api 2025-04-24 13:20:47,751 - DEBUG - Project uuid in Contrail is a2ca70b3-49e9-42cf-aac9-183d94bfb25c 2025-04-24 13:20:47,868 - DEBUG - Requesting: http://10.0.0.41:8108/Snh_NamespaceDatabaseList?namespace_uuid=da288c2f-8d74-4dea-bd56-34c5149cf5d1 2025-04-24 13:20:47,872 - INFO - Namespace ctest-new-default-51845398 with uuid da288c2f-8d74-4dea-bd56-34c5149cf5d1 found in kube manager 2025-04-24 13:20:47,872 - INFO - Namespace ctest-new-default-51845398 verification passed 2025-04-24 13:20:48,075 - INFO - Project default-domain:k8s-ctest-non-default-84573982 for Namespace ctest-non-default-84573982 is seen in contrail-api 2025-04-24 13:20:48,075 - DEBUG - Project uuid in Contrail is 005555b2-11ca-4ed3-a658-2c91e1be27dd 2025-04-24 13:20:48,076 - DEBUG - Requesting: http://10.0.0.41:8108/Snh_NamespaceDatabaseList?namespace_uuid=0b8a3007-fb35-4fa3-b02d-e9e54cfbf3be 2025-04-24 13:20:48,087 - INFO - Namespace ctest-non-default-84573982 with uuid 0b8a3007-fb35-4fa3-b02d-e9e54cfbf3be found in kube manager 2025-04-24 13:20:48,087 - INFO - Namespace ctest-non-default-84573982 verification passed 2025-04-24 13:20:48,093 - DEBUG - Pod ctest-pod-01098602 not in running state.Currently in Pending 2025-04-24 13:20:53,099 - DEBUG - Pod ctest-pod-01098602 not in running state.Currently in Pending 2025-04-24 13:20:58,105 - INFO - Pod ctest-pod-01098602 is in running state.Got IP 10.47.255.251 2025-04-24 13:20:58,116 - DEBUG - Pod ctest-pod-01098602 has vmi f02d5048-210e-11f0-a9b4-fa163eebfe9f 2025-04-24 13:20:58,116 - INFO - Verified pod ctest-pod-01098602 in contrail-api 2025-04-24 13:20:58,127 - DEBUG - VMI f02d5048-210e-11f0-a9b4-fa163eebfe9f is active in agent 10.0.0.27 2025-04-24 13:20:58,128 - INFO - Verified Pod ctest-pod-01098602 in agent 10.0.0.27 2025-04-24 13:20:58,128 - DEBUG - Requesting: http://10.0.0.41:8108/Snh_VirtualMachineDatabaseList?x=f7ee65a6-04db-4683-9c3d-99070ef96598 2025-04-24 13:20:58,131 - INFO - Pod ctest-pod-01098602 with uuid f7ee65a6-04db-4683-9c3d-99070ef96598 found in kube manager 2025-04-24 13:20:58,131 - INFO - Pod ctest-pod-01098602 verification passed 2025-04-24 13:20:58,136 - DEBUG - Pod ctest-pod-30357626 not in running state.Currently in Pending 2025-04-24 13:21:03,148 - INFO - Pod ctest-pod-30357626 is in running state.Got IP 10.47.255.250 2025-04-24 13:21:03,200 - DEBUG - Pod ctest-pod-30357626 has vmi f09f5dbe-210e-11f0-a9b4-fa163eebfe9f 2025-04-24 13:21:03,200 - INFO - Verified pod ctest-pod-30357626 in contrail-api 2025-04-24 13:21:03,211 - DEBUG - VMI f09f5dbe-210e-11f0-a9b4-fa163eebfe9f is active in agent 10.0.0.27 2025-04-24 13:21:03,211 - INFO - Verified Pod ctest-pod-30357626 in agent 10.0.0.27 2025-04-24 13:21:03,211 - DEBUG - Requesting: http://10.0.0.41:8108/Snh_VirtualMachineDatabaseList?x=beba0172-a3cf-4861-92f7-6ca871efd8fb 2025-04-24 13:21:03,215 - INFO - Pod ctest-pod-30357626 with uuid beba0172-a3cf-4861-92f7-6ca871efd8fb found in kube manager 2025-04-24 13:21:03,215 - INFO - Pod ctest-pod-30357626 verification passed 2025-04-24 13:21:03,220 - INFO - Pod ctest-pod-39501585 is in running state.Got IP 10.47.255.249 2025-04-24 13:21:03,230 - DEBUG - Pod ctest-pod-39501585 has vmi f133cb66-210e-11f0-a9b4-fa163eebfe9f 2025-04-24 13:21:03,231 - INFO - Verified pod ctest-pod-39501585 in contrail-api 2025-04-24 13:21:03,242 - DEBUG - VMI f133cb66-210e-11f0-a9b4-fa163eebfe9f is active in agent 10.0.0.27 2025-04-24 13:21:03,242 - INFO - Verified Pod ctest-pod-39501585 in agent 10.0.0.27 2025-04-24 13:21:03,242 - DEBUG - Requesting: http://10.0.0.41:8108/Snh_VirtualMachineDatabaseList?x=8febe692-2cae-4b7e-a160-574d16543292 2025-04-24 13:21:03,246 - INFO - Pod ctest-pod-39501585 with uuid 8febe692-2cae-4b7e-a160-574d16543292 found in kube manager 2025-04-24 13:21:03,246 - INFO - Pod ctest-pod-39501585 verification passed 2025-04-24 13:21:03,250 - INFO - Pod ctest-pod-71544246 is in running state.Got IP 10.47.255.248 2025-04-24 13:21:03,302 - DEBUG - Pod ctest-pod-71544246 has vmi f1a6ff0a-210e-11f0-a9b4-fa163eebfe9f 2025-04-24 13:21:03,302 - INFO - Verified pod ctest-pod-71544246 in contrail-api 2025-04-24 13:21:03,314 - DEBUG - VMI f1a6ff0a-210e-11f0-a9b4-fa163eebfe9f is active in agent 10.0.0.27 2025-04-24 13:21:03,314 - INFO - Verified Pod ctest-pod-71544246 in agent 10.0.0.27 2025-04-24 13:21:03,315 - DEBUG - Requesting: http://10.0.0.41:8108/Snh_VirtualMachineDatabaseList?x=1bdbf034-43e6-4f02-b9d1-b15809652329 2025-04-24 13:21:03,321 - INFO - Pod ctest-pod-71544246 with uuid 1bdbf034-43e6-4f02-b9d1-b15809652329 found in kube manager 2025-04-24 13:21:03,321 - INFO - Pod ctest-pod-71544246 verification passed 2025-04-24 13:21:03,327 - INFO - Pod ctest-pod-01650019 is in running state.Got IP 10.47.255.247 2025-04-24 13:21:03,337 - DEBUG - Pod ctest-pod-01650019 has vmi f223b20c-210e-11f0-a9b4-fa163eebfe9f 2025-04-24 13:21:03,337 - INFO - Verified pod ctest-pod-01650019 in contrail-api 2025-04-24 13:21:03,348 - DEBUG - VMI f223b20c-210e-11f0-a9b4-fa163eebfe9f is active in agent 10.0.0.27 2025-04-24 13:21:03,348 - INFO - Verified Pod ctest-pod-01650019 in agent 10.0.0.27 2025-04-24 13:21:03,348 - DEBUG - Requesting: http://10.0.0.41:8108/Snh_VirtualMachineDatabaseList?x=6618b694-24c8-484d-9cb9-de92c37826d9 2025-04-24 13:21:03,352 - INFO - Pod ctest-pod-01650019 with uuid 6618b694-24c8-484d-9cb9-de92c37826d9 found in kube manager 2025-04-24 13:21:03,352 - INFO - Pod ctest-pod-01650019 verification passed 2025-04-24 13:21:03,357 - INFO - Pod ctest-pod-49117006 is in running state.Got IP 10.47.255.245 2025-04-24 13:21:03,411 - DEBUG - Pod ctest-pod-49117006 has vmi f2f7b340-210e-11f0-a9b4-fa163eebfe9f 2025-04-24 13:21:03,411 - INFO - Verified pod ctest-pod-49117006 in contrail-api 2025-04-24 13:21:03,422 - DEBUG - VMI f2f7b340-210e-11f0-a9b4-fa163eebfe9f is active in agent 10.0.0.27 2025-04-24 13:21:03,422 - INFO - Verified Pod ctest-pod-49117006 in agent 10.0.0.27 2025-04-24 13:21:03,422 - DEBUG - Requesting: http://10.0.0.41:8108/Snh_VirtualMachineDatabaseList?x=1cb961d1-22db-4b68-a695-942b1060d2b9 2025-04-24 13:21:03,426 - INFO - Pod ctest-pod-49117006 with uuid 1cb961d1-22db-4b68-a695-942b1060d2b9 found in kube manager 2025-04-24 13:21:03,426 - INFO - Pod ctest-pod-49117006 verification passed 2025-04-24 13:21:03,432 - INFO - Pod ctest-pod-38201478 is in running state.Got IP 10.47.255.246 2025-04-24 13:21:03,484 - DEBUG - Pod ctest-pod-38201478 has vmi f29452dc-210e-11f0-a9b4-fa163eebfe9f 2025-04-24 13:21:03,484 - INFO - Verified pod ctest-pod-38201478 in contrail-api 2025-04-24 13:21:03,495 - DEBUG - VMI f29452dc-210e-11f0-a9b4-fa163eebfe9f is active in agent 10.0.0.27 2025-04-24 13:21:03,495 - INFO - Verified Pod ctest-pod-38201478 in agent 10.0.0.27 2025-04-24 13:21:03,495 - DEBUG - Requesting: http://10.0.0.41:8108/Snh_VirtualMachineDatabaseList?x=839d34c8-23f1-4935-9760-d86e39a0eafe 2025-04-24 13:21:03,499 - INFO - Pod ctest-pod-38201478 with uuid 839d34c8-23f1-4935-9760-d86e39a0eafe found in kube manager 2025-04-24 13:21:03,499 - INFO - Pod ctest-pod-38201478 verification passed 2025-04-24 13:21:03,503 - INFO - Pod ctest-pod-76290634 is in running state.Got IP 10.47.255.244 2025-04-24 13:21:03,514 - DEBUG - Pod ctest-pod-76290634 has vmi f363dfb6-210e-11f0-a9b4-fa163eebfe9f 2025-04-24 13:21:03,514 - INFO - Verified pod ctest-pod-76290634 in contrail-api 2025-04-24 13:21:03,524 - DEBUG - VMI f363dfb6-210e-11f0-a9b4-fa163eebfe9f is active in agent 10.0.0.27 2025-04-24 13:21:03,524 - INFO - Verified Pod ctest-pod-76290634 in agent 10.0.0.27 2025-04-24 13:21:03,525 - DEBUG - Requesting: http://10.0.0.41:8108/Snh_VirtualMachineDatabaseList?x=3c4ec23f-2b67-4bc3-a84b-7fa09d59489d 2025-04-24 13:21:03,528 - INFO - Pod ctest-pod-76290634 with uuid 3c4ec23f-2b67-4bc3-a84b-7fa09d59489d found in kube manager 2025-04-24 13:21:03,528 - INFO - Pod ctest-pod-76290634 verification passed 2025-04-24 13:21:03,528 - INFO - ================================================================================ 2025-04-24 13:21:03,528 - INFO - STARTING TEST : test_ingress_rule_on_pod_with_service 2025-04-24 13:21:03,528 - INFO - TEST DESCRIPTION : Verify that ingress rules on a pod work fine when pods are accessed via a service Steps: 1. Create a service in namespace 1 over a pod which are endpoints of a service 2. Create a network policy to allow 1 pod of namespace 1 as ingress. 3. Verify that service respond correctly to the specified pod in the policy and service do not respond to anyone else. 4. Extend the policy and add the namespaceselector as namespace 2. 5. Verify that service become accessible to the pods of namespace 2. 2025-04-24 13:21:03,710 - DEBUG - Nothing to compare xmpp stats {'10.0.0.27': {'10.0.0.41': '0'}} with 2025-04-24 13:21:03,710 - INFO - Initial checks done. Running the testcase now 2025-04-24 13:21:03,710 - INFO - 2025-04-24 13:21:03,758 - DEBUG - [Pod ctest-pod-01098602] Cmd: echo ctest-pod-01098602 > /usr/share/nginx/html/index.html, Output: 2025-04-24 13:21:03,807 - DEBUG - [Pod ctest-pod-30357626] Cmd: echo ctest-pod-30357626 > /usr/share/nginx/html/index.html, Output: 2025-04-24 13:21:03,858 - DEBUG - [Pod ctest-pod-39501585] Cmd: echo ctest-pod-39501585 > /usr/share/nginx/html/index.html, Output: 2025-04-24 13:21:03,906 - DEBUG - [Pod ctest-pod-71544246] Cmd: echo ctest-pod-71544246 > /usr/share/nginx/html/index.html, Output: 2025-04-24 13:21:03,909 - DEBUG - Service ctest-nginx-svc-25324750 not present 2025-04-24 13:21:03,909 - INFO - Creating service ctest-nginx-svc-25324750 2025-04-24 13:21:03,935 - INFO - Created Service ctest-nginx-svc-25324750 2025-04-24 13:21:03,942 - WARNING - Service UUID 7e872847-f4cb-4893-b791-df7c45ccc632 not yet found in contrail-api 2025-04-24 13:21:04,953 - INFO - Validated that Service ctest-nginx-svc-25324750 is seen in contrail-api 2025-04-24 13:21:04,953 - DEBUG - Requesting: http://10.0.0.41:8108/Snh_LoadbalancerDatabaseList?x=7e872847-f4cb-4893-b791-df7c45ccc632 2025-04-24 13:21:04,957 - INFO - Service ctest-nginx-svc-25324750 with uuid 7e872847-f4cb-4893-b791-df7c45ccc632 found in kube manager 2025-04-24 13:21:04,957 - INFO - Service ctest-nginx-svc-25324750 verification passed 2025-04-24 13:21:05,020 - DEBUG - [Pod ctest-pod-01650019] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: ctest-pod-30357626 Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-04-24 13:21:05,020 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 passed 2025-04-24 13:21:05,020 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd output: ctest-pod-30357626 Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-04-24 13:21:05,065 - DEBUG - [Pod ctest-pod-01650019] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-30357626 2025-04-24 13:21:05,065 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 passed 2025-04-24 13:21:05,065 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd output: Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-30357626 2025-04-24 13:21:05,115 - DEBUG - [Pod ctest-pod-01650019] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: ctest-pod-30357626 Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-04-24 13:21:05,115 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 passed 2025-04-24 13:21:05,115 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd output: ctest-pod-30357626 Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-04-24 13:21:05,170 - DEBUG - [Pod ctest-pod-01650019] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: ctest-pod-30357626 Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-04-24 13:21:05,170 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 passed 2025-04-24 13:21:05,170 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd output: ctest-pod-30357626 Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-04-24 13:21:05,222 - DEBUG - [Pod ctest-pod-01650019] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-30357626 2025-04-24 13:21:05,222 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 passed 2025-04-24 13:21:05,222 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd output: Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-30357626 2025-04-24 13:21:05,280 - DEBUG - [Pod ctest-pod-01650019] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: ctest-pod-01098602 Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-04-24 13:21:05,280 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 passed 2025-04-24 13:21:05,280 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd output: ctest-pod-01098602 Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-04-24 13:21:05,280 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-01098602': 1, 'ctest-pod-30357626': 5} 2025-04-24 13:21:05,336 - DEBUG - [Pod ctest-pod-49117006] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-30357626 2025-04-24 13:21:05,336 - DEBUG - [Pod Pod ctest-pod-49117006] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 passed 2025-04-24 13:21:05,336 - DEBUG - [Pod Pod ctest-pod-49117006] Cmd output: Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-30357626 2025-04-24 13:21:05,382 - DEBUG - [Pod ctest-pod-49117006] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: ctest-pod-30357626 Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-04-24 13:21:05,382 - DEBUG - [Pod Pod ctest-pod-49117006] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 passed 2025-04-24 13:21:05,382 - DEBUG - [Pod Pod ctest-pod-49117006] Cmd output: ctest-pod-30357626 Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-04-24 13:21:05,430 - DEBUG - [Pod ctest-pod-49117006] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-01098602 2025-04-24 13:21:05,430 - DEBUG - [Pod Pod ctest-pod-49117006] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 passed 2025-04-24 13:21:05,430 - DEBUG - [Pod Pod ctest-pod-49117006] Cmd output: Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-01098602 2025-04-24 13:21:05,430 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-01098602': 1, 'ctest-pod-30357626': 2} 2025-04-24 13:21:05,489 - DEBUG - Network policy ingress-on-service not present 2025-04-24 13:21:05,489 - INFO - Creating Network Policy ingress-on-service 2025-04-24 13:21:05,500 - INFO - Network policy found in k8s 2025-04-24 13:21:05,501 - DEBUG - Requesting: http://10.0.0.41:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=6dc2ef7b-6ba0-4beb-af15-ec80a1ae94b8 2025-04-24 13:21:05,507 - INFO - Network Policy ingress-on-service with uuid 6dc2ef7b-6ba0-4beb-af15-ec80a1ae94b8 found in kube manager 2025-04-24 13:21:05,534 - INFO - Network policy with name default-policy-management:k8s-allowall found in agent 2025-04-24 13:21:05,535 - INFO - Network policy with name default-policy-management:k8s-allowall associated with default ks8 2025-04-24 13:21:05,535 - DEBUG - Requesting: http://10.0.0.41:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=6dc2ef7b-6ba0-4beb-af15-ec80a1ae94b8 2025-04-24 13:21:05,560 - WARNING - Network policy with name ingress-on-service not found in agent 2025-04-24 13:21:07,561 - DEBUG - Requesting: http://10.0.0.41:8108/Snh_NetworkPolicyDatabaseList?network_policy_uuid=6dc2ef7b-6ba0-4beb-af15-ec80a1ae94b8 2025-04-24 13:21:07,588 - INFO - Network policy with name ingress-on-service found in agent 2025-04-24 13:21:07,588 - INFO - Network Policy ingress-on-service verification passed 2025-04-24 13:21:12,632 - DEBUG - [Pod ctest-pod-01650019] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:21:12,632 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:21:12,632 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:21:17,681 - DEBUG - [Pod ctest-pod-01650019] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:21:17,681 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:21:17,681 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:21:22,730 - DEBUG - [Pod ctest-pod-01650019] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:21:22,730 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:21:22,730 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:21:27,786 - DEBUG - [Pod ctest-pod-01650019] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:21:27,786 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:21:27,786 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:21:32,828 - DEBUG - [Pod ctest-pod-01650019] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:21:32,828 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:21:32,828 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:21:37,877 - DEBUG - [Pod ctest-pod-01650019] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:21:37,877 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:21:37,877 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:21:42,920 - DEBUG - [Pod ctest-pod-01650019] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:21:42,920 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:21:42,920 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:21:47,968 - DEBUG - [Pod ctest-pod-01650019] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:21:47,968 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:21:47,968 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:21:53,024 - DEBUG - [Pod ctest-pod-01650019] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:21:53,024 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:21:53,024 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:21:58,072 - DEBUG - [Pod ctest-pod-01650019] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:21:58,072 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:21:58,072 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:21:58,072 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-01098602': 0, 'ctest-pod-30357626': 0} 2025-04-24 13:21:58,121 - DEBUG - [Pod ctest-pod-38201478] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: ctest-pod-01098602 Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-04-24 13:21:58,121 - DEBUG - [Pod Pod ctest-pod-38201478] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 passed 2025-04-24 13:21:58,121 - DEBUG - [Pod Pod ctest-pod-38201478] Cmd output: ctest-pod-01098602 Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-04-24 13:21:58,170 - DEBUG - [Pod ctest-pod-38201478] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: ctest-pod-01098602 Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-04-24 13:21:58,172 - DEBUG - [Pod Pod ctest-pod-38201478] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 passed 2025-04-24 13:21:58,172 - DEBUG - [Pod Pod ctest-pod-38201478] Cmd output: ctest-pod-01098602 Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-04-24 13:21:58,222 - DEBUG - [Pod ctest-pod-38201478] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-01098602 2025-04-24 13:21:58,222 - DEBUG - [Pod Pod ctest-pod-38201478] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 passed 2025-04-24 13:21:58,222 - DEBUG - [Pod Pod ctest-pod-38201478] Cmd output: Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-01098602 2025-04-24 13:21:58,269 - DEBUG - [Pod ctest-pod-38201478] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: ctest-pod-01098602 Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-04-24 13:21:58,269 - DEBUG - [Pod Pod ctest-pod-38201478] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 passed 2025-04-24 13:21:58,269 - DEBUG - [Pod Pod ctest-pod-38201478] Cmd output: ctest-pod-01098602 Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-04-24 13:21:58,314 - DEBUG - [Pod ctest-pod-38201478] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: ctest-pod-01098602 Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-04-24 13:21:58,314 - DEBUG - [Pod Pod ctest-pod-38201478] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 passed 2025-04-24 13:21:58,314 - DEBUG - [Pod Pod ctest-pod-38201478] Cmd output: ctest-pod-01098602 Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-04-24 13:21:58,366 - DEBUG - [Pod ctest-pod-38201478] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: ctest-pod-30357626 Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-04-24 13:21:58,366 - DEBUG - [Pod Pod ctest-pod-38201478] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 passed 2025-04-24 13:21:58,366 - DEBUG - [Pod Pod ctest-pod-38201478] Cmd output: ctest-pod-30357626 Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-04-24 13:21:58,366 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-01098602': 5, 'ctest-pod-30357626': 1} 2025-04-24 13:22:03,424 - DEBUG - [Pod ctest-pod-49117006] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:22:03,424 - DEBUG - [Pod Pod ctest-pod-49117006] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:22:03,424 - DEBUG - [Pod Pod ctest-pod-49117006] Cmd output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:22:08,473 - DEBUG - [Pod ctest-pod-49117006] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:22:08,473 - DEBUG - [Pod Pod ctest-pod-49117006] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:22:08,473 - DEBUG - [Pod Pod ctest-pod-49117006] Cmd output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:22:13,530 - DEBUG - [Pod ctest-pod-49117006] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:22:13,530 - DEBUG - [Pod Pod ctest-pod-49117006] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:22:13,530 - DEBUG - [Pod Pod ctest-pod-49117006] Cmd output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:22:18,580 - DEBUG - [Pod ctest-pod-49117006] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:22:18,580 - DEBUG - [Pod Pod ctest-pod-49117006] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:22:18,580 - DEBUG - [Pod Pod ctest-pod-49117006] Cmd output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:22:23,625 - DEBUG - [Pod ctest-pod-49117006] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:22:23,625 - DEBUG - [Pod Pod ctest-pod-49117006] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:22:23,625 - DEBUG - [Pod Pod ctest-pod-49117006] Cmd output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:22:28,680 - DEBUG - [Pod ctest-pod-49117006] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:22:28,680 - DEBUG - [Pod Pod ctest-pod-49117006] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:22:28,680 - DEBUG - [Pod Pod ctest-pod-49117006] Cmd output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:22:33,733 - DEBUG - [Pod ctest-pod-49117006] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:22:33,733 - DEBUG - [Pod Pod ctest-pod-49117006] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:22:33,733 - DEBUG - [Pod Pod ctest-pod-49117006] Cmd output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:22:38,785 - DEBUG - [Pod ctest-pod-49117006] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:22:38,785 - DEBUG - [Pod Pod ctest-pod-49117006] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:22:38,785 - DEBUG - [Pod Pod ctest-pod-49117006] Cmd output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:22:43,833 - DEBUG - [Pod ctest-pod-49117006] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:22:43,833 - DEBUG - [Pod Pod ctest-pod-49117006] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:22:43,833 - DEBUG - [Pod Pod ctest-pod-49117006] Cmd output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:22:48,880 - DEBUG - [Pod ctest-pod-49117006] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:22:48,880 - DEBUG - [Pod Pod ctest-pod-49117006] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:22:48,880 - DEBUG - [Pod Pod ctest-pod-49117006] Cmd output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:22:48,880 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-01098602': 0, 'ctest-pod-30357626': 0} 2025-04-24 13:22:50,925 - DEBUG - [Pod ctest-pod-38201478] Cmd: ping -c 3 10.47.255.247, Output: PING 10.47.255.247 (10.47.255.247): 56 data bytes 64 bytes from 10.47.255.247: seq=0 ttl=63 time=2.068 ms 64 bytes from 10.47.255.247: seq=1 ttl=63 time=0.067 ms 64 bytes from 10.47.255.247: seq=2 ttl=63 time=0.080 ms --- 10.47.255.247 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.067/0.738/2.068 ms 2025-04-24 13:22:50,925 - INFO - Ping check to IP 10.47.255.247 from pod ctest-pod-38201478 with expectation True passed 2025-04-24 13:22:52,972 - DEBUG - [Pod ctest-pod-01650019] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=0.376 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.076 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.119 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.076/0.190/0.376 ms 2025-04-24 13:22:52,972 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-01650019 with expectation True passed 2025-04-24 13:22:55,021 - DEBUG - [Pod ctest-pod-49117006] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=0.898 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.072 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.091 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.072/0.353/0.898 ms 2025-04-24 13:22:55,021 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-49117006 with expectation True passed 2025-04-24 13:22:55,080 - DEBUG - [Pod ctest-pod-38201478] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-04-24 13:22:55,080 - DEBUG - [Pod Pod ctest-pod-38201478] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2025-04-24 13:22:55,080 - DEBUG - [Pod Pod ctest-pod-38201478] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-04-24 13:22:55,080 - INFO - wget check of of http://10.47.255.251 from ctest-pod-38201478 passed 2025-04-24 13:22:55,131 - DEBUG - [Pod ctest-pod-38201478] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-04-24 13:22:55,131 - DEBUG - [Pod Pod ctest-pod-38201478] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2025-04-24 13:22:55,131 - DEBUG - [Pod Pod ctest-pod-38201478] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-04-24 13:22:55,131 - INFO - wget check of of http://10.47.255.250 from ctest-pod-38201478 passed 2025-04-24 13:23:00,161 - DEBUG - [Pod ctest-pod-01650019] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2025-04-24 13:23:00,161 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2025-04-24 13:23:00,161 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2025-04-24 13:23:00,161 - INFO - wget check of of http://10.47.255.250 from ctest-pod-01650019 passed 2025-04-24 13:23:05,209 - DEBUG - [Pod ctest-pod-76290634] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) wget: download timed out 2025-04-24 13:23:05,209 - DEBUG - [Pod Pod ctest-pod-76290634] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.251 (10.47.255.251:80) wget: download timed out 2025-04-24 13:23:05,209 - DEBUG - [Pod Pod ctest-pod-76290634] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) wget: download timed out 2025-04-24 13:23:05,209 - INFO - wget check of of http://10.47.255.251 from ctest-pod-76290634 passed 2025-04-24 13:23:05,216 - INFO - Updating Network Policy ingress-on-service 2025-04-24 13:23:10,272 - DEBUG - [Pod ctest-pod-01650019] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:23:10,272 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:23:10,272 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:23:10,750 - WARNING - ApiException is caught (0) Reason: Connection is already closed. . Retrying... 2025-04-24 13:23:21,060 - DEBUG - [Pod ctest-pod-01650019] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:23:21,060 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:23:21,060 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:23:26,104 - DEBUG - [Pod ctest-pod-01650019] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:23:26,105 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:23:26,105 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:23:31,152 - DEBUG - [Pod ctest-pod-01650019] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:23:31,152 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:23:31,152 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:23:36,196 - DEBUG - [Pod ctest-pod-01650019] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:23:36,196 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:23:36,196 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:23:41,245 - DEBUG - [Pod ctest-pod-01650019] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:23:41,245 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:23:41,245 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:23:46,284 - DEBUG - [Pod ctest-pod-01650019] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:23:46,284 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:23:46,284 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:23:51,332 - DEBUG - [Pod ctest-pod-01650019] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:23:51,332 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:23:51,332 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:23:56,376 - DEBUG - [Pod ctest-pod-01650019] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:23:56,377 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:23:56,377 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:24:01,420 - DEBUG - [Pod ctest-pod-01650019] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:24:01,421 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 failed. Output :Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:24:01,421 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd output: Connecting to 10.103.245.21:80 (10.103.245.21:80) wget: download timed out 2025-04-24 13:24:01,421 - INFO - As expected, responses not seen from podsHits : {'ctest-pod-01098602': 0, 'ctest-pod-30357626': 0} 2025-04-24 13:24:01,474 - DEBUG - [Pod ctest-pod-38201478] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: ctest-pod-01098602 Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-04-24 13:24:01,474 - DEBUG - [Pod Pod ctest-pod-38201478] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 passed 2025-04-24 13:24:01,474 - DEBUG - [Pod Pod ctest-pod-38201478] Cmd output: ctest-pod-01098602 Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-04-24 13:24:01,523 - DEBUG - [Pod ctest-pod-38201478] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: ctest-pod-01098602 Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-04-24 13:24:01,523 - DEBUG - [Pod Pod ctest-pod-38201478] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 passed 2025-04-24 13:24:01,523 - DEBUG - [Pod Pod ctest-pod-38201478] Cmd output: ctest-pod-01098602 Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-04-24 13:24:01,582 - DEBUG - [Pod ctest-pod-38201478] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: ctest-pod-01098602 Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-04-24 13:24:01,582 - DEBUG - [Pod Pod ctest-pod-38201478] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 passed 2025-04-24 13:24:01,582 - DEBUG - [Pod Pod ctest-pod-38201478] Cmd output: ctest-pod-01098602 Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-04-24 13:24:01,633 - DEBUG - [Pod ctest-pod-38201478] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-30357626 2025-04-24 13:24:01,634 - DEBUG - [Pod Pod ctest-pod-38201478] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 passed 2025-04-24 13:24:01,634 - DEBUG - [Pod Pod ctest-pod-38201478] Cmd output: Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-30357626 2025-04-24 13:24:01,634 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-01098602': 3, 'ctest-pod-30357626': 1} 2025-04-24 13:24:01,678 - DEBUG - [Pod ctest-pod-49117006] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: ctest-pod-30357626 Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-04-24 13:24:01,678 - DEBUG - [Pod Pod ctest-pod-49117006] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 passed 2025-04-24 13:24:01,678 - DEBUG - [Pod Pod ctest-pod-49117006] Cmd output: ctest-pod-30357626 Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout 2025-04-24 13:24:01,728 - DEBUG - [Pod ctest-pod-49117006] Cmd: wget http://10.103.245.21:80/ -O - -T 5 -t 1 , Output: Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-01098602 2025-04-24 13:24:01,728 - DEBUG - [Pod Pod ctest-pod-49117006] Cmd wget http://10.103.245.21:80/ -O - -T 5 -t 1 passed 2025-04-24 13:24:01,728 - DEBUG - [Pod Pod ctest-pod-49117006] Cmd output: Connecting to 10.103.245.21:80 (10.103.245.21:80) writing to stdout - 100% |********************************| 19 0:00:00 ETA written to stdout ctest-pod-01098602 2025-04-24 13:24:01,728 - INFO - Responses seen from all pods, lb seems fine.Hits : {'ctest-pod-01098602': 1, 'ctest-pod-30357626': 1} 2025-04-24 13:24:03,772 - DEBUG - [Pod ctest-pod-38201478] Cmd: ping -c 3 10.47.255.247, Output: PING 10.47.255.247 (10.47.255.247): 56 data bytes 64 bytes from 10.47.255.247: seq=0 ttl=63 time=0.506 ms 64 bytes from 10.47.255.247: seq=1 ttl=63 time=0.062 ms 64 bytes from 10.47.255.247: seq=2 ttl=63 time=0.073 ms --- 10.47.255.247 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.062/0.213/0.506 ms 2025-04-24 13:24:03,773 - INFO - Ping check to IP 10.47.255.247 from pod ctest-pod-38201478 with expectation True passed 2025-04-24 13:24:05,824 - DEBUG - [Pod ctest-pod-01650019] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=0.607 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.055 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.068 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.055/0.243/0.607 ms 2025-04-24 13:24:05,825 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-01650019 with expectation True passed 2025-04-24 13:24:07,868 - DEBUG - [Pod ctest-pod-49117006] Cmd: ping -c 3 10.47.255.246, Output: PING 10.47.255.246 (10.47.255.246): 56 data bytes 64 bytes from 10.47.255.246: seq=0 ttl=63 time=1.226 ms 64 bytes from 10.47.255.246: seq=1 ttl=63 time=0.057 ms 64 bytes from 10.47.255.246: seq=2 ttl=63 time=0.059 ms --- 10.47.255.246 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.057/0.447/1.226 ms 2025-04-24 13:24:07,868 - INFO - Ping check to IP 10.47.255.246 from pod ctest-pod-49117006 with expectation True passed 2025-04-24 13:24:07,920 - DEBUG - [Pod ctest-pod-38201478] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-04-24 13:24:07,920 - DEBUG - [Pod Pod ctest-pod-38201478] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2025-04-24 13:24:07,921 - DEBUG - [Pod Pod ctest-pod-38201478] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-04-24 13:24:07,921 - INFO - wget check of of http://10.47.255.251 from ctest-pod-38201478 passed 2025-04-24 13:24:07,966 - DEBUG - [Pod ctest-pod-38201478] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-04-24 13:24:07,966 - DEBUG - [Pod Pod ctest-pod-38201478] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 passed 2025-04-24 13:24:07,966 - DEBUG - [Pod Pod ctest-pod-38201478] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-04-24 13:24:07,966 - INFO - wget check of of http://10.47.255.250 from ctest-pod-38201478 passed 2025-04-24 13:24:13,016 - DEBUG - [Pod ctest-pod-01650019] Cmd: wget http://10.47.255.250 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2025-04-24 13:24:13,016 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd wget http://10.47.255.250 -O /dev/null -T 5 -t 1 failed. Output :Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2025-04-24 13:24:13,016 - DEBUG - [Pod Pod ctest-pod-01650019] Cmd output: Connecting to 10.47.255.250 (10.47.255.250:80) wget: download timed out 2025-04-24 13:24:13,016 - INFO - wget check of of http://10.47.255.250 from ctest-pod-01650019 passed 2025-04-24 13:24:13,078 - DEBUG - [Pod ctest-pod-76290634] Cmd: wget http://10.47.255.251 -O /dev/null -T 5 -t 1 , Output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-04-24 13:24:13,078 - DEBUG - [Pod Pod ctest-pod-76290634] Cmd wget http://10.47.255.251 -O /dev/null -T 5 -t 1 passed 2025-04-24 13:24:13,078 - DEBUG - [Pod Pod ctest-pod-76290634] Cmd output: Connecting to 10.47.255.251 (10.47.255.251:80) saving to '/dev/null' null 100% |********************************| 19 0:00:00 ETA '/dev/null' saved 2025-04-24 13:24:13,078 - INFO - wget check of of http://10.47.255.251 from ctest-pod-76290634 passed 2025-04-24 13:24:13,078 - INFO - Deleting Network Policy : ingress-on-service 2025-04-24 13:24:13,079 - INFO - Deleting service : ctest-nginx-svc-25324750 2025-04-24 13:24:13,288 - DEBUG - No XMPP flaps were noticed during the test