Started by upstream project "pipeline-nightly" build number 10 originally caused by: Started by timer Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on slave-openstack-gz1_1 (openstack) in workspace /home/jenkins/workspace/deploy-tf-k8s-manifests [WS-CLEANUP] Deleting project workspace... [WS-CLEANUP] Deferred wipeout is disabled by the job configuration... [WS-CLEANUP] Done The recommended git tool is: git No credentials specified Wiping out workspace first. Cloning the remote Git repository Cloning repository https://github.com/opensdn-io/tf-jenkins.git > git init /home/jenkins/workspace/deploy-tf-k8s-manifests/src/opensdn-io/tf-jenkins # timeout=10 Fetching upstream changes from https://github.com/opensdn-io/tf-jenkins.git > git --version # timeout=10 > git --version # 'git version 2.25.1' > git fetch --tags --force --progress -- https://github.com/opensdn-io/tf-jenkins.git +refs/heads/*:refs/remotes/origin/* # timeout=10 > git config remote.origin.url https://github.com/opensdn-io/tf-jenkins.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10 Avoid second fetch > git rev-parse refs/remotes/origin/master^{commit} # timeout=10 Checking out Revision e0df994dbfba82155f5afe7a4c84a1a04232f1c2 (refs/remotes/origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f e0df994dbfba82155f5afe7a4c84a1a04232f1c2 # timeout=10 Commit message: "boost169 also requires openblas (which is needed for boost169)" > git rev-list --no-walk 66b4bb2b5d2301f497d58e946b38788181d3ec33 # timeout=10 The recommended git tool is: NONE No credentials specified Wiping out workspace first. Cloning the remote Git repository Cloning repository https://github.com/opensdn-io/tf-devstack.git > git init /home/jenkins/workspace/deploy-tf-k8s-manifests/src/opensdn-io/tf-devstack # timeout=10 Fetching upstream changes from https://github.com/opensdn-io/tf-devstack.git > git --version # timeout=10 > git --version # 'git version 2.25.1' > git fetch --tags --force --progress -- https://github.com/opensdn-io/tf-devstack.git +refs/heads/*:refs/remotes/origin/* # timeout=10 > git config remote.origin.url https://github.com/opensdn-io/tf-devstack.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10 Avoid second fetch > git rev-parse refs/remotes/origin/master^{commit} # timeout=10 Checking out Revision a70e1b89218f73b6c9f6780530074f429c49b896 (refs/remotes/origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f a70e1b89218f73b6c9f6780530074f429c49b896 # timeout=10 Commit message: "tune CONFIG_API_WORKER_COUNT for ansible" > git rev-list --no-walk a70e1b89218f73b6c9f6780530074f429c49b896 # timeout=10 Copied 3 artifacts from "pipeline-nightly" build number 10 [deploy-tf-k8s-manifests] $ /bin/bash -xe /tmp/jenkins1398931205406790541.sh + source /home/jenkins/workspace/deploy-tf-k8s-manifests/global.env ++ export PIPELINE_BUILD_TAG=jenkins-pipeline-nightly-10 ++ PIPELINE_BUILD_TAG=jenkins-pipeline-nightly-10 ++ export SLAVE=openstack ++ SLAVE=openstack ++ export SLAVE_REGION=gz1 ++ SLAVE_REGION=gz1 ++ export LOGS_HOST=nexus.gz1.opensdn.io ++ LOGS_HOST=nexus.gz1.opensdn.io ++ export LOGS_PATH=/var/www/logs/jenkins_logs/nightly/pipeline_10 ++ LOGS_PATH=/var/www/logs/jenkins_logs/nightly/pipeline_10 ++ export LOGS_URL=http://nexus.gz1.opensdn.io:8082/jenkins_logs/nightly/pipeline_10 ++ LOGS_URL=http://nexus.gz1.opensdn.io:8082/jenkins_logs/nightly/pipeline_10 ++ export SITE_MIRROR=http://nexus.gz1.opensdn.io/repository ++ SITE_MIRROR=http://nexus.gz1.opensdn.io/repository ++ export CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5102 ++ CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5102 ++ export DEPLOYER_CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5102 ++ DEPLOYER_CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5102 ++ export CONTRAIL_CONTAINER_TAG=nightly ++ CONTRAIL_CONTAINER_TAG=nightly ++ export CONTRAIL_DEPLOYER_CONTAINER_TAG=nightly ++ CONTRAIL_DEPLOYER_CONTAINER_TAG=nightly ++ export CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5102 ++ CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5102 ++ export DEPLOYER_CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5102 ++ DEPLOYER_CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5102 ++ export CONTRAIL_CONTAINER_TAG_ORIGINAL=nightly ++ CONTRAIL_CONTAINER_TAG_ORIGINAL=nightly ++ export CONTRAIL_DEPLOYER_CONTAINER_TAG_ORIGINAL=nightly ++ CONTRAIL_DEPLOYER_CONTAINER_TAG_ORIGINAL=nightly ++ export GERRIT_PIPELINE=nightly ++ GERRIT_PIPELINE=nightly ++ export GERRIT_BRANCH=master ++ GERRIT_BRANCH=master ++ export REPOS_CHANNEL=latest ++ REPOS_CHANNEL=latest + desc='Pipeline: pipeline-nightly-10 Random: 42953 Stream: k8s-manifests' + desc+='
Job logs: http://nexus.gz1.opensdn.io:8082/jenkins_logs/nightly/pipeline_10/k8s-manifests' + echo 'DESCRIPTION Pipeline: pipeline-nightly-10 Random: 42953 Stream: k8s-manifests
Job logs: http://nexus.gz1.opensdn.io:8082/jenkins_logs/nightly/pipeline_10/k8s-manifests' DESCRIPTION Pipeline: pipeline-nightly-10 Random: 42953 Stream: k8s-manifests
Job logs: http://nexus.gz1.opensdn.io:8082/jenkins_logs/nightly/pipeline_10/k8s-manifests [description-setter] Description set: Pipeline: pipeline-nightly-10 Random: 42953 Stream: k8s-manifests
Job logs: http://nexus.gz1.opensdn.io:8082/jenkins_logs/nightly/pipeline_10/k8s-manifests' [deploy-tf-k8s-manifests] $ /bin/bash -xe /tmp/jenkins9499750942124380562.sh + set -eo pipefail + source /home/jenkins/workspace/deploy-tf-k8s-manifests/global.env ++ export PIPELINE_BUILD_TAG=jenkins-pipeline-nightly-10 ++ PIPELINE_BUILD_TAG=jenkins-pipeline-nightly-10 ++ export SLAVE=openstack ++ SLAVE=openstack ++ export SLAVE_REGION=gz1 ++ SLAVE_REGION=gz1 ++ export LOGS_HOST=nexus.gz1.opensdn.io ++ LOGS_HOST=nexus.gz1.opensdn.io ++ export LOGS_PATH=/var/www/logs/jenkins_logs/nightly/pipeline_10 ++ LOGS_PATH=/var/www/logs/jenkins_logs/nightly/pipeline_10 ++ export LOGS_URL=http://nexus.gz1.opensdn.io:8082/jenkins_logs/nightly/pipeline_10 ++ LOGS_URL=http://nexus.gz1.opensdn.io:8082/jenkins_logs/nightly/pipeline_10 ++ export SITE_MIRROR=http://nexus.gz1.opensdn.io/repository ++ SITE_MIRROR=http://nexus.gz1.opensdn.io/repository ++ export CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5102 ++ CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5102 ++ export DEPLOYER_CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5102 ++ DEPLOYER_CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5102 ++ export CONTRAIL_CONTAINER_TAG=nightly ++ CONTRAIL_CONTAINER_TAG=nightly ++ export CONTRAIL_DEPLOYER_CONTAINER_TAG=nightly ++ CONTRAIL_DEPLOYER_CONTAINER_TAG=nightly ++ export CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5102 ++ CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5102 ++ export DEPLOYER_CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5102 ++ DEPLOYER_CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5102 ++ export CONTRAIL_CONTAINER_TAG_ORIGINAL=nightly ++ CONTRAIL_CONTAINER_TAG_ORIGINAL=nightly ++ export CONTRAIL_DEPLOYER_CONTAINER_TAG_ORIGINAL=nightly ++ CONTRAIL_DEPLOYER_CONTAINER_TAG_ORIGINAL=nightly ++ export GERRIT_PIPELINE=nightly ++ GERRIT_PIPELINE=nightly ++ export GERRIT_BRANCH=master ++ GERRIT_BRANCH=master ++ export REPOS_CHANNEL=latest ++ REPOS_CHANNEL=latest + ./src/opensdn-io/tf-jenkins/infra/gerrit/apply_patchsets.sh ./src opensdn-io/tf-jenkins ./patchsets-info.json + ./src/opensdn-io/tf-jenkins/infra/gerrit/apply_patchsets.sh ./src opensdn-io/tf-devstack ./patchsets-info.json [deploy-tf-k8s-manifests] $ /bin/bash -xe /tmp/jenkins2776330154143154970.sh + set -eo pipefail + source /home/jenkins/workspace/deploy-tf-k8s-manifests/global.env ++ export PIPELINE_BUILD_TAG=jenkins-pipeline-nightly-10 ++ PIPELINE_BUILD_TAG=jenkins-pipeline-nightly-10 ++ export SLAVE=openstack ++ SLAVE=openstack ++ export SLAVE_REGION=gz1 ++ SLAVE_REGION=gz1 ++ export LOGS_HOST=nexus.gz1.opensdn.io ++ LOGS_HOST=nexus.gz1.opensdn.io ++ export LOGS_PATH=/var/www/logs/jenkins_logs/nightly/pipeline_10 ++ LOGS_PATH=/var/www/logs/jenkins_logs/nightly/pipeline_10 ++ export LOGS_URL=http://nexus.gz1.opensdn.io:8082/jenkins_logs/nightly/pipeline_10 ++ LOGS_URL=http://nexus.gz1.opensdn.io:8082/jenkins_logs/nightly/pipeline_10 ++ export SITE_MIRROR=http://nexus.gz1.opensdn.io/repository ++ SITE_MIRROR=http://nexus.gz1.opensdn.io/repository ++ export CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5102 ++ CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5102 ++ export DEPLOYER_CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5102 ++ DEPLOYER_CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5102 ++ export CONTRAIL_CONTAINER_TAG=nightly ++ CONTRAIL_CONTAINER_TAG=nightly ++ export CONTRAIL_DEPLOYER_CONTAINER_TAG=nightly ++ CONTRAIL_DEPLOYER_CONTAINER_TAG=nightly ++ export CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5102 ++ CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5102 ++ export DEPLOYER_CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5102 ++ DEPLOYER_CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5102 ++ export CONTRAIL_CONTAINER_TAG_ORIGINAL=nightly ++ CONTRAIL_CONTAINER_TAG_ORIGINAL=nightly ++ export CONTRAIL_DEPLOYER_CONTAINER_TAG_ORIGINAL=nightly ++ CONTRAIL_DEPLOYER_CONTAINER_TAG_ORIGINAL=nightly ++ export GERRIT_PIPELINE=nightly ++ GERRIT_PIPELINE=nightly ++ export GERRIT_BRANCH=master ++ GERRIT_BRANCH=master ++ export REPOS_CHANNEL=latest ++ REPOS_CHANNEL=latest + source ./src/opensdn-io/tf-jenkins/infra/openstack/definitions ++ export PROVIDER=openstack ++ PROVIDER=openstack ++ export OS_NETWORK=management ++ OS_NETWORK=management ++ export OS_DATA_NETWORK=data ++ OS_DATA_NETWORK=data ++ export OS_SG=default ++ OS_SG=default ++ export RESERVED_INSTANCES_COUNT=3 ++ RESERVED_INSTANCES_COUNT=3 ++ export RESERVED_CORES_COUNT=16 ++ RESERVED_CORES_COUNT=16 ++ export VM_BOOT_RETRIES=120 ++ VM_BOOT_RETRIES=120 ++ export VM_BOOT_DELAY=60 ++ VM_BOOT_DELAY=60 ++ export VOLUME_TYPE=ceph-ssd ++ VOLUME_TYPE=ceph-ssd ++ ENVIRONMENT_OS=rocky9 ++ VM_TYPES=(['xsmall']='STD3-1-4' ['small']='STD3-2-8' ['medium']='STD3-4-16' ['large']='STD3-8-32') ++ declare -A VM_TYPES ++ VOLUME_SIZE=(['xsmall']='50' ['small']='50' ['medium']='80' ['large']='80') ++ declare -A VOLUME_SIZE ++ OS_IMAGES=(['CENTOS7']='prepared-centos7' ['CENTOS8']='prepared-centos8' ['UBUNTU20']='prepared-ubuntu20' ['UBUNTU22']='prepared-ubuntu22' ['ROCKY9']='prepared-rocky92') ++ declare -A OS_IMAGES ++ OS_IMAGE_USERS=(['CENTOS7']='centos' ['CENTOS8']='centos' ['UBUNTU20']='ubuntu' ['UBUNTU22']='ubuntu' ['ROCKY9']='rocky' ['ROCKY92']='rocky') ++ declare -A OS_IMAGE_USERS ++ OS_IMAGES_UP=(['CENTOS7']='centos' ['CENTOS8']='centos' ['UBUNTU20']='ubuntu' ['UBUNTU22']='ubuntu' ['ROCKY9']='rocky' ['ROCKY92']='rocky') ++ declare -A OS_IMAGES_UP ++ OS_IMAGES_DOWN=(['CENTOS7']='' ['CENTOS8']='' ['UBUNTU20']='' ['UBUNTU22']='' ['ROCKY9']='' ['ROCKY92']='') ++ declare -A OS_IMAGES_DOWN ++ SSH_OPTIONS='-T -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o PasswordAuthentication=no' + source /home/jenkins/workspace/deploy-tf-k8s-manifests/deps.deploy-tf-k8s-manifests.42953.env ++ export PROVIDER=openstack ++ PROVIDER=openstack ++ export ENVIRONMENT_OS=rocky9 ++ ENVIRONMENT_OS=rocky9 ++ export IMAGE=aa1bd152-a8d4-49ef-85ea-82a4dab8cf7f ++ IMAGE=aa1bd152-a8d4-49ef-85ea-82a4dab8cf7f ++ export IMAGE_SSH_USER=rocky ++ IMAGE_SSH_USER=rocky ++ export INSTANCE_IDS=116c73ad-7d0f-4dee-9a8e-06cb5f9179fb, ++ INSTANCE_IDS=116c73ad-7d0f-4dee-9a8e-06cb5f9179fb, ++ export instance_ip=10.0.0.23 ++ instance_ip=10.0.0.23 ++ export CONTROLLER_NODES=10.0.0.23, ++ CONTROLLER_NODES=10.0.0.23, ++ export ORCHESTRATOR=kubernetes ++ ORCHESTRATOR=kubernetes ++ export DEPLOYER=k8s_manifests ++ DEPLOYER=k8s_manifests ++ export JOB_LOGS_PATH=k8s-manifests ++ JOB_LOGS_PATH=k8s-manifests + source /home/jenkins/workspace/deploy-tf-k8s-manifests/vars.deploy-tf-k8s-manifests.42953.env ++ export MONITORING_DEPLOY_TARGET=AIO ++ MONITORING_DEPLOY_TARGET=AIO ++ export MONITORING_DEPLOYER=k8s_manifests ++ MONITORING_DEPLOYER=k8s_manifests ++ export MONITORING_ORCHESTRATOR=k8s ++ MONITORING_ORCHESTRATOR=k8s + export FULL_LOGS_PATH=/var/www/logs/jenkins_logs/nightly/pipeline_10/k8s-manifests + FULL_LOGS_PATH=/var/www/logs/jenkins_logs/nightly/pipeline_10/k8s-manifests + ./src/opensdn-io/tf-jenkins/jobs/devstack/k8s_manifests/deploy_tf.sh INFO: Deploy k8s_manifests/ (deploy-tf-k8s-manifests) Warning: Permanently added '10.0.0.23' (ECDSA) to the list of known hosts. Warning: Permanently added '10.0.0.23' (ECDSA) to the list of known hosts. INFO: =================== Mon Sep 9 02:13:09 AM UTC 2024 =================== [there is no tf devenv configuration to load] INFO: Applying stages k8s manifest tf wait INFO: Running stage k8s at Mon Sep 9 02:13:09 AM UTC 2024 Skipping stage k8s because it's finished INFO: Stage k8s was run successfully Mon Sep 9 02:13:09 AM UTC 2024 INFO: Running stage manifest at Mon Sep 9 02:13:09 AM UTC 2024 nexus.gz1.opensdn.io is an alias for nexus.opensdn.io. nexus.opensdn.io has address 212.233.90.199 INFO: ishost = true namespace = registry_url = nexus.gz1.opensdn.io:5102 INFO: Downloading 'tf-container-builder-src:nightly@nightly' (1 layers)... INFO: Download of images into '/tmp/tmp.6HkmwZPtPL' complete. INFO: Use something like the following to load the result into a Docker daemon: INFO: tar -cC '/tmp/tmp.6HkmwZPtPL' . | docker load INFO: Stage manifest was run successfully Mon Sep 9 02:13:09 AM UTC 2024 INFO: Running stage tf at Mon Sep 9 02:13:09 AM UTC 2024 INFO: check time sync on nodes and force sync Mon Sep 9 02:13:09 AM UTC 2024 INFO: controller nodes - 10.0.0.23 INFO: agent nodes - 10.0.0.23 INFO: openstack controller nodes - INFO: sync time on machine rocky@10.0.0.23 Warning: Permanently added '10.0.0.23' (ED25519) to the list of known hosts. Warning: Permanently added '10.0.0.23' (ED25519) to the list of known hosts. INFO: Mon Sep 9 02:13:10 AM UTC 2024: ensure time is synced (chronyd) INFO: Mon Sep 9 02:13:10 AM UTC 2024: time is synced INFO: done in 0s INFO: done in 0s 10.0.0.23 node/cn-jenkins-deploy-platform-k8s-manifests-86-1 labeled INFO: done in 0s 10.0.0.23 node/cn-jenkins-deploy-platform-k8s-manifests-86-1 not labeled INFO: done in 0s 10.0.0.23 node/cn-jenkins-deploy-platform-k8s-manifests-86-1 labeled INFO: done in 0s 10.0.0.23 node/cn-jenkins-deploy-platform-k8s-manifests-86-1 labeled INFO: done in 0s 10.0.0.23 node/cn-jenkins-deploy-platform-k8s-manifests-86-1 labeled INFO: done in 0s 10.0.0.23 node/cn-jenkins-deploy-platform-k8s-manifests-86-1 labeled INFO: done in 0s 10.0.0.23 node/cn-jenkins-deploy-platform-k8s-manifests-86-1 labeled INFO: done in 0s 10.0.0.23 node/cn-jenkins-deploy-platform-k8s-manifests-86-1 labeled INFO: done in 0s 10.0.0.23 node/cn-jenkins-deploy-platform-k8s-manifests-86-1 labeled INFO: done in 0s 10.0.0.23 node/cn-jenkins-deploy-platform-k8s-manifests-86-1 labeled configmap/env created configmap/defaults-env created configmap/configzookeeperenv created configmap/contrail-analyticsdb-config created configmap/contrail-configdb-config created configmap/rabbitmq-config created configmap/kube-manager-config created Warning: spec.template.spec.imagePullSecrets[0].name: invalid empty name "" daemonset.apps/config-zookeeper created daemonset.apps/contrail-analyticsdb created daemonset.apps/contrail-configdb created daemonset.apps/contrail-analytics created daemonset.apps/contrail-analytics-snmp created daemonset.apps/contrail-analytics-alarm created daemonset.apps/contrail-controller-control created daemonset.apps/contrail-controller-config created daemonset.apps/contrail-controller-webui created daemonset.apps/redis created daemonset.apps/rabbitmq created daemonset.apps/contrail-kube-manager created daemonset.apps/contrail-agent created clusterrole.rbac.authorization.k8s.io/contrail-kube-manager created serviceaccount/contrail-kube-manager created clusterrolebinding.rbac.authorization.k8s.io/contrail-kube-manager created secret/contrail-kube-manager-token created TF Web UI will be available at https://10.0.0.23:8143 Use admin/contrail123 to log in INFO: Stage tf was run successfully Mon Sep 9 02:13:13 AM UTC 2024 INFO: Running stage wait at Mon Sep 9 02:13:13 AM UTC 2024 ....................................................................................................................................................... ERROR: wait failed in 1510s ERROR: wait failed Mon Sep 9 02:38:32 AM UTC 2024 + is_active + check_pods_active + local tool=kubectl + declare -a pods + readarray -t pods ++ kubectl get pods --all-namespaces --no-headers + [[ 19 == \0 ]] + local pod + for pod in "${pods[@]}" ++ echo kube-system config-zookeeper-znqqh 1/1 Running 0 24m ++ awk '{print $4}' + local status=Running + [[ Running == \C\o\m\p\l\e\t\e\d ]] + [[ Running != \R\u\n\n\i\n\g ]] ++ echo kube-system config-zookeeper-znqqh 1/1 Running 0 24m ++ awk '{print $3}' ++ cut -f 1 -d / + local containers_running=1 ++ echo kube-system config-zookeeper-znqqh 1/1 Running 0 24m ++ cut -f 2 -d / ++ awk '{print $3}' + local containers_total=1 + '[' 1 '!=' 1 ']' + for pod in "${pods[@]}" ++ echo kube-system contrail-agent-jjg8b 2/3 CrashLoopBackOff 7 '(2m2s' 'ago)' 25m ++ awk '{print $4}' + local status=CrashLoopBackOff + [[ CrashLoopBackOff == \C\o\m\p\l\e\t\e\d ]] + [[ CrashLoopBackOff != \R\u\n\n\i\n\g ]] + return 1 + trap_exit + [[ k8s_manifests == \h\e\l\m ]] ++ jobs -p + local childs= + echo 'DEBUG: kill running child jobs: ' DEBUG: kill running child jobs: + local p INFO: Deploy finished + res=1 + exit 1 Build step 'Execute shell' marked build as failure Archiving artifacts Finished: FAILURE