Triggered by Gerrit: https://gerrit.opensdn.io/c/opensdn-io/tf-controller/+/3034 Obtained pipelines/main.groovy from git https://github.com/opensdn-io/tf-jenkins.git [Pipeline] Start of Pipeline [Pipeline] timestamps [Pipeline] { [Pipeline] timeout [2026-08-26T07:01:41.814Z] Timeout set to expire in 8 hr 0 min [Pipeline] { [Pipeline] node [2026-08-26T07:01:41.871Z] Running on slave-openstack-gz1_1 in /home/jenkins/workspace/pipeline-check@2 [Pipeline] { [Pipeline] stage [Pipeline] { (init) [Pipeline] cleanWs [2026-08-26T07:01:41.975Z] [WS-CLEANUP] Deleting project workspace... [2026-08-26T07:01:41.975Z] [WS-CLEANUP] Deferred wipeout is disabled by the job configuration... [2026-08-26T07:01:42.141Z] [WS-CLEANUP] done [Pipeline] checkout [2026-08-26T07:01:42.169Z] The recommended git tool is: git [2026-08-26T07:01:42.176Z] No credentials specified [2026-08-26T07:01:42.178Z] Cloning the remote Git repository [2026-08-26T07:01:42.181Z] Cloning repository https://github.com/opensdn-io/tf-jenkins.git [2026-08-26T07:01:42.182Z] > git init /home/jenkins/workspace/pipeline-check@2/src/opensdn-io/tf-jenkins # timeout=10 [2026-08-26T07:01:42.189Z] Fetching upstream changes from https://github.com/opensdn-io/tf-jenkins.git [2026-08-26T07:01:42.189Z] > git --version # timeout=10 [2026-08-26T07:01:42.191Z] > git --version # 'git version 2.25.1' [2026-08-26T07:01:42.191Z] > git fetch --tags --force --progress -- https://github.com/opensdn-io/tf-jenkins.git +refs/heads/*:refs/remotes/origin/* # timeout=10 [2026-08-26T07:01:43.980Z] Avoid second fetch [2026-08-26T07:01:44.041Z] Checking out Revision 9c86492d4a22712d2c3b4ab27040cd93556012b4 (refs/remotes/origin/master) [2026-08-26T07:01:43.575Z] > git config remote.origin.url https://github.com/opensdn-io/tf-jenkins.git # timeout=10 [2026-08-26T07:01:43.582Z] > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10 [2026-08-26T07:01:44.014Z] > git rev-parse refs/remotes/origin/master^{commit} # timeout=10 [2026-08-26T07:01:44.080Z] > git config core.sparsecheckout # timeout=10 [2026-08-26T07:01:44.084Z] > git checkout -f 9c86492d4a22712d2c3b4ab27040cd93556012b4 # timeout=10 [2026-08-26T07:01:44.255Z] Commit message: "Merge "install cloud-init from site mirror"" [2026-08-26T07:01:44.257Z] > git rev-list --no-walk 9c86492d4a22712d2c3b4ab27040cd93556012b4 # timeout=10 [Pipeline] load [Pipeline] { (/home/jenkins/workspace/pipeline-check@2/src/opensdn-io/tf-jenkins/pipelines/utils/gerrit.groovy) [Pipeline] } [Pipeline] // load [Pipeline] echo [2026-08-26T07:01:46.269Z] INFO: resolved gerrit URL is https://gerrit.opensdn.io/ [Pipeline] sh [2026-08-26T07:01:47.034Z] + /home/jenkins/workspace/pipeline-check@2/src/opensdn-io/tf-jenkins/infra/gerrit/resolve_patchsets.py --gerrit https://gerrit.opensdn.io/ --review I48b6aa7a1c5904c6b68afa3ae8ec4e7d367c3615 --branch master --changed_files --output /home/jenkins/workspace/pipeline-check@2/patchsets-info.json [2026-08-26T07:01:47.035Z] DEBUG: Change: {'id': 'opensdn-io%2Ftf-controller~master~I48b6aa7a1c5904c6b68afa3ae8ec4e7d367c3615', 'project': 'opensdn-io/tf-controller', 'branch': 'master', 'topic': 'route-target-lifecycle', 'attention_set': {'1000021': {'account': {'_account_id': 1000021}, 'last_update': '2026-08-26 04:01:56.000000000', 'reason': 'Someone else replied on the change'}}, 'removed_from_attention_set': {'1000003': {'account': {'_account_id': 1000003}, 'last_update': '2026-08-26 07:01:32.000000000', 'reason': ' replied on the change', 'reason_account': {'_account_id': 1000003}}, '1000001': {'account': {'_account_id': 1000001}, 'last_update': '2026-08-20 08:23:10.000000000', 'reason': 'removed on reply'}}, 'hashtags': [], 'change_id': 'I48b6aa7a1c5904c6b68afa3ae8ec4e7d367c3615', 'subject': 'ST: always release the logical-router route-target ID on delete', 'status': 'NEW', 'created': '2026-08-06 06:10:38.000000000', 'updated': '2026-08-26 07:01:32.000000000', 'submit_type': 'MERGE_IF_NECESSARY', 'insertions': 38, 'deletions': 6, 'total_comment_count': 9, 'unresolved_comment_count': 1, 'has_review_started': True, 'meta_rev_id': '406d6807ea6d64b5effce926769ae47695b602f7', '_number': 3034, 'owner': {'_account_id': 1000021}, 'labels': {'Code-Review': {'all': [{'value': 0, 'permitted_voting_range': {'min': -2, 'max': 2}, '_account_id': 1000003}, {'value': 0, 'permitted_voting_range': {'min': -2, 'max': 2}, '_account_id': 1000001}], 'values': {'-2': 'This shall not be submitted', '-1': 'I would prefer this is not submitted as is', ' 0': 'No score', '+1': 'Looks good to me, but someone else must approve', '+2': 'Looks good to me, approved'}, 'description': '', 'default_value': 0}, 'Verified': {'all': [{'value': 0, 'permitted_voting_range': {'min': -2, 'max': 2}, '_account_id': 1000003}, {'value': -1, 'date': '2026-08-26 06:33:01.000000000', 'permitted_voting_range': {'min': -2, 'max': 2}, '_account_id': 1000001}], 'values': {'-2': 'Fails', '-1': 'Fails', ' 0': 'No score', '+1': 'Verified', '+2': 'Gating'}, 'description': '', 'default_value': 0}}, 'removable_reviewers': [], 'reviewers': {'REVIEWER': [{'_account_id': 1000001}, {'_account_id': 1000003}], 'CC': [{'_account_id': 1000002}]}, 'pending_reviewers': {}, 'current_revision': 'bf640c0e985af7951b386684af01e66c5de7e548', 'revisions': {'bf640c0e985af7951b386684af01e66c5de7e548': {'kind': 'REWORK', '_number': 3, 'created': '2026-08-20 10:54:43.000000000', 'uploader': {'_account_id': 1000021}, 'ref': 'refs/changes/34/3034/3', 'fetch': {'anonymous http': {'url': 'https://gerrit.opensdn.io/opensdn-io/tf-controller', 'ref': 'refs/changes/34/3034/3'}}, 'commit': {'parents': [{'commit': '9112407ed0a26aaec3ee49b7513d267bf7e93597', 'subject': 'Switching to oneTBB: old includes removal'}], 'author': {'name': 'Charnsilp Chinprasert(Nipa)', 'email': 'justice.form@gmail.com', 'date': '2026-08-06 06:05:31.000000000', 'tz': 420}, 'committer': {'name': 'Charnsilp Chinprasert(Nipa)', 'email': 'justice.form@gmail.com', 'date': '2026-08-20 10:54:21.000000000', 'tz': 420}, 'subject': 'ST: always release the logical-router route-target ID on delete', 'message': "ST: always release the logical-router route-target ID on delete\n\nLogicalRouterST.delete_obj() runs five steps before it releases the\nroute-target ID, and every one of them is an API or datastore call that can\nraise:\n\n update_multiple_refs('virtual_machine_interface', {})\n update_multiple_refs('route_table', {})\n update_multiple_refs('bgpvpn', {})\n update_virtual_networks()\n delete_route_targets([self.route_target])\n asn = ...get_autonomous_system()\n self._object_db.free_route_target(self.name, asn)\n\nIf any of them raises, free_route_target() never runs and the ID stays\nallocated. Nothing recovers it: the logical router is gone, so delete_obj()\nis never called again, and a Zookeeper index with no row in route_target_table\nhas no reader - alloc_route_target() starts from the row, so an index without\none is never examined.\n\nA production config DB shows six route_target_table bindings naming logical\nrouters that no longer exist. Three of them still have their route-target\nobject and three do not, which places the failure between delete_route_targets()\nand free_route_target() in the first case and at or before\ndelete_route_targets() in the second.\n\nRead the ASN before the steps that can raise, and release the ID in a finally\nblock, so it is released on every path. The ASN lookup is deliberately outside\nthe try: if it stayed inside and raised - which is one of the two things that\nexplains the three-case split above - the finally clause would have no asn to\nuse.\n\nEach teardown step is attempted independently rather than under one try, so a\nfailure in an early step does not skip the later ones and the log names the\nstep that actually failed. Raised in review on patchset 1.\n\nThe per-step handling is about diagnosis rather than damage, and it is worth\nbeing precise about why. None of the five steps leaves unrecoverable state on\nits own: RouteTargetST.reinit() already deletes route-target objects that have\nno routing-instance or logical-router back-refs, and walks _RT_CF freeing any\nrow whose object is gone (resources/route_target.py:20-38); the ref sets are\nrebuilt from the DB at the same reinit. The ID release is the one step with no\nsuch collector - alloc_route_target() starts from the _RT_CF row, so a\nZookeeper index whose row is gone is never examined again - which is why it,\nand only it, sits in finally.\n\nOne consequence to note: update_virtual_networks() reads the ref sets that the\nearlier steps clear, so if an earlier step fails it works from stale membership\nand may leave a route target on a VN. That is now logged instead of being\nhidden behind an abandoned try, and reinit recomputes it. It does not justify\nskipping the steps after it.\n\nThe catch is deliberately broad. The goal is that the release below always\nruns, and narrowing it to the vnc_api exception family would let anything\nunexpected skip the remaining steps again. If the project would rather see it\nnarrowed, say so and I will.\n\nThis does not make delete_obj() idempotent, and it does not help if\nfree_route_target() itself raises partway - that is handled separately.\n\nTested on this patchset: tox -e pep8 passes clean. tox -e py39 runs 83 tests\nwith 16 skipped and one failure, test_security_group.test_stale_acl, which\nfails the same way on an unpatched tree - it hashes a dict in\n_access_control_list_update() and is unrelated to this change.\n\nChange-Id: I48b6aa7a1c5904c6b68afa3ae8ec4e7d367c3615\nSigned-off-by: Charnsilp Chinprasert(Nipa) \n"}}}, 'requirements': [], 'submit_records': [{'rule_name': 'gerrit~DefaultSubmitRule', 'status': 'NOT_READY', 'labels': [{'label': 'Code-Review', 'status': 'NEED'}, {'label': 'Verified', 'status': 'NEED'}]}]} [2026-08-26T07:01:47.035Z] DEBUG: Change: {'id': 'opensdn-io%2Ftf-controller~master~I0c7161ee31e11ff4cc6c1962657b13e8e8b3181d', 'project': 'opensdn-io/tf-controller', 'branch': 'master', 'attention_set': {}, 'removed_from_attention_set': {'1000003': {'account': {'_account_id': 1000003}, 'last_update': '2026-07-13 10:34:15.000000000', 'reason': 'Only change owner, uploader, reviewers, and cc can be in the attention set'}, '1000002': {'account': {'_account_id': 1000002}, 'last_update': '2026-07-18 08:19:38.000000000', 'reason': 'Change was submitted'}, '1000001': {'account': {'_account_id': 1000001}, 'last_update': '2026-07-16 20:29:55.000000000', 'reason': 'removed on reply'}, '1000012': {'account': {'_account_id': 1000012}, 'last_update': '2026-07-18 08:19:38.000000000', 'reason': 'Change was submitted'}}, 'hashtags': [], 'change_id': 'I0c7161ee31e11ff4cc6c1962657b13e8e8b3181d', 'subject': 'Switching to oneTBB: old includes removal', 'status': 'MERGED', 'created': '2026-07-10 13:51:24.000000000', 'updated': '2026-07-18 08:20:05.000000000', 'submitted': '2026-07-18 08:19:38.000000000', 'submitter': {'_account_id': 1000003}, 'insertions': 23, 'deletions': 24, 'total_comment_count': 11, 'unresolved_comment_count': 0, 'has_review_started': True, 'submission_id': '3008', 'meta_rev_id': '831fab326a1db08f76cc019e3e969dab5f0c75ea', '_number': 3008, 'owner': {'_account_id': 1000012}, 'labels': {'Code-Review': {'all': [{'value': 2, 'date': '2026-07-18 08:19:17.000000000', 'permitted_voting_range': {'min': 2, 'max': 2}, '_account_id': 1000003}, {'value': 0, '_account_id': 1000001}], 'values': {'-2': 'This shall not be submitted', '-1': 'I would prefer this is not submitted as is', ' 0': 'No score', '+1': 'Looks good to me, but someone else must approve', '+2': 'Looks good to me, approved'}, 'description': '', 'default_value': 0}, 'Verified': {'all': [{'value': 0, '_account_id': 1000003}, {'value': 2, 'date': '2026-07-18 06:07:58.000000000', 'permitted_voting_range': {'min': 2, 'max': 2}, '_account_id': 1000001}], 'values': {'-2': 'Fails', '-1': 'Fails', ' 0': 'No score', '+1': 'Verified', '+2': 'Gating'}, 'description': '', 'default_value': 0}}, 'removable_reviewers': [], 'reviewers': {'REVIEWER': [{'_account_id': 1000001}, {'_account_id': 1000003}], 'CC': [{'_account_id': 1000002}]}, 'pending_reviewers': {}, 'current_revision': '9112407ed0a26aaec3ee49b7513d267bf7e93597', 'revisions': {'9112407ed0a26aaec3ee49b7513d267bf7e93597': {'kind': 'REWORK', '_number': 6, 'created': '2026-07-16 20:29:40.000000000', 'uploader': {'_account_id': 1000012}, 'ref': 'refs/changes/08/3008/6', 'fetch': {'anonymous http': {'url': 'https://gerrit.opensdn.io/opensdn-io/tf-controller', 'ref': 'refs/changes/08/3008/6'}}, 'commit': {'parents': [{'commit': 'd0821a530939acc57c8e9ff48f2c5cae733f3bc5', 'subject': 'Merge "VxLAN bug: VRF NH flapping during a VM live migration"'}], 'author': {'name': 'Matvey Kraposhin', 'email': 'mkraposhin@gmail.com', 'date': '2026-07-10 13:51:19.000000000', 'tz': 180}, 'committer': {'name': 'Matvey Kraposhin', 'email': 'mkraposhin@gmail.com', 'date': '2026-07-16 20:29:34.000000000', 'tz': 180}, 'subject': 'Switching to oneTBB: old includes removal', 'message': 'Switching to oneTBB: old includes removal\n\nDepends-On: I44aa62b1ba281c7668739e0d52f6c53300ccb097\n\nChange-Id: I0c7161ee31e11ff4cc6c1962657b13e8e8b3181d\n'}}}, 'requirements': [], 'submit_records': [{'rule_name': 'gerrit~DefaultSubmitRule', 'status': 'CLOSED', 'labels': [{'label': 'Code-Review', 'status': 'OK', 'applied_by': {'_account_id': 1000003}}, {'label': 'Verified', 'status': 'OK', 'applied_by': {'_account_id': 1000001}}]}]} [2026-08-26T07:01:47.035Z] DEBUG: Change: I48b6aa7a1c5904c6b68afa3ae8ec4e7d367c3615: depends_on: [] [Pipeline] archiveArtifacts [2026-08-26T07:01:47.172Z] Archiving artifacts [Pipeline] sh [Pipeline] load [Pipeline] { (/home/jenkins/workspace/pipeline-check@2/src/opensdn-io/tf-jenkins/pipelines/utils/gerrit.groovy) [Pipeline] } [Pipeline] // load [Pipeline] load [Pipeline] { (/home/jenkins/workspace/pipeline-check@2/src/opensdn-io/tf-jenkins/pipelines/constants.groovy) [Pipeline] } [Pipeline] // load [Pipeline] load [Pipeline] { (/home/jenkins/workspace/pipeline-check@2/src/opensdn-io/tf-jenkins/pipelines/utils/config.groovy) [Pipeline] } [Pipeline] // load [Pipeline] load [Pipeline] { (/home/jenkins/workspace/pipeline-check@2/src/opensdn-io/tf-jenkins/pipelines/utils/jobs.groovy) [Pipeline] } [Pipeline] // load [Pipeline] load [Pipeline] { (/home/jenkins/workspace/pipeline-check@2/src/opensdn-io/tf-jenkins/pipelines/utils/gate.groovy) [Pipeline] } [Pipeline] // load [Pipeline] } [Pipeline] // stage [Pipeline] withCredentials [2026-08-26T07:01:53.592Z] Masking supported pattern matches of $GERRIT_API_PASSWORD [Pipeline] { [Pipeline] echo [2026-08-26T07:01:53.677Z] INFO: resolved gerrit URL is https://gerrit.opensdn.io/ [Pipeline] sh [2026-08-26T07:01:54.170Z] + /home/jenkins/workspace/pipeline-check@2/src/opensdn-io/tf-jenkins/infra/gerrit/is_merged.py --gerrit https://gerrit.opensdn.io/ --user tf-gerrit --password **** --review I48b6aa7a1c5904c6b68afa3ae8ec4e7d367c3615 --branch master [2026-08-26T07:01:54.285Z] DEBUG: Change: {'id': 'opensdn-io%2Ftf-controller~master~I48b6aa7a1c5904c6b68afa3ae8ec4e7d367c3615', 'project': 'opensdn-io/tf-controller', 'branch': 'master', 'topic': 'route-target-lifecycle', 'attention_set': {'1000021': {'account': {'_account_id': 1000021}, 'last_update': '2026-08-26 04:01:56.000000000', 'reason': 'Someone else replied on the change'}}, 'removed_from_attention_set': {'1000003': {'account': {'_account_id': 1000003}, 'last_update': '2026-08-26 07:01:32.000000000', 'reason': ' replied on the change', 'reason_account': {'_account_id': 1000003}}, '1000001': {'account': {'_account_id': 1000001}, 'last_update': '2026-08-20 08:23:10.000000000', 'reason': 'removed on reply'}}, 'hashtags': [], 'change_id': 'I48b6aa7a1c5904c6b68afa3ae8ec4e7d367c3615', 'subject': 'ST: always release the logical-router route-target ID on delete', 'status': 'NEW', 'created': '2026-08-06 06:10:38.000000000', 'updated': '2026-08-26 07:01:32.000000000', 'submit_type': 'MERGE_IF_NECESSARY', 'insertions': 38, 'deletions': 6, 'total_comment_count': 9, 'unresolved_comment_count': 1, 'has_review_started': True, 'meta_rev_id': '406d6807ea6d64b5effce926769ae47695b602f7', '_number': 3034, 'owner': {'_account_id': 1000021}, 'labels': {'Code-Review': {'all': [{'value': 0, 'permitted_voting_range': {'min': -2, 'max': 2}, '_account_id': 1000003}, {'value': 0, 'permitted_voting_range': {'min': -2, 'max': 2}, '_account_id': 1000001}], 'values': {'-2': 'This shall not be submitted', '-1': 'I would prefer this is not submitted as is', ' 0': 'No score', '+1': 'Looks good to me, but someone else must approve', '+2': 'Looks good to me, approved'}, 'description': '', 'default_value': 0}, 'Verified': {'all': [{'value': 0, 'permitted_voting_range': {'min': -2, 'max': 2}, '_account_id': 1000003}, {'value': -1, 'date': '2026-08-26 06:33:01.000000000', 'permitted_voting_range': {'min': -2, 'max': 2}, '_account_id': 1000001}], 'values': {'-2': 'Fails', '-1': 'Fails', ' 0': 'No score', '+1': 'Verified', '+2': 'Gating'}, 'description': '', 'default_value': 0}}, 'permitted_labels': {'Code-Review': ['-2', '-1', ' 0', '+1', '+2'], 'Verified': ['-2', '-1', ' 0', '+1', '+2']}, 'removable_labels': {'Verified': {'-1': [{'_account_id': 1000001}]}}, 'removable_reviewers': [{'_account_id': 1000003}, {'_account_id': 1000002}, {'_account_id': 1000001}], 'reviewers': {'REVIEWER': [{'_account_id': 1000001}, {'_account_id': 1000003}], 'CC': [{'_account_id': 1000002}]}, 'pending_reviewers': {}, 'current_revision': 'bf640c0e985af7951b386684af01e66c5de7e548', 'revisions': {'bf640c0e985af7951b386684af01e66c5de7e548': {'kind': 'REWORK', '_number': 3, 'created': '2026-08-20 10:54:43.000000000', 'uploader': {'_account_id': 1000021}, 'ref': 'refs/changes/34/3034/3', 'fetch': {'ssh': {'url': 'ssh://tf-gerrit@gerrit.opensdn.io:29418/opensdn-io/tf-controller', 'ref': 'refs/changes/34/3034/3'}, 'anonymous http': {'url': 'https://gerrit.opensdn.io/opensdn-io/tf-controller', 'ref': 'refs/changes/34/3034/3'}, 'http': {'url': 'https://tf-gerrit@gerrit.opensdn.io/a/opensdn-io/tf-controller', 'ref': 'refs/changes/34/3034/3'}}, 'commit': {'parents': [{'commit': '9112407ed0a26aaec3ee49b7513d267bf7e93597', 'subject': 'Switching to oneTBB: old includes removal'}], 'author': {'name': 'Charnsilp Chinprasert(Nipa)', 'email': 'justice.form@gmail.com', 'date': '2026-08-06 06:05:31.000000000', 'tz': 420}, 'committer': {'name': 'Charnsilp Chinprasert(Nipa)', 'email': 'justice.form@gmail.com', 'date': '2026-08-20 10:54:21.000000000', 'tz': 420}, 'subject': 'ST: always release the logical-router route-target ID on delete', 'message': "ST: always release the logical-router route-target ID on delete\n\nLogicalRouterST.delete_obj() runs five steps before it releases the\nroute-target ID, and every one of them is an API or datastore call that can\nraise:\n\n update_multiple_refs('virtual_machine_interface', {})\n update_multiple_refs('route_table', {})\n update_multiple_refs('bgpvpn', {})\n update_virtual_networks()\n delete_route_targets([self.route_target])\n asn = ...get_autonomous_system()\n self._object_db.free_route_target(self.name, asn)\n\nIf any of them raises, free_route_target() never runs and the ID stays\nallocated. Nothing recovers it: the logical router is gone, so delete_obj()\nis never called again, and a Zookeeper index with no row in route_target_table\nhas no reader - alloc_route_target() starts from the row, so an index without\none is never examined.\n\nA production config DB shows six route_target_table bindings naming logical\nrouters that no longer exist. Three of them still have their route-target\nobject and three do not, which places the failure between delete_route_targets()\nand free_route_target() in the first case and at or before\ndelete_route_targets() in the second.\n\nRead the ASN before the steps that can raise, and release the ID in a finally\nblock, so it is released on every path. The ASN lookup is deliberately outside\nthe try: if it stayed inside and raised - which is one of the two things that\nexplains the three-case split above - the finally clause would have no asn to\nuse.\n\nEach teardown step is attempted independently rather than under one try, so a\nfailure in an early step does not skip the later ones and the log names the\nstep that actually failed. Raised in review on patchset 1.\n\nThe per-step handling is about diagnosis rather than damage, and it is worth\nbeing precise about why. None of the five steps leaves unrecoverable state on\nits own: RouteTargetST.reinit() already deletes route-target objects that have\nno routing-instance or logical-router back-refs, and walks _RT_CF freeing any\nrow whose object is gone (resources/route_target.py:20-38); the ref sets are\nrebuilt from the DB at the same reinit. The ID release is the one step with no\nsuch collector - alloc_route_target() starts from the _RT_CF row, so a\nZookeeper index whose row is gone is never examined again - which is why it,\nand only it, sits in finally.\n\nOne consequence to note: update_virtual_networks() reads the ref sets that the\nearlier steps clear, so if an earlier step fails it works from stale membership\nand may leave a route target on a VN. That is now logged instead of being\nhidden behind an abandoned try, and reinit recomputes it. It does not justify\nskipping the steps after it.\n\nThe catch is deliberately broad. The goal is that the release below always\nruns, and narrowing it to the vnc_api exception family would let anything\nunexpected skip the remaining steps again. If the project would rather see it\nnarrowed, say so and I will.\n\nThis does not make delete_obj() idempotent, and it does not help if\nfree_route_target() itself raises partway - that is handled separately.\n\nTested on this patchset: tox -e pep8 passes clean. tox -e py39 runs 83 tests\nwith 16 skipped and one failure, test_security_group.test_stale_acl, which\nfails the same way on an unpatched tree - it hashes a dict in\n_access_control_list_update() and is unrelated to this change.\n\nChange-Id: I48b6aa7a1c5904c6b68afa3ae8ec4e7d367c3615\nSigned-off-by: Charnsilp Chinprasert(Nipa) \n"}}}, 'requirements': [], 'submit_records': [{'rule_name': 'gerrit~DefaultSubmitRule', 'status': 'NOT_READY', 'labels': [{'label': 'Code-Review', 'status': 'NEED'}, {'label': 'Verified', 'status': 'NEED'}]}]} [Pipeline] echo [2026-08-26T07:01:54.315Z] is_merged.py returns non-zero code. It means that review is not merged for now. [Pipeline] echo [2026-08-26T07:01:54.342Z] script returned exit code 1 [Pipeline] } [Pipeline] // withCredentials [Pipeline] stage [Pipeline] { (Pre-build) [Pipeline] echo [2026-08-26T07:01:54.585Z] Logs URL: http://nexus.gz1.opensdn.io:8082/jenkins_logs/gerrit/34/3034/3/check_4729 [Pipeline] echo [2026-08-26T07:01:54.638Z] terminate_runs_by_review_number: start [Pipeline] echo [2026-08-26T07:02:11.755Z] terminate_runs_by_review_number: terminated builds = [] [Pipeline] sh [Pipeline] echo [2026-08-26T07:02:12.704Z] Pipeline to run: check [Pipeline] sh [Pipeline] sh [Pipeline] archiveArtifacts [2026-08-26T07:02:13.856Z] Archiving artifacts [Pipeline] readYaml [Pipeline] readYaml [Pipeline] readYaml [Pipeline] readYaml [Pipeline] readYaml [Pipeline] readYaml [Pipeline] readYaml [Pipeline] readYaml [Pipeline] readYaml [Pipeline] readYaml [Pipeline] readYaml [Pipeline] echo [2026-08-26T07:02:19.058Z] Streams from config: [test-unit:[vars:[LINUX_DISTR:rockylinux, LINUX_DISTR_VER:9], jobs:[test-unit-agent, test-unit-opserver, test-unit-bgp, test-unit-group-one, test-unit-group-two, test-unit-ungrouped, test-code-coverage]], prepare-ut-sandbox:[vars:[LINUX_DISTR:rockylinux, LINUX_DISTR_VER:9], jobs:[prepare-sandbox-ut-sandbox, fetch-sources-ut-sandbox, configure-ut-sandbox, cleanup-worker-ut-sandbox]], ansible-os:[vars:[OPENSTACK_VERSION:2024.2, MONITORING_DEPLOY_TARGET:AIO, MONITORING_DEPLOYER:ansible, MONITORING_ORCHESTRATOR:os, USE_DATAPLANE_NETWORK:true], jobs:[deploy-platform-ansible-os, deploy-tf-ansible-os, test-sanity-ansible-os, collect-logs-and-cleanup-ansible-os]], build-rocky9:[vars:[LINUX_DISTR:rockylinux, LINUX_DISTR_VER:9], jobs:[prepare-sandbox-rocky9, fetch-sources-rocky9, configure-rocky9, package-deployers-rocky9, compile-rocky9, package-rocky9, freeze-rocky9, cleanup-rocky9]]] [Pipeline] echo [2026-08-26T07:02:19.076Z] Jobs from config: [test-unit-agent:[stream:test-unit, depends-on:[[configure-ut-sandbox:[inherit-keys:[DEVENV_TAG, UNITTEST_TARGETS]]]]], test-unit-opserver:[stream:test-unit, depends-on:[[configure-ut-sandbox:[inherit-keys:[DEVENV_TAG, UNITTEST_TARGETS]]]]], test-unit-bgp:[stream:test-unit, depends-on:[[configure-ut-sandbox:[inherit-keys:[DEVENV_TAG, UNITTEST_TARGETS]]]]], test-unit-group-one:[stream:test-unit, depends-on:[[configure-ut-sandbox:[inherit-keys:[DEVENV_TAG, UNITTEST_TARGETS]]]]], test-unit-group-two:[stream:test-unit, depends-on:[[configure-ut-sandbox:[inherit-keys:[DEVENV_TAG, UNITTEST_TARGETS]]]]], test-unit-ungrouped:[stream:test-unit, depends-on:[[configure-ut-sandbox:[inherit-keys:[DEVENV_TAG, UNITTEST_TARGETS]]]]], test-code-coverage:[stream:test-unit, depends-on:[[configure-ut-sandbox:[inherit-keys:[DEVENV_TAG]]], test-unit-agent, test-unit-bgp, test-unit-opserver, test-unit-group-one, test-unit-group-two, test-unit-ungrouped]], prepare-sandbox-ut-sandbox:[job-name:build-prepare-sandbox, stream:prepare-ut-sandbox, vars:[PUBLISH_TYPE:stable, VM_TYPE:small]], fetch-sources-ut-sandbox:[job-name:build-fetch-sources, stream:prepare-ut-sandbox, depends-on:[prepare-sandbox-ut-sandbox]], configure-ut-sandbox:[job-name:build-configure, stream:prepare-ut-sandbox, depends-on:[fetch-sources-ut-sandbox], vars:[PUBLISH_TYPE:build]], cleanup-worker-ut-sandbox:[job-name:build-cleanup, stream:prepare-ut-sandbox, type:post-hook, depends-on:[configure-ut-sandbox]], deploy-platform-ansible-os:[stream:ansible-os, depends-on:[package-rocky9]], deploy-tf-ansible-os:[stream:ansible-os, depends-on:[package-rocky9, deploy-platform-ansible-os]], test-sanity-ansible-os:[job-name:test-sanity, stream:ansible-os, depends-on:[deploy-tf-ansible-os]], collect-logs-and-cleanup-ansible-os:[job-name:collect-logs-and-cleanup, stream:ansible-os, type:post-hook, depends-on:[deploy-platform-ansible-os, deploy-tf-ansible-os, test-sanity-ansible-os]], prepare-sandbox-rocky9:[job-name:build-prepare-sandbox, stream:build-rocky9, vars:[DEVENV_TAG:frozen]], fetch-sources-rocky9:[job-name:build-fetch-sources, stream:build-rocky9, depends-on:[prepare-sandbox-rocky9]], configure-rocky9:[job-name:build-configure, stream:build-rocky9, depends-on:[fetch-sources-rocky9]], package-deployers-rocky9:[job-name:build-package-deployers, stream:build-rocky9, depends-on:[configure-rocky9]], compile-rocky9:[job-name:build-compile, stream:build-rocky9, depends-on:[package-deployers-rocky9]], package-rocky9:[job-name:build-package, stream:build-rocky9, depends-on:[compile-rocky9]], freeze-rocky9:[job-name:build-freeze, stream:build-rocky9, depends-on:[package-rocky9], vars:[PUBLISH_TYPE:frozen]], cleanup-rocky9:[job-name:build-cleanup, stream:build-rocky9, type:post-hook, depends-on:[freeze-rocky9]]] [Pipeline] echo [2026-08-26T07:02:19.098Z] Post Jobs from config: [cleanup-workers:[:]] [Pipeline] echo [2026-08-26T07:02:19.183Z] Notify gerrit verified=0, submit=false, msg= [2026-08-26T07:02:19.183Z] TF CI Build Started (check) https://jenkins.opensdn.io/job/pipeline-check/4729/ [Pipeline] withCredentials [2026-08-26T07:02:19.214Z] Masking supported pattern matches of $GERRIT_API_PASSWORD [Pipeline] { [Pipeline] echo [2026-08-26T07:02:19.464Z] INFO: resolved gerrit URL is https://gerrit.opensdn.io/ [Pipeline] sh [2026-08-26T07:02:20.283Z] + /home/jenkins/workspace/pipeline-check@2/src/opensdn-io/tf-jenkins/infra/gerrit/notify.py --gerrit https://gerrit.opensdn.io/ --user tf-gerrit --password **** --review I48b6aa7a1c5904c6b68afa3ae8ec4e7d367c3615 --patchset 3 --branch master --labels Verified=0 --message 'TF CI Build Started (check) https://jenkins.opensdn.io/job/pipeline-check/4729/' [2026-08-26T07:02:20.285Z] DEBUG: Change: {'id': 'opensdn-io%2Ftf-controller~master~I48b6aa7a1c5904c6b68afa3ae8ec4e7d367c3615', 'project': 'opensdn-io/tf-controller', 'branch': 'master', 'topic': 'route-target-lifecycle', 'attention_set': {'1000021': {'account': {'_account_id': 1000021}, 'last_update': '2026-08-26 04:01:56.000000000', 'reason': 'Someone else replied on the change'}}, 'removed_from_attention_set': {'1000003': {'account': {'_account_id': 1000003}, 'last_update': '2026-08-26 07:01:32.000000000', 'reason': ' replied on the change', 'reason_account': {'_account_id': 1000003}}, '1000001': {'account': {'_account_id': 1000001}, 'last_update': '2026-08-20 08:23:10.000000000', 'reason': 'removed on reply'}}, 'hashtags': [], 'change_id': 'I48b6aa7a1c5904c6b68afa3ae8ec4e7d367c3615', 'subject': 'ST: always release the logical-router route-target ID on delete', 'status': 'NEW', 'created': '2026-08-06 06:10:38.000000000', 'updated': '2026-08-26 07:01:32.000000000', 'submit_type': 'MERGE_IF_NECESSARY', 'insertions': 38, 'deletions': 6, 'total_comment_count': 9, 'unresolved_comment_count': 1, 'has_review_started': True, 'meta_rev_id': '406d6807ea6d64b5effce926769ae47695b602f7', '_number': 3034, 'owner': {'_account_id': 1000021}, 'labels': {'Code-Review': {'all': [{'value': 0, 'permitted_voting_range': {'min': -2, 'max': 2}, '_account_id': 1000003}, {'value': 0, 'permitted_voting_range': {'min': -2, 'max': 2}, '_account_id': 1000001}], 'values': {'-2': 'This shall not be submitted', '-1': 'I would prefer this is not submitted as is', ' 0': 'No score', '+1': 'Looks good to me, but someone else must approve', '+2': 'Looks good to me, approved'}, 'description': '', 'default_value': 0}, 'Verified': {'all': [{'value': 0, 'permitted_voting_range': {'min': -2, 'max': 2}, '_account_id': 1000003}, {'value': -1, 'date': '2026-08-26 06:33:01.000000000', 'permitted_voting_range': {'min': -2, 'max': 2}, '_account_id': 1000001}], 'values': {'-2': 'Fails', '-1': 'Fails', ' 0': 'No score', '+1': 'Verified', '+2': 'Gating'}, 'description': '', 'default_value': 0}}, 'permitted_labels': {'Code-Review': ['-2', '-1', ' 0', '+1', '+2'], 'Verified': ['-2', '-1', ' 0', '+1', '+2']}, 'removable_labels': {'Verified': {'-1': [{'_account_id': 1000001}]}}, 'removable_reviewers': [{'_account_id': 1000003}, {'_account_id': 1000002}, {'_account_id': 1000001}], 'reviewers': {'REVIEWER': [{'_account_id': 1000001}, {'_account_id': 1000003}], 'CC': [{'_account_id': 1000002}]}, 'pending_reviewers': {}, 'current_revision': 'bf640c0e985af7951b386684af01e66c5de7e548', 'revisions': {'bf640c0e985af7951b386684af01e66c5de7e548': {'kind': 'REWORK', '_number': 3, 'created': '2026-08-20 10:54:43.000000000', 'uploader': {'_account_id': 1000021}, 'ref': 'refs/changes/34/3034/3', 'fetch': {'ssh': {'url': 'ssh://tf-gerrit@gerrit.opensdn.io:29418/opensdn-io/tf-controller', 'ref': 'refs/changes/34/3034/3'}, 'anonymous http': {'url': 'https://gerrit.opensdn.io/opensdn-io/tf-controller', 'ref': 'refs/changes/34/3034/3'}, 'http': {'url': 'https://tf-gerrit@gerrit.opensdn.io/a/opensdn-io/tf-controller', 'ref': 'refs/changes/34/3034/3'}}, 'commit': {'parents': [{'commit': '9112407ed0a26aaec3ee49b7513d267bf7e93597', 'subject': 'Switching to oneTBB: old includes removal'}], 'author': {'name': 'Charnsilp Chinprasert(Nipa)', 'email': 'justice.form@gmail.com', 'date': '2026-08-06 06:05:31.000000000', 'tz': 420}, 'committer': {'name': 'Charnsilp Chinprasert(Nipa)', 'email': 'justice.form@gmail.com', 'date': '2026-08-20 10:54:21.000000000', 'tz': 420}, 'subject': 'ST: always release the logical-router route-target ID on delete', 'message': "ST: always release the logical-router route-target ID on delete\n\nLogicalRouterST.delete_obj() runs five steps before it releases the\nroute-target ID, and every one of them is an API or datastore call that can\nraise:\n\n update_multiple_refs('virtual_machine_interface', {})\n update_multiple_refs('route_table', {})\n update_multiple_refs('bgpvpn', {})\n update_virtual_networks()\n delete_route_targets([self.route_target])\n asn = ...get_autonomous_system()\n self._object_db.free_route_target(self.name, asn)\n\nIf any of them raises, free_route_target() never runs and the ID stays\nallocated. Nothing recovers it: the logical router is gone, so delete_obj()\nis never called again, and a Zookeeper index with no row in route_target_table\nhas no reader - alloc_route_target() starts from the row, so an index without\none is never examined.\n\nA production config DB shows six route_target_table bindings naming logical\nrouters that no longer exist. Three of them still have their route-target\nobject and three do not, which places the failure between delete_route_targets()\nand free_route_target() in the first case and at or before\ndelete_route_targets() in the second.\n\nRead the ASN before the steps that can raise, and release the ID in a finally\nblock, so it is released on every path. The ASN lookup is deliberately outside\nthe try: if it stayed inside and raised - which is one of the two things that\nexplains the three-case split above - the finally clause would have no asn to\nuse.\n\nEach teardown step is attempted independently rather than under one try, so a\nfailure in an early step does not skip the later ones and the log names the\nstep that actually failed. Raised in review on patchset 1.\n\nThe per-step handling is about diagnosis rather than damage, and it is worth\nbeing precise about why. None of the five steps leaves unrecoverable state on\nits own: RouteTargetST.reinit() already deletes route-target objects that have\nno routing-instance or logical-router back-refs, and walks _RT_CF freeing any\nrow whose object is gone (resources/route_target.py:20-38); the ref sets are\nrebuilt from the DB at the same reinit. The ID release is the one step with no\nsuch collector - alloc_route_target() starts from the _RT_CF row, so a\nZookeeper index whose row is gone is never examined again - which is why it,\nand only it, sits in finally.\n\nOne consequence to note: update_virtual_networks() reads the ref sets that the\nearlier steps clear, so if an earlier step fails it works from stale membership\nand may leave a route target on a VN. That is now logged instead of being\nhidden behind an abandoned try, and reinit recomputes it. It does not justify\nskipping the steps after it.\n\nThe catch is deliberately broad. The goal is that the release below always\nruns, and narrowing it to the vnc_api exception family would let anything\nunexpected skip the remaining steps again. If the project would rather see it\nnarrowed, say so and I will.\n\nThis does not make delete_obj() idempotent, and it does not help if\nfree_route_target() itself raises partway - that is handled separately.\n\nTested on this patchset: tox -e pep8 passes clean. tox -e py39 runs 83 tests\nwith 16 skipped and one failure, test_security_group.test_stale_acl, which\nfails the same way on an unpatched tree - it hashes a dict in\n_access_control_list_update() and is unrelated to this change.\n\nChange-Id: I48b6aa7a1c5904c6b68afa3ae8ec4e7d367c3615\nSigned-off-by: Charnsilp Chinprasert(Nipa) \n"}}}, 'requirements': [], 'submit_records': [{'rule_name': 'gerrit~DefaultSubmitRule', 'status': 'NOT_READY', 'labels': [{'label': 'Code-Review', 'status': 'NEED'}, {'label': 'Verified', 'status': 'NEED'}]}]} [2026-08-26T07:02:20.285Z] DEBUG: push message data: {'labels': {'Verified': '0'}, 'message': 'TF CI Build Started (check) https://jenkins.opensdn.io/job/pipeline-check/4729/'} [Pipeline] } [Pipeline] // withCredentials [Pipeline] echo [2026-08-26T07:02:20.479Z] Notify discord for new job is started [Pipeline] } [Pipeline] // stage [Pipeline] parallel [Pipeline] { (Branch: stream-test-unit) [Pipeline] { (Branch: stream-prepare-ut-sandbox) [Pipeline] { (Branch: stream-ansible-os) [Pipeline] { (Branch: stream-build-rocky9) [Pipeline] parallel [Pipeline] { (Branch: test-unit-agent) [Pipeline] { (Branch: test-unit-opserver) [Pipeline] { (Branch: test-unit-bgp) [Pipeline] { (Branch: test-unit-group-one) [Pipeline] { (Branch: test-unit-group-two) [Pipeline] { (Branch: test-unit-ungrouped) [Pipeline] { (Branch: test-code-coverage) [Pipeline] parallel [Pipeline] { (Branch: prepare-sandbox-ut-sandbox) [Pipeline] { (Branch: fetch-sources-ut-sandbox) [Pipeline] { (Branch: configure-ut-sandbox) [Pipeline] { (Branch: cleanup-worker-ut-sandbox) [Pipeline] parallel [Pipeline] { (Branch: deploy-platform-ansible-os) [Pipeline] { (Branch: deploy-tf-ansible-os) [Pipeline] { (Branch: test-sanity-ansible-os) [Pipeline] { (Branch: collect-logs-and-cleanup-ansible-os) [Pipeline] parallel [Pipeline] { (Branch: prepare-sandbox-rocky9) [Pipeline] { (Branch: fetch-sources-rocky9) [Pipeline] { (Branch: configure-rocky9) [Pipeline] { (Branch: package-deployers-rocky9) [Pipeline] { (Branch: compile-rocky9) [Pipeline] { (Branch: package-rocky9) [Pipeline] { (Branch: freeze-rocky9) [Pipeline] { (Branch: cleanup-rocky9) [Pipeline] stage [Pipeline] { (test-unit-agent) [Pipeline] stage [Pipeline] { (test-unit-opserver) [Pipeline] stage [Pipeline] { (test-unit-bgp) [Pipeline] stage [Pipeline] { (test-unit-group-one) [Pipeline] stage [Pipeline] { (test-unit-group-two) [Pipeline] stage [Pipeline] { (test-unit-ungrouped) [Pipeline] stage [Pipeline] { (test-code-coverage) [Pipeline] stage [Pipeline] { (prepare-sandbox-ut-sandbox) [Pipeline] stage [Pipeline] { (fetch-sources-ut-sandbox) [Pipeline] stage [Pipeline] { (configure-ut-sandbox) [Pipeline] stage [Pipeline] { (cleanup-worker-ut-sandbox) [Pipeline] stage [Pipeline] { (deploy-platform-ansible-os) [Pipeline] stage [Pipeline] { (deploy-tf-ansible-os) [Pipeline] stage [Pipeline] { (test-sanity-ansible-os) [Pipeline] stage [Pipeline] { (collect-logs-and-cleanup-ansible-os) [Pipeline] stage [Pipeline] { (prepare-sandbox-rocky9) [Pipeline] stage [Pipeline] { (fetch-sources-rocky9) [Pipeline] stage [Pipeline] { (configure-rocky9) [Pipeline] stage [Pipeline] { (package-deployers-rocky9) [Pipeline] stage [Pipeline] { (compile-rocky9) [Pipeline] stage [Pipeline] { (package-rocky9) [Pipeline] stage [Pipeline] { (freeze-rocky9) [Pipeline] stage [Pipeline] { (cleanup-rocky9) [Pipeline] echo [2026-08-26T07:02:26.342Z] JOB test-unit-agent: waiting for dependecies [Pipeline] waitUntil [Pipeline] { [Pipeline] echo [2026-08-26T07:02:26.535Z] JOB test-unit-opserver: waiting for dependecies [Pipeline] waitUntil [Pipeline] { [Pipeline] echo [2026-08-26T07:02:26.666Z] JOB test-unit-bgp: waiting for dependecies [Pipeline] waitUntil [Pipeline] { [Pipeline] echo [2026-08-26T07:02:26.819Z] JOB test-unit-group-one: waiting for dependecies [Pipeline] waitUntil [Pipeline] { [Pipeline] echo [2026-08-26T07:02:27.172Z] JOB test-unit-group-two: waiting for dependecies [Pipeline] waitUntil [Pipeline] { [Pipeline] echo [2026-08-26T07:02:27.301Z] JOB test-unit-ungrouped: waiting for dependecies [Pipeline] waitUntil [Pipeline] { [Pipeline] echo [2026-08-26T07:02:27.416Z] JOB test-code-coverage: waiting for dependecies [Pipeline] waitUntil [Pipeline] { [Pipeline] echo [2026-08-26T07:02:27.493Z] JOB fetch-sources-ut-sandbox: waiting for dependecies [Pipeline] waitUntil [Pipeline] { [Pipeline] echo [2026-08-26T07:02:27.587Z] JOB configure-ut-sandbox: waiting for dependecies [Pipeline] waitUntil [Pipeline] { [Pipeline] echo [2026-08-26T07:02:28.083Z] JOB cleanup-worker-ut-sandbox: waiting for dependecies [Pipeline] waitUntil [Pipeline] { [Pipeline] echo [2026-08-26T07:02:28.197Z] JOB deploy-platform-ansible-os: waiting for dependecies [Pipeline] waitUntil [Pipeline] { [Pipeline] echo [2026-08-26T07:02:28.307Z] JOB deploy-tf-ansible-os: waiting for dependecies [Pipeline] waitUntil [Pipeline] { [Pipeline] echo [2026-08-26T07:02:28.415Z] JOB test-sanity-ansible-os: waiting for dependecies [Pipeline] waitUntil [Pipeline] { [Pipeline] echo [2026-08-26T07:02:28.488Z] JOB collect-logs-and-cleanup-ansible-os: waiting for dependecies [Pipeline] waitUntil [Pipeline] { [Pipeline] echo [2026-08-26T07:02:28.592Z] JOB fetch-sources-rocky9: waiting for dependecies [Pipeline] waitUntil [Pipeline] { [Pipeline] echo [2026-08-26T07:02:28.909Z] JOB configure-rocky9: waiting for dependecies [Pipeline] waitUntil [Pipeline] { [Pipeline] echo [2026-08-26T07:02:28.974Z] JOB package-deployers-rocky9: waiting for dependecies [Pipeline] waitUntil [Pipeline] { [Pipeline] echo [2026-08-26T07:02:29.165Z] JOB compile-rocky9: waiting for dependecies [Pipeline] waitUntil [Pipeline] { [Pipeline] echo [2026-08-26T07:02:29.474Z] JOB package-rocky9: waiting for dependecies [Pipeline] waitUntil [Pipeline] { [Pipeline] echo [2026-08-26T07:02:29.739Z] JOB freeze-rocky9: waiting for dependecies [Pipeline] waitUntil [Pipeline] { [Pipeline] echo [2026-08-26T07:02:29.916Z] JOB cleanup-rocky9: waiting for dependecies [Pipeline] waitUntil [Pipeline] { [Pipeline] echo [2026-08-26T07:02:30.114Z] JOB prepare-sandbox-ut-sandbox: entering run_job [Pipeline] echo [2026-08-26T07:02:30.141Z] JOB prepare-sandbox-rocky9: entering run_job [Pipeline] echo [2026-08-26T07:02:30.218Z] JOB test-unit-agent: waiting for dependecy [configure-ut-sandbox:null] [Pipeline] echo [2026-08-26T07:02:30.252Z] JOB test-unit-agent: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:02:30.276Z] JOB test-unit-opserver: waiting for dependecy [configure-ut-sandbox:null] [Pipeline] echo [2026-08-26T07:02:30.447Z] JOB test-unit-opserver: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:02:30.598Z] JOB test-unit-bgp: waiting for dependecy [configure-ut-sandbox:null] [Pipeline] echo [2026-08-26T07:02:30.692Z] JOB test-unit-bgp: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:02:30.898Z] JOB test-unit-group-one: waiting for dependecy [configure-ut-sandbox:null] [Pipeline] echo [2026-08-26T07:02:30.982Z] JOB test-unit-group-one: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:02:31.262Z] JOB test-unit-group-two: waiting for dependecy [configure-ut-sandbox:null] [Pipeline] echo [2026-08-26T07:02:31.356Z] JOB test-unit-group-two: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:02:31.431Z] JOB test-unit-ungrouped: waiting for dependecy [configure-ut-sandbox:null] [Pipeline] echo [2026-08-26T07:02:31.458Z] JOB test-unit-ungrouped: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:02:31.503Z] JOB fetch-sources-ut-sandbox: waiting for dependecy [prepare-sandbox-ut-sandbox:null] [Pipeline] echo [2026-08-26T07:02:31.572Z] JOB fetch-sources-ut-sandbox: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:02:31.662Z] JOB configure-ut-sandbox: waiting for dependecy [fetch-sources-ut-sandbox:null] [Pipeline] echo [2026-08-26T07:02:31.778Z] JOB configure-ut-sandbox: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:02:32.007Z] JOB cleanup-worker-ut-sandbox: waiting for dependecy [configure-ut-sandbox:null] [Pipeline] echo [2026-08-26T07:02:32.107Z] JOB cleanup-worker-ut-sandbox: waiting for all = false [Pipeline] } [Pipeline] echo [2026-08-26T07:02:32.173Z] JOB deploy-platform-ansible-os: waiting for dependecy [package-rocky9:null] [Pipeline] echo [2026-08-26T07:02:32.296Z] JOB deploy-platform-ansible-os: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:02:32.540Z] JOB test-sanity-ansible-os: waiting for dependecy [deploy-tf-ansible-os:null] [Pipeline] echo [2026-08-26T07:02:32.655Z] JOB test-sanity-ansible-os: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:02:32.679Z] JOB fetch-sources-rocky9: waiting for dependecy [prepare-sandbox-rocky9:null] [Pipeline] echo [2026-08-26T07:02:32.727Z] JOB fetch-sources-rocky9: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:02:32.790Z] JOB configure-rocky9: waiting for dependecy [fetch-sources-rocky9:null] [Pipeline] echo [2026-08-26T07:02:32.955Z] JOB configure-rocky9: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:02:33.332Z] JOB package-deployers-rocky9: waiting for dependecy [configure-rocky9:null] [Pipeline] echo [2026-08-26T07:02:33.499Z] JOB package-deployers-rocky9: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:02:33.579Z] JOB compile-rocky9: waiting for dependecy [package-deployers-rocky9:null] [Pipeline] echo [2026-08-26T07:02:33.897Z] JOB compile-rocky9: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:02:34.023Z] JOB package-rocky9: waiting for dependecy [compile-rocky9:null] [Pipeline] echo [2026-08-26T07:02:34.227Z] JOB package-rocky9: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:02:34.301Z] JOB freeze-rocky9: waiting for dependecy [package-rocky9:null] [Pipeline] echo [2026-08-26T07:02:34.456Z] JOB freeze-rocky9: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:02:34.607Z] JOB cleanup-rocky9: waiting for dependecy [freeze-rocky9:null] [Pipeline] echo [2026-08-26T07:02:34.635Z] JOB cleanup-rocky9: waiting for all = false [Pipeline] } [Pipeline] echo [2026-08-26T07:02:34.710Z] JOB deploy-tf-ansible-os: waiting for dependecy [package-rocky9:null, deploy-platform-ansible-os:null] [Pipeline] echo [2026-08-26T07:02:34.829Z] JOB deploy-tf-ansible-os: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] writeFile [Pipeline] echo [2026-08-26T07:02:35.090Z] JOB collect-logs-and-cleanup-ansible-os: waiting for dependecy [deploy-platform-ansible-os:null, deploy-tf-ansible-os:null, test-sanity-ansible-os:null] [Pipeline] echo [2026-08-26T07:02:35.131Z] JOB collect-logs-and-cleanup-ansible-os: waiting for all = false [Pipeline] } [Pipeline] writeFile [Pipeline] echo [2026-08-26T07:02:35.264Z] JOB test-code-coverage: waiting for dependecy [configure-ut-sandbox:null, test-unit-agent:null, test-unit-bgp:null, test-unit-opserver:null, test-unit-group-one:null, test-unit-group-two:null, test-unit-ungrouped:null] [Pipeline] echo [2026-08-26T07:02:35.295Z] JOB test-code-coverage: fails were not found, unfinished jobs are still present [Pipeline] } [2026-08-26T07:02:35.575Z] Will try again after 15 sec [2026-08-26T07:02:36.040Z] Will try again after 15 sec [2026-08-26T07:02:36.355Z] Will try again after 15 sec [2026-08-26T07:02:36.613Z] Will try again after 15 sec [2026-08-26T07:02:37.147Z] Will try again after 15 sec [2026-08-26T07:02:37.288Z] Will try again after 15 sec [2026-08-26T07:02:37.426Z] Will try again after 15 sec [2026-08-26T07:02:37.836Z] Will try again after 15 sec [2026-08-26T07:02:38.062Z] Will try again after 15 sec [2026-08-26T07:02:38.320Z] Will try again after 15 sec [2026-08-26T07:02:38.379Z] Will try again after 15 sec [2026-08-26T07:02:38.525Z] Will try again after 15 sec [2026-08-26T07:02:38.666Z] Will try again after 15 sec [2026-08-26T07:02:38.693Z] Will try again after 15 sec [2026-08-26T07:02:38.879Z] Will try again after 15 sec [2026-08-26T07:02:39.231Z] Will try again after 15 sec [2026-08-26T07:02:39.607Z] Will try again after 15 sec [2026-08-26T07:02:39.923Z] Will try again after 15 sec [2026-08-26T07:02:40.266Z] Will try again after 15 sec [2026-08-26T07:02:40.387Z] Will try again after 15 sec [Pipeline] archiveArtifacts [2026-08-26T07:02:40.460Z] Archiving artifacts [Pipeline] archiveArtifacts [2026-08-26T07:02:40.796Z] Archiving artifacts [2026-08-26T07:02:41.081Z] Will try again after 15 sec [Pipeline] echo [2026-08-26T07:02:46.471Z] JOB prepare-sandbox-rocky9: Starting job: build-prepare-sandbox rnd: #78533 [Pipeline] timeout [2026-08-26T07:02:46.498Z] Timeout set to expire in 3 hr 0 min [Pipeline] { [Pipeline] echo [2026-08-26T07:02:46.528Z] JOB prepare-sandbox-ut-sandbox: Starting job: build-prepare-sandbox rnd: #21070 [Pipeline] timeout [2026-08-26T07:02:46.551Z] Timeout set to expire in 3 hr 0 min [Pipeline] { [Pipeline] build [2026-08-26T07:02:46.572Z] Scheduling project: build-prepare-sandbox [Pipeline] build [2026-08-26T07:02:46.952Z] Scheduling project: build-prepare-sandbox [Pipeline] { [Pipeline] echo [2026-08-26T07:02:50.612Z] JOB test-unit-agent: waiting for dependecy [configure-ut-sandbox:null] [Pipeline] echo [2026-08-26T07:02:50.626Z] JOB test-unit-agent: fails were not found, unfinished jobs are still present [Pipeline] } [2026-08-26T07:02:50.822Z] Will try again after 15 sec [Pipeline] { [Pipeline] echo [2026-08-26T07:02:51.116Z] JOB test-unit-opserver: waiting for dependecy [configure-ut-sandbox:null] [Pipeline] echo [2026-08-26T07:02:51.312Z] JOB test-unit-opserver: fails were not found, unfinished jobs are still present [Pipeline] } [2026-08-26T07:02:51.548Z] Will try again after 15 sec [Pipeline] { [Pipeline] { [Pipeline] echo [2026-08-26T07:02:51.756Z] JOB test-unit-bgp: waiting for dependecy [configure-ut-sandbox:null] [Pipeline] echo [2026-08-26T07:02:51.779Z] JOB test-unit-bgp: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:02:51.812Z] JOB test-unit-group-one: waiting for dependecy [configure-ut-sandbox:null] [Pipeline] echo [2026-08-26T07:02:51.863Z] JOB test-unit-group-one: fails were not found, unfinished jobs are still present [Pipeline] } [2026-08-26T07:02:52.139Z] Will try again after 15 sec [2026-08-26T07:02:52.170Z] Will try again after 15 sec [Pipeline] { [Pipeline] { [Pipeline] echo [2026-08-26T07:02:52.484Z] JOB test-unit-group-two: waiting for dependecy [configure-ut-sandbox:null] [Pipeline] echo [2026-08-26T07:02:52.501Z] JOB test-unit-group-two: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:02:52.539Z] JOB test-unit-ungrouped: waiting for dependecy [configure-ut-sandbox:null] [Pipeline] echo [2026-08-26T07:02:52.566Z] JOB test-unit-ungrouped: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] { [2026-08-26T07:02:52.997Z] Will try again after 15 sec [2026-08-26T07:02:53.331Z] Will try again after 15 sec [Pipeline] { [Pipeline] echo [2026-08-26T07:02:53.540Z] JOB fetch-sources-ut-sandbox: waiting for dependecy [prepare-sandbox-ut-sandbox:null] [Pipeline] echo [2026-08-26T07:02:53.564Z] JOB fetch-sources-ut-sandbox: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] { [Pipeline] { [Pipeline] echo [2026-08-26T07:02:53.849Z] JOB configure-ut-sandbox: waiting for dependecy [fetch-sources-ut-sandbox:null] [Pipeline] echo [2026-08-26T07:02:53.872Z] JOB configure-ut-sandbox: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] { [Pipeline] { [2026-08-26T07:02:54.387Z] Will try again after 15 sec [Pipeline] echo [2026-08-26T07:02:54.417Z] JOB cleanup-worker-ut-sandbox: waiting for dependecy [configure-ut-sandbox:null] [Pipeline] echo [2026-08-26T07:02:54.496Z] JOB cleanup-worker-ut-sandbox: waiting for all = false [Pipeline] } [Pipeline] echo [2026-08-26T07:02:54.562Z] JOB deploy-platform-ansible-os: waiting for dependecy [package-rocky9:null] [Pipeline] echo [2026-08-26T07:02:54.587Z] JOB deploy-platform-ansible-os: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] { [Pipeline] { [Pipeline] echo [2026-08-26T07:02:54.675Z] JOB test-sanity-ansible-os: waiting for dependecy [deploy-tf-ansible-os:null] [Pipeline] echo [2026-08-26T07:02:54.839Z] JOB test-sanity-ansible-os: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:02:54.875Z] JOB fetch-sources-rocky9: waiting for dependecy [prepare-sandbox-rocky9:null] [Pipeline] echo [2026-08-26T07:02:54.930Z] JOB fetch-sources-rocky9: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] { [2026-08-26T07:02:55.168Z] Will try again after 15 sec [Pipeline] echo [2026-08-26T07:02:55.237Z] JOB configure-rocky9: waiting for dependecy [fetch-sources-rocky9:null] [Pipeline] echo [2026-08-26T07:02:55.408Z] JOB configure-rocky9: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:02:55.494Z] JOB package-deployers-rocky9: waiting for dependecy [configure-rocky9:null] [Pipeline] echo [2026-08-26T07:02:55.670Z] JOB package-deployers-rocky9: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:02:55.772Z] JOB compile-rocky9: waiting for dependecy [package-deployers-rocky9:null] [Pipeline] echo [2026-08-26T07:02:55.878Z] JOB compile-rocky9: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] { [2026-08-26T07:02:56.484Z] Starting building: build-prepare-sandbox #15885 [2026-08-26T07:02:56.769Z] Will try again after 15 sec [2026-08-26T07:02:56.863Z] Will try again after 15 sec [Pipeline] { [Pipeline] echo [2026-08-26T07:02:56.913Z] JOB package-rocky9: waiting for dependecy [compile-rocky9:null] [Pipeline] echo [2026-08-26T07:02:56.999Z] JOB package-rocky9: fails were not found, unfinished jobs are still present [Pipeline] } [2026-08-26T07:02:57.159Z] Will try again after 15 sec [Pipeline] { [2026-08-26T07:02:57.474Z] Will try again after 15 sec [Pipeline] echo [2026-08-26T07:02:57.525Z] JOB freeze-rocky9: waiting for dependecy [package-rocky9:null] [Pipeline] echo [2026-08-26T07:02:57.555Z] JOB freeze-rocky9: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:02:57.774Z] JOB cleanup-rocky9: waiting for dependecy [freeze-rocky9:null] [Pipeline] echo [2026-08-26T07:02:57.789Z] JOB cleanup-rocky9: waiting for all = false [Pipeline] } [Pipeline] { [Pipeline] { [2026-08-26T07:02:57.976Z] Will try again after 15 sec [2026-08-26T07:02:58.127Z] Will try again after 15 sec [2026-08-26T07:02:58.258Z] Will try again after 15 sec [Pipeline] { [Pipeline] echo [2026-08-26T07:02:58.588Z] JOB deploy-tf-ansible-os: waiting for dependecy [package-rocky9:null, deploy-platform-ansible-os:null] [Pipeline] echo [2026-08-26T07:02:58.607Z] JOB deploy-tf-ansible-os: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:02:58.908Z] JOB collect-logs-and-cleanup-ansible-os: waiting for dependecy [deploy-platform-ansible-os:null, deploy-tf-ansible-os:null, test-sanity-ansible-os:null] [Pipeline] echo [2026-08-26T07:02:58.968Z] JOB collect-logs-and-cleanup-ansible-os: waiting for all = false [Pipeline] } [2026-08-26T07:02:59.313Z] Will try again after 15 sec [2026-08-26T07:02:59.812Z] Will try again after 15 sec [Pipeline] echo [2026-08-26T07:02:59.913Z] JOB test-code-coverage: waiting for dependecy [configure-ut-sandbox:null, test-unit-agent:null, test-unit-bgp:null, test-unit-opserver:null, test-unit-group-one:null, test-unit-group-two:null, test-unit-ungrouped:null] [Pipeline] echo [2026-08-26T07:02:59.944Z] JOB test-code-coverage: fails were not found, unfinished jobs are still present [Pipeline] } [2026-08-26T07:03:00.210Z] Will try again after 15 sec [2026-08-26T07:03:00.450Z] Will try again after 15 sec [2026-08-26T07:03:00.563Z] Will try again after 15 sec [2026-08-26T07:03:00.671Z] Will try again after 15 sec [2026-08-26T07:03:01.653Z] Starting building: build-prepare-sandbox #15886 [Pipeline] { [Pipeline] { [Pipeline] { [Pipeline] { [Pipeline] echo [2026-08-26T07:03:07.520Z] JOB test-unit-agent: waiting for dependecy [configure-ut-sandbox:null] [Pipeline] echo [2026-08-26T07:03:07.569Z] JOB test-unit-agent: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:03:07.624Z] JOB test-unit-opserver: waiting for dependecy [configure-ut-sandbox:null] [Pipeline] echo [2026-08-26T07:03:07.678Z] JOB test-unit-opserver: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:03:07.715Z] JOB test-unit-bgp: waiting for dependecy [configure-ut-sandbox:null] [Pipeline] echo [2026-08-26T07:03:07.744Z] JOB test-unit-bgp: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:03:07.769Z] JOB test-unit-group-one: waiting for dependecy [configure-ut-sandbox:null] [Pipeline] echo [2026-08-26T07:03:07.787Z] JOB test-unit-group-one: fails were not found, unfinished jobs are still present [Pipeline] } [2026-08-26T07:03:07.927Z] Will try again after 15 sec [2026-08-26T07:03:08.002Z] Will try again after 15 sec [2026-08-26T07:03:08.054Z] Will try again after 15 sec [2026-08-26T07:03:08.097Z] Will try again after 15 sec [Pipeline] { [Pipeline] { [Pipeline] echo [2026-08-26T07:03:08.744Z] JOB test-unit-group-two: waiting for dependecy [configure-ut-sandbox:null] [Pipeline] echo [2026-08-26T07:03:08.763Z] JOB test-unit-group-two: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:03:08.797Z] JOB test-unit-ungrouped: waiting for dependecy [configure-ut-sandbox:null] [Pipeline] echo [2026-08-26T07:03:08.848Z] JOB test-unit-ungrouped: fails were not found, unfinished jobs are still present [Pipeline] } [2026-08-26T07:03:09.059Z] Will try again after 15 sec [2026-08-26T07:03:09.104Z] Will try again after 15 sec [Pipeline] { [Pipeline] echo [2026-08-26T07:03:09.456Z] JOB fetch-sources-ut-sandbox: waiting for dependecy [prepare-sandbox-ut-sandbox:null] [Pipeline] echo [2026-08-26T07:03:09.670Z] JOB fetch-sources-ut-sandbox: fails were not found, unfinished jobs are still present [Pipeline] } [2026-08-26T07:03:09.828Z] Will try again after 15 sec [Pipeline] { [Pipeline] echo [2026-08-26T07:03:10.229Z] JOB configure-ut-sandbox: waiting for dependecy [fetch-sources-ut-sandbox:null] [Pipeline] echo [2026-08-26T07:03:10.255Z] JOB configure-ut-sandbox: fails were not found, unfinished jobs are still present [Pipeline] } [2026-08-26T07:03:10.342Z] Build build-prepare-sandbox #15885 completed: FAILURE [2026-08-26T07:03:10.358Z] Build build-prepare-sandbox #15886 completed: FAILURE [2026-08-26T07:03:10.495Z] Will try again after 15 sec [Pipeline] } [Pipeline] } [Pipeline] // timeout [Pipeline] // timeout [Pipeline] echo [2026-08-26T07:03:11.050Z] JOB prepare-sandbox-ut-sandbox: Failed [Pipeline] copyArtifacts [Pipeline] echo [2026-08-26T07:03:11.832Z] JOB prepare-sandbox-rocky9: Failed [Pipeline] copyArtifacts [Pipeline] { [Pipeline] { [Pipeline] echo [2026-08-26T07:03:12.346Z] JOB prepare-sandbox-ut-sandbox: error in job!!! [Pipeline] echo [2026-08-26T07:03:12.522Z] JOB prepare-sandbox-ut-sandbox: Err - org.jenkinsci.plugins.workflow.support.steps.AgentOfflineException: Unable to create live FilePath for slave-openstack-gz1_1; slave-openstack-gz1_1 was marked offline: Connection was broken [Pipeline] echo [2026-08-26T07:03:12.588Z] JOB prepare-sandbox-ut-sandbox: Message - Unable to create live FilePath for slave-openstack-gz1_1; slave-openstack-gz1_1 was marked offline: Connection was broken [Pipeline] echo [2026-08-26T07:03:12.605Z] JOB prepare-sandbox-ut-sandbox: Cause - null [Pipeline] echo [2026-08-26T07:03:12.624Z] JOB prepare-sandbox-ut-sandbox: Stacktrace - [PluginClassLoader for workflow-durable-task-step//org.jenkinsci.plugins.workflow.support.steps.ExecutorStepDynamicContext$FilePathTranslator.get(ExecutorStepDynamicContext.java:188), PluginClassLoader for workflow-durable-task-step//org.jenkinsci.plugins.workflow.support.steps.ExecutorStepDynamicContext$FilePathTranslator.get(ExecutorStepDynamicContext.java:160), PluginClassLoader for workflow-durable-task-step//org.jenkinsci.plugins.workflow.support.steps.ExecutorStepDynamicContext$Translator.get(ExecutorStepDynamicContext.java:153), PluginClassLoader for workflow-durable-task-step//org.jenkinsci.plugins.workflow.support.steps.ExecutorStepDynamicContext$FilePathTranslator.get(ExecutorStepDynamicContext.java:170), PluginClassLoader for workflow-durable-task-step//org.jenkinsci.plugins.workflow.support.steps.ExecutorStepDynamicContext$FilePathTranslator.get(ExecutorStepDynamicContext.java:160), PluginClassLoader for workflow-step-api//org.jenkinsci.plugins.workflow.steps.DynamicContext$Typed.get(DynamicContext.java:95), PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.ContextVariableSet.get(ContextVariableSet.java:139), PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsThread.getContextVariable(CpsThread.java:135), PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsStepContext.doGet(CpsStepContext.java:297), PluginClassLoader for workflow-support//org.jenkinsci.plugins.workflow.support.DefaultStepContext.get(DefaultStepContext.java:75), PluginClassLoader for workflow-basic-steps//org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:83), PluginClassLoader for workflow-basic-steps//org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:71), PluginClassLoader for workflow-step-api//org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47), java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source), java.base/java.util.concurrent.FutureTask.run(Unknown Source), java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source), java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source), java.base/java.lang.Thread.run(Unknown Source)] [Pipeline] } [Pipeline] echo [2026-08-26T07:03:12.666Z] JOB prepare-sandbox-rocky9: error in job!!! [Pipeline] echo [2026-08-26T07:03:12.680Z] JOB prepare-sandbox-rocky9: Err - org.jenkinsci.plugins.workflow.support.steps.AgentOfflineException: Unable to create live FilePath for slave-openstack-gz1_1; slave-openstack-gz1_1 was marked offline: Connection was broken [Pipeline] echo [2026-08-26T07:03:12.695Z] JOB prepare-sandbox-rocky9: Message - Unable to create live FilePath for slave-openstack-gz1_1; slave-openstack-gz1_1 was marked offline: Connection was broken [Pipeline] echo [2026-08-26T07:03:12.711Z] JOB prepare-sandbox-rocky9: Cause - null [Pipeline] echo [2026-08-26T07:03:12.729Z] JOB prepare-sandbox-rocky9: Stacktrace - [PluginClassLoader for workflow-durable-task-step//org.jenkinsci.plugins.workflow.support.steps.ExecutorStepDynamicContext$FilePathTranslator.get(ExecutorStepDynamicContext.java:188), PluginClassLoader for workflow-durable-task-step//org.jenkinsci.plugins.workflow.support.steps.ExecutorStepDynamicContext$FilePathTranslator.get(ExecutorStepDynamicContext.java:160), PluginClassLoader for workflow-durable-task-step//org.jenkinsci.plugins.workflow.support.steps.ExecutorStepDynamicContext$Translator.get(ExecutorStepDynamicContext.java:153), PluginClassLoader for workflow-durable-task-step//org.jenkinsci.plugins.workflow.support.steps.ExecutorStepDynamicContext$FilePathTranslator.get(ExecutorStepDynamicContext.java:170), PluginClassLoader for workflow-durable-task-step//org.jenkinsci.plugins.workflow.support.steps.ExecutorStepDynamicContext$FilePathTranslator.get(ExecutorStepDynamicContext.java:160), PluginClassLoader for workflow-step-api//org.jenkinsci.plugins.workflow.steps.DynamicContext$Typed.get(DynamicContext.java:95), PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.ContextVariableSet.get(ContextVariableSet.java:139), PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsThread.getContextVariable(CpsThread.java:135), PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsStepContext.doGet(CpsStepContext.java:297), PluginClassLoader for workflow-support//org.jenkinsci.plugins.workflow.support.DefaultStepContext.get(DefaultStepContext.java:75), PluginClassLoader for workflow-basic-steps//org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:83), PluginClassLoader for workflow-basic-steps//org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:71), PluginClassLoader for workflow-step-api//org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47), java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source), java.base/java.util.concurrent.FutureTask.run(Unknown Source), java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source), java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source), java.base/java.lang.Thread.run(Unknown Source)] [Pipeline] } [Pipeline] { [Pipeline] echo [2026-08-26T07:03:12.783Z] JOB cleanup-worker-ut-sandbox: waiting for dependecy [configure-ut-sandbox:null] [Pipeline] echo [2026-08-26T07:03:12.803Z] JOB cleanup-worker-ut-sandbox: waiting for all = false [Pipeline] } [Pipeline] echo [2026-08-26T07:03:12.829Z] JOB deploy-platform-ansible-os: waiting for dependecy [package-rocky9:null] [Pipeline] echo [2026-08-26T07:03:12.847Z] JOB deploy-platform-ansible-os: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:03:12.872Z] JOB test-sanity-ansible-os: waiting for dependecy [deploy-tf-ansible-os:null] [Pipeline] echo [2026-08-26T07:03:12.909Z] JOB test-sanity-ansible-os: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] { [Pipeline] // stage [Pipeline] // stage [Pipeline] } [2026-08-26T07:03:13.501Z] Failed in branch prepare-sandbox-ut-sandbox [Pipeline] } [2026-08-26T07:03:13.532Z] Failed in branch prepare-sandbox-rocky9 [Pipeline] echo [2026-08-26T07:03:13.571Z] JOB fetch-sources-rocky9: waiting for dependecy [prepare-sandbox-rocky9:FAILURE] [Pipeline] } [2026-08-26T07:03:13.742Z] Will try again after 15 sec [2026-08-26T07:03:13.797Z] Will try again after 15 sec [2026-08-26T07:03:13.833Z] Will try again after 15 sec [Pipeline] { [Pipeline] { [Pipeline] { [Pipeline] echo [2026-08-26T07:03:13.903Z] JOB configure-rocky9: waiting for dependecy [fetch-sources-rocky9:null] [Pipeline] echo [2026-08-26T07:03:13.917Z] JOB configure-rocky9: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:03:13.946Z] JOB package-deployers-rocky9: waiting for dependecy [configure-rocky9:null] [Pipeline] echo [2026-08-26T07:03:13.960Z] JOB package-deployers-rocky9: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:03:14.284Z] JOB compile-rocky9: waiting for dependecy [package-deployers-rocky9:null] [Pipeline] echo [2026-08-26T07:03:14.319Z] JOB compile-rocky9: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] // waitUntil [Pipeline] echo [2026-08-26T07:03:14.707Z] JOB fetch-sources-rocky9: wait finished. overall result = false [Pipeline] } [2026-08-26T07:03:15.004Z] Will try again after 15 sec [2026-08-26T07:03:15.059Z] Will try again after 15 sec [Pipeline] { [2026-08-26T07:03:15.238Z] Will try again after 15 sec [Pipeline] // stage [Pipeline] { [Pipeline] } [Pipeline] echo [2026-08-26T07:03:15.421Z] JOB package-rocky9: waiting for dependecy [compile-rocky9:null] [Pipeline] echo [2026-08-26T07:03:15.440Z] JOB package-rocky9: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:03:15.619Z] JOB freeze-rocky9: waiting for dependecy [package-rocky9:null] [Pipeline] echo [2026-08-26T07:03:15.633Z] JOB freeze-rocky9: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] { [2026-08-26T07:03:15.786Z] Will try again after 15 sec [Pipeline] { [Pipeline] { [2026-08-26T07:03:15.848Z] Will try again after 15 sec [Pipeline] echo [2026-08-26T07:03:15.863Z] JOB cleanup-rocky9: waiting for dependecy [freeze-rocky9:null] [Pipeline] echo [2026-08-26T07:03:15.878Z] JOB cleanup-rocky9: waiting for all = false [Pipeline] } [Pipeline] { [Pipeline] echo [2026-08-26T07:03:15.953Z] JOB deploy-tf-ansible-os: waiting for dependecy [package-rocky9:null, deploy-platform-ansible-os:null] [Pipeline] echo [2026-08-26T07:03:15.970Z] JOB deploy-tf-ansible-os: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:03:16.004Z] JOB collect-logs-and-cleanup-ansible-os: waiting for dependecy [deploy-platform-ansible-os:null, deploy-tf-ansible-os:null, test-sanity-ansible-os:null] [Pipeline] echo [2026-08-26T07:03:16.018Z] JOB collect-logs-and-cleanup-ansible-os: waiting for all = false [Pipeline] } [2026-08-26T07:03:16.231Z] Will try again after 15 sec [2026-08-26T07:03:16.334Z] Will try again after 15 sec [Pipeline] echo [2026-08-26T07:03:16.380Z] JOB test-code-coverage: waiting for dependecy [configure-ut-sandbox:null, test-unit-agent:null, test-unit-bgp:null, test-unit-opserver:null, test-unit-group-one:null, test-unit-group-two:null, test-unit-ungrouped:null] [Pipeline] echo [2026-08-26T07:03:16.400Z] JOB test-code-coverage: fails were not found, unfinished jobs are still present [Pipeline] } [2026-08-26T07:03:16.567Z] Will try again after 15 sec [2026-08-26T07:03:16.746Z] Will try again after 15 sec [Pipeline] { [Pipeline] { [Pipeline] echo [2026-08-26T07:03:23.079Z] JOB test-unit-agent: waiting for dependecy [configure-ut-sandbox:null] [Pipeline] echo [2026-08-26T07:03:23.120Z] JOB test-unit-agent: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:03:23.252Z] JOB test-unit-opserver: waiting for dependecy [configure-ut-sandbox:null] [Pipeline] echo [2026-08-26T07:03:23.342Z] JOB test-unit-opserver: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] { [Pipeline] { [2026-08-26T07:03:23.739Z] Will try again after 15 sec [2026-08-26T07:03:23.811Z] Will try again after 15 sec [Pipeline] echo [2026-08-26T07:03:23.956Z] JOB test-unit-bgp: waiting for dependecy [configure-ut-sandbox:null] [Pipeline] echo [2026-08-26T07:03:23.981Z] JOB test-unit-bgp: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:03:24.020Z] JOB test-unit-group-one: waiting for dependecy [configure-ut-sandbox:null] [Pipeline] echo [2026-08-26T07:03:24.039Z] JOB test-unit-group-one: fails were not found, unfinished jobs are still present [Pipeline] } [2026-08-26T07:03:24.623Z] Will try again after 15 sec [2026-08-26T07:03:24.718Z] Will try again after 15 sec [Pipeline] { [Pipeline] { [Pipeline] { [Pipeline] echo [2026-08-26T07:03:25.220Z] JOB test-unit-ungrouped: waiting for dependecy [configure-ut-sandbox:null] [Pipeline] echo [2026-08-26T07:03:25.248Z] JOB test-unit-ungrouped: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:03:25.343Z] JOB test-unit-group-two: waiting for dependecy [configure-ut-sandbox:null] [Pipeline] echo [2026-08-26T07:03:25.451Z] JOB test-unit-group-two: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:03:25.746Z] JOB fetch-sources-ut-sandbox: waiting for dependecy [prepare-sandbox-ut-sandbox:FAILURE] [Pipeline] } [2026-08-26T07:03:26.170Z] Will try again after 15 sec [2026-08-26T07:03:28.957Z] Will try again after 15 sec [Pipeline] { [Pipeline] // waitUntil [Pipeline] { [Pipeline] { [Pipeline] { [Pipeline] echo [2026-08-26T07:03:29.545Z] JOB fetch-sources-ut-sandbox: wait finished. overall result = false [Pipeline] } [Pipeline] echo [2026-08-26T07:03:29.599Z] JOB configure-ut-sandbox: waiting for dependecy [fetch-sources-ut-sandbox:NOT_BUILT] [Pipeline] } [Pipeline] echo [2026-08-26T07:03:29.631Z] JOB cleanup-worker-ut-sandbox: waiting for dependecy [configure-ut-sandbox:null] [Pipeline] echo [2026-08-26T07:03:29.668Z] JOB cleanup-worker-ut-sandbox: waiting for all = false [Pipeline] } [Pipeline] echo [2026-08-26T07:03:29.696Z] JOB deploy-platform-ansible-os: waiting for dependecy [package-rocky9:null] [Pipeline] echo [2026-08-26T07:03:29.769Z] JOB deploy-platform-ansible-os: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:03:29.814Z] JOB test-sanity-ansible-os: waiting for dependecy [deploy-tf-ansible-os:null] [Pipeline] echo [2026-08-26T07:03:29.899Z] JOB test-sanity-ansible-os: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] // stage [Pipeline] } [Pipeline] // waitUntil [2026-08-26T07:03:30.639Z] Will try again after 15 sec [2026-08-26T07:03:30.679Z] Will try again after 15 sec [2026-08-26T07:03:30.866Z] Will try again after 15 sec [Pipeline] { [Pipeline] { [Pipeline] { [Pipeline] echo [2026-08-26T07:03:31.208Z] JOB configure-ut-sandbox: wait finished. overall result = false [Pipeline] } [Pipeline] { [Pipeline] { [Pipeline] echo [2026-08-26T07:03:31.908Z] JOB configure-rocky9: waiting for dependecy [fetch-sources-rocky9:NOT_BUILT] [Pipeline] } [Pipeline] echo [2026-08-26T07:03:31.942Z] JOB package-deployers-rocky9: waiting for dependecy [configure-rocky9:null] [Pipeline] echo [2026-08-26T07:03:31.989Z] JOB package-deployers-rocky9: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:03:32.176Z] JOB compile-rocky9: waiting for dependecy [package-deployers-rocky9:null] [Pipeline] echo [2026-08-26T07:03:32.248Z] JOB compile-rocky9: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:03:32.381Z] JOB package-rocky9: waiting for dependecy [compile-rocky9:null] [Pipeline] echo [2026-08-26T07:03:32.403Z] JOB package-rocky9: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:03:32.452Z] JOB freeze-rocky9: waiting for dependecy [package-rocky9:null] [Pipeline] echo [2026-08-26T07:03:32.555Z] JOB freeze-rocky9: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] // stage [Pipeline] } [Pipeline] { [Pipeline] { [Pipeline] { [Pipeline] { [Pipeline] echo [2026-08-26T07:03:33.482Z] JOB cleanup-rocky9: waiting for dependecy [freeze-rocky9:null] [Pipeline] echo [2026-08-26T07:03:33.498Z] JOB cleanup-rocky9: waiting for all = false [Pipeline] } [Pipeline] // waitUntil [2026-08-26T07:03:33.629Z] Will try again after 15 sec [2026-08-26T07:03:33.705Z] Will try again after 15 sec [Pipeline] echo [2026-08-26T07:03:33.725Z] JOB configure-rocky9: wait finished. overall result = false [Pipeline] } [Pipeline] echo [2026-08-26T07:03:33.750Z] JOB deploy-tf-ansible-os: waiting for dependecy [package-rocky9:null, deploy-platform-ansible-os:null] [Pipeline] echo [2026-08-26T07:03:33.763Z] JOB deploy-tf-ansible-os: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:03:34.014Z] JOB collect-logs-and-cleanup-ansible-os: waiting for dependecy [deploy-platform-ansible-os:null, deploy-tf-ansible-os:null, test-sanity-ansible-os:null] [Pipeline] echo [2026-08-26T07:03:34.046Z] JOB collect-logs-and-cleanup-ansible-os: waiting for all = false [Pipeline] } [2026-08-26T07:03:34.082Z] Will try again after 15 sec [2026-08-26T07:03:34.106Z] Will try again after 15 sec [Pipeline] echo [2026-08-26T07:03:34.213Z] JOB test-code-coverage: waiting for dependecy [configure-ut-sandbox:NOT_BUILT, test-unit-agent:null, test-unit-bgp:null, test-unit-opserver:null, test-unit-group-one:null, test-unit-group-two:null, test-unit-ungrouped:null] [Pipeline] } [2026-08-26T07:03:34.338Z] Will try again after 15 sec [Pipeline] // stage [2026-08-26T07:03:34.546Z] Will try again after 15 sec [Pipeline] } [2026-08-26T07:03:35.105Z] Will try again after 15 sec [Pipeline] // waitUntil [Pipeline] echo [2026-08-26T07:03:35.644Z] JOB test-code-coverage: wait finished. overall result = false [Pipeline] } [Pipeline] // stage [Pipeline] } [Pipeline] { [Pipeline] { [Pipeline] echo [2026-08-26T07:03:39.050Z] JOB test-unit-agent: waiting for dependecy [configure-ut-sandbox:NOT_BUILT] [Pipeline] } [Pipeline] echo [2026-08-26T07:03:39.117Z] JOB test-unit-opserver: waiting for dependecy [configure-ut-sandbox:NOT_BUILT] [Pipeline] } [Pipeline] // waitUntil [Pipeline] // waitUntil [Pipeline] echo [2026-08-26T07:03:39.838Z] JOB test-unit-agent: wait finished. overall result = false [Pipeline] } [Pipeline] echo [2026-08-26T07:03:39.857Z] JOB test-unit-opserver: wait finished. overall result = false [Pipeline] } [Pipeline] { [Pipeline] { [Pipeline] // stage [Pipeline] // stage [Pipeline] } [Pipeline] } [Pipeline] echo [2026-08-26T07:03:40.459Z] JOB test-unit-bgp: waiting for dependecy [configure-ut-sandbox:NOT_BUILT] [Pipeline] } [Pipeline] echo [2026-08-26T07:03:40.484Z] JOB test-unit-group-one: waiting for dependecy [configure-ut-sandbox:NOT_BUILT] [Pipeline] } [Pipeline] // waitUntil [Pipeline] // waitUntil [Pipeline] echo [2026-08-26T07:03:40.996Z] JOB test-unit-bgp: wait finished. overall result = false [Pipeline] } [Pipeline] echo [2026-08-26T07:03:41.121Z] JOB test-unit-group-one: wait finished. overall result = false [Pipeline] } [Pipeline] // stage [Pipeline] // stage [Pipeline] { [Pipeline] } [Pipeline] } [Pipeline] echo [2026-08-26T07:03:41.687Z] JOB test-unit-ungrouped: waiting for dependecy [configure-ut-sandbox:NOT_BUILT] [Pipeline] } [Pipeline] // waitUntil [Pipeline] echo [2026-08-26T07:03:42.234Z] JOB test-unit-ungrouped: wait finished. overall result = false [Pipeline] } [Pipeline] // stage [Pipeline] } [Pipeline] { [Pipeline] echo [2026-08-26T07:03:44.059Z] JOB test-unit-group-two: waiting for dependecy [configure-ut-sandbox:NOT_BUILT] [Pipeline] } [Pipeline] // waitUntil [Pipeline] echo [2026-08-26T07:03:44.281Z] JOB test-unit-group-two: wait finished. overall result = false [Pipeline] } [Pipeline] // stage [Pipeline] } [Pipeline] // parallel [Pipeline] } [Pipeline] { [Pipeline] { [Pipeline] echo [2026-08-26T07:03:45.915Z] JOB cleanup-worker-ut-sandbox: waiting for dependecy [configure-ut-sandbox:NOT_BUILT] [Pipeline] echo [2026-08-26T07:03:45.959Z] JOB cleanup-worker-ut-sandbox: waiting for all = true [Pipeline] } [Pipeline] echo [2026-08-26T07:03:45.999Z] JOB deploy-platform-ansible-os: waiting for dependecy [package-rocky9:null] [Pipeline] echo [2026-08-26T07:03:46.028Z] JOB deploy-platform-ansible-os: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] { [Pipeline] // waitUntil [2026-08-26T07:03:46.630Z] Will try again after 15 sec [Pipeline] echo [2026-08-26T07:03:46.668Z] JOB cleanup-worker-ut-sandbox: wait finished. overall result = true [Pipeline] echo [2026-08-26T07:03:46.919Z] JOB cleanup-worker-ut-sandbox: entering run_job [Pipeline] echo [2026-08-26T07:03:47.912Z] JOB test-sanity-ansible-os: waiting for dependecy [deploy-tf-ansible-os:null] [Pipeline] echo [2026-08-26T07:03:48.062Z] JOB test-sanity-ansible-os: fails were not found, unfinished jobs are still present [Pipeline] } [2026-08-26T07:03:48.343Z] Will try again after 15 sec [Pipeline] writeFile [Pipeline] echo [2026-08-26T07:03:48.722Z] JOB cleanup-worker-ut-sandbox: Failed [Pipeline] echo [2026-08-26T07:03:48.810Z] JOB cleanup-worker-ut-sandbox: err msg: Unable to create live FilePath for slave-openstack-gz1_1; slave-openstack-gz1_1 was marked offline: Connection was broken [Pipeline] echo [2026-08-26T07:03:51.377Z] JOB cleanup-worker-ut-sandbox: Error in obtaining failed job result No signature of method: org.jenkinsci.plugins.workflow.support.steps.AgentOfflineException.getCauses() is applicable for argument types: () values: [] [2026-08-26T07:03:51.377Z] Possible solutions: getCause(), getClass(), initCause(java.lang.Throwable), metaClass(groovy.lang.Closure) [Pipeline] echo [2026-08-26T07:03:51.423Z] JOB cleanup-worker-ut-sandbox: error in job!!! [Pipeline] echo [2026-08-26T07:03:51.446Z] JOB cleanup-worker-ut-sandbox: Err - org.jenkinsci.plugins.workflow.support.steps.AgentOfflineException: Unable to create live FilePath for slave-openstack-gz1_1; slave-openstack-gz1_1 was marked offline: Connection was broken [Pipeline] echo [2026-08-26T07:03:51.494Z] JOB cleanup-worker-ut-sandbox: Message - Unable to create live FilePath for slave-openstack-gz1_1; slave-openstack-gz1_1 was marked offline: Connection was broken [Pipeline] echo [2026-08-26T07:03:51.556Z] JOB cleanup-worker-ut-sandbox: Cause - null [Pipeline] echo [2026-08-26T07:03:51.580Z] JOB cleanup-worker-ut-sandbox: Stacktrace - [PluginClassLoader for workflow-durable-task-step//org.jenkinsci.plugins.workflow.support.steps.ExecutorStepDynamicContext$FilePathTranslator.get(ExecutorStepDynamicContext.java:188), PluginClassLoader for workflow-durable-task-step//org.jenkinsci.plugins.workflow.support.steps.ExecutorStepDynamicContext$FilePathTranslator.get(ExecutorStepDynamicContext.java:160), PluginClassLoader for workflow-durable-task-step//org.jenkinsci.plugins.workflow.support.steps.ExecutorStepDynamicContext$Translator.get(ExecutorStepDynamicContext.java:153), PluginClassLoader for workflow-durable-task-step//org.jenkinsci.plugins.workflow.support.steps.ExecutorStepDynamicContext$FilePathTranslator.get(ExecutorStepDynamicContext.java:170), PluginClassLoader for workflow-durable-task-step//org.jenkinsci.plugins.workflow.support.steps.ExecutorStepDynamicContext$FilePathTranslator.get(ExecutorStepDynamicContext.java:160), PluginClassLoader for workflow-step-api//org.jenkinsci.plugins.workflow.steps.DynamicContext$Typed.get(DynamicContext.java:95), PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.ContextVariableSet.get(ContextVariableSet.java:139), PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsThread.getContextVariable(CpsThread.java:135), PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsStepContext.doGet(CpsStepContext.java:297), PluginClassLoader for workflow-support//org.jenkinsci.plugins.workflow.support.DefaultStepContext.get(DefaultStepContext.java:75), PluginClassLoader for workflow-step-api//org.jenkinsci.plugins.workflow.steps.StepDescriptor.checkContextAvailability(StepDescriptor.java:263), PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:300), PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:196), PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:124), jdk.internal.reflect.GeneratedMethodAccessor575682.invoke(Unknown Source), java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source), java.base/java.lang.reflect.Method.invoke(Unknown Source), org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:98), groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325), groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1225), groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1034), org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:41), org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47), org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116), PluginClassLoader for script-security//org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:180), PluginClassLoader for script-security//org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23), PluginClassLoader for script-security//org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:163), PluginClassLoader for script-security//org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:148), PluginClassLoader for script-security//org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:178), PluginClassLoader for script-security//org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:182), PluginClassLoader for workflow-cps//com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17), PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.LoggingInvoker.methodCall(LoggingInvoker.java:117), Script5._job_params_to_file(Script5.groovy:461), Script5._run_job(Script5.groovy:547), Script5._process_job(Script5.groovy:340), ___cps.transform___(Native Method), PluginClassLoader for workflow-cps//com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:90), PluginClassLoader for workflow-cps//com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:116), PluginClassLoader for workflow-cps//com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:85), jdk.internal.reflect.GeneratedMethodAccessor79.invoke(Unknown Source), java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source), java.base/java.lang.reflect.Method.invoke(Unknown Source), PluginClassLoader for workflow-cps//com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72), PluginClassLoader for workflow-cps//com.cloudbees.groovy.cps.impl.CollectionLiteralBlock$ContinuationImpl.dispatch(CollectionLiteralBlock.java:55), PluginClassLoader for workflow-cps//com.cloudbees.groovy.cps.impl.CollectionLiteralBlock$ContinuationImpl.item(CollectionLiteralBlock.java:45), jdk.internal.reflect.GeneratedMethodAccessor71.invoke(Unknown Source), java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source), java.base/java.lang.reflect.Method.invoke(Unknown Source), PluginClassLoader for workflow-cps//com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72), PluginClassLoader for workflow-cps//com.cloudbees.groovy.cps.impl.LocalVariableBlock$LocalVariable.get(LocalVariableBlock.java:39), PluginClassLoader for workflow-cps//com.cloudbees.groovy.cps.LValueBlock$GetAdapter.receive(LValueBlock.java:30), PluginClassLoader for workflow-cps//com.cloudbees.groovy.cps.impl.LocalVariableBlock.evalLValue(LocalVariableBlock.java:28), PluginClassLoader for workflow-cps//com.cloudbees.groovy.cps.LValueBlock$BlockImpl.eval(LValueBlock.java:55), PluginClassLoader for workflow-cps//com.cloudbees.groovy.cps.LValueBlock.eval(LValueBlock.java:16), PluginClassLoader for workflow-cps//com.cloudbees.groovy.cps.Next.step(Next.java:83), PluginClassLoader for workflow-cps//com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:147), PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:17), PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:49), PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:180), PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:423), PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:331), PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:295), PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService.lambda$wrap$4(CpsVmExecutorService.java:140), java.base/java.util.concurrent.FutureTask.run(Unknown Source), hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:139), jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28), jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68), jenkins.util.ErrorLoggingExecutorService.lambda$wrap$0(ErrorLoggingExecutorService.java:51), java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source), java.base/java.util.concurrent.FutureTask.run(Unknown Source), java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source), java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source), PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$1.call(CpsVmExecutorService.java:53), PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$1.call(CpsVmExecutorService.java:50), org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:136), org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:275), PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService.lambda$categoryThreadFactory$0(CpsVmExecutorService.java:50), java.base/java.lang.Thread.run(Unknown Source)] [Pipeline] } [Pipeline] { [Pipeline] { [Pipeline] { [Pipeline] { [Pipeline] { [Pipeline] { [Pipeline] { [Pipeline] // stage [Pipeline] } [2026-08-26T07:03:51.980Z] Failed in branch cleanup-worker-ut-sandbox [Pipeline] echo [2026-08-26T07:03:52.012Z] JOB package-deployers-rocky9: waiting for dependecy [configure-rocky9:NOT_BUILT] [Pipeline] } [Pipeline] echo [2026-08-26T07:03:52.032Z] JOB compile-rocky9: waiting for dependecy [package-deployers-rocky9:null] [Pipeline] echo [2026-08-26T07:03:52.045Z] JOB compile-rocky9: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:03:52.169Z] JOB package-rocky9: waiting for dependecy [compile-rocky9:null] [Pipeline] echo [2026-08-26T07:03:52.189Z] JOB package-rocky9: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:03:52.212Z] JOB freeze-rocky9: waiting for dependecy [package-rocky9:null] [Pipeline] echo [2026-08-26T07:03:52.235Z] JOB freeze-rocky9: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:03:52.259Z] JOB cleanup-rocky9: waiting for dependecy [freeze-rocky9:null] [Pipeline] echo [2026-08-26T07:03:52.277Z] JOB cleanup-rocky9: waiting for all = false [Pipeline] } [Pipeline] echo [2026-08-26T07:03:52.336Z] JOB deploy-tf-ansible-os: waiting for dependecy [package-rocky9:null, deploy-platform-ansible-os:null] [Pipeline] echo [2026-08-26T07:03:52.359Z] JOB deploy-tf-ansible-os: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] echo [2026-08-26T07:03:52.423Z] JOB collect-logs-and-cleanup-ansible-os: waiting for dependecy [deploy-platform-ansible-os:null, deploy-tf-ansible-os:null, test-sanity-ansible-os:null] [Pipeline] echo [2026-08-26T07:03:52.475Z] JOB collect-logs-and-cleanup-ansible-os: waiting for all = false [Pipeline] } [Pipeline] // parallel [Pipeline] } [2026-08-26T07:03:53.119Z] Failed in branch stream-prepare-ut-sandbox [Pipeline] // waitUntil [2026-08-26T07:03:53.530Z] Will try again after 15 sec [2026-08-26T07:03:53.717Z] Will try again after 15 sec [2026-08-26T07:03:53.830Z] Will try again after 15 sec [2026-08-26T07:03:53.872Z] Will try again after 15 sec [Pipeline] echo [2026-08-26T07:03:53.892Z] JOB package-deployers-rocky9: wait finished. overall result = false [Pipeline] } [2026-08-26T07:03:54.175Z] Will try again after 15 sec [2026-08-26T07:03:54.599Z] Will try again after 15 sec [Pipeline] // stage [Pipeline] } [Pipeline] { [Pipeline] echo [2026-08-26T07:04:01.668Z] JOB deploy-platform-ansible-os: waiting for dependecy [package-rocky9:null] [Pipeline] echo [2026-08-26T07:04:01.689Z] JOB deploy-platform-ansible-os: fails were not found, unfinished jobs are still present [Pipeline] } [2026-08-26T07:04:01.816Z] Will try again after 15 sec [Pipeline] { [Pipeline] echo [2026-08-26T07:04:03.418Z] JOB test-sanity-ansible-os: waiting for dependecy [deploy-tf-ansible-os:null] [Pipeline] echo [2026-08-26T07:04:03.451Z] JOB test-sanity-ansible-os: fails were not found, unfinished jobs are still present [Pipeline] } [2026-08-26T07:04:03.650Z] Will try again after 15 sec [Pipeline] { [Pipeline] echo [2026-08-26T07:04:08.626Z] JOB compile-rocky9: waiting for dependecy [package-deployers-rocky9:NOT_BUILT] [Pipeline] } [Pipeline] // waitUntil [Pipeline] { [Pipeline] { [Pipeline] echo [2026-08-26T07:04:08.970Z] JOB compile-rocky9: wait finished. overall result = false [Pipeline] } [Pipeline] { [Pipeline] echo [2026-08-26T07:04:09.112Z] JOB package-rocky9: waiting for dependecy [compile-rocky9:NOT_BUILT] [Pipeline] } [Pipeline] echo [2026-08-26T07:04:09.347Z] JOB freeze-rocky9: waiting for dependecy [package-rocky9:null] [Pipeline] echo [2026-08-26T07:04:12.204Z] JOB freeze-rocky9: fails were not found, unfinished jobs are still present [Pipeline] } [Pipeline] // stage [Pipeline] } [Pipeline] echo [2026-08-26T07:04:12.468Z] JOB cleanup-rocky9: waiting for dependecy [freeze-rocky9:null] [Pipeline] echo [2026-08-26T07:04:12.497Z] JOB cleanup-rocky9: waiting for all = false [Pipeline] } [Pipeline] // waitUntil [Pipeline] { [Pipeline] { [2026-08-26T07:04:12.713Z] Will try again after 15 sec [Pipeline] echo [2026-08-26T07:04:12.732Z] JOB package-rocky9: wait finished. overall result = false [Pipeline] } [2026-08-26T07:04:13.048Z] Will try again after 15 sec [Pipeline] echo [2026-08-26T07:04:13.272Z] JOB deploy-tf-ansible-os: waiting for dependecy [package-rocky9:NOT_BUILT, deploy-platform-ansible-os:null] [Pipeline] } [Pipeline] echo [2026-08-26T07:04:13.343Z] JOB collect-logs-and-cleanup-ansible-os: waiting for dependecy [deploy-platform-ansible-os:null, deploy-tf-ansible-os:null, test-sanity-ansible-os:null] [Pipeline] echo [2026-08-26T07:04:13.370Z] JOB collect-logs-and-cleanup-ansible-os: waiting for all = false [Pipeline] } [Pipeline] // stage [Pipeline] } [Pipeline] // waitUntil [Pipeline] echo [2026-08-26T07:04:14.352Z] JOB deploy-tf-ansible-os: wait finished. overall result = false [Pipeline] } [2026-08-26T07:04:14.430Z] Will try again after 15 sec [Pipeline] // stage [Pipeline] } [Pipeline] { [Pipeline] echo [2026-08-26T07:04:16.884Z] JOB deploy-platform-ansible-os: waiting for dependecy [package-rocky9:NOT_BUILT] [Pipeline] } [Pipeline] // waitUntil [Pipeline] echo [2026-08-26T07:04:17.124Z] JOB deploy-platform-ansible-os: wait finished. overall result = false [Pipeline] } [Pipeline] // stage [Pipeline] } [Pipeline] { [Pipeline] echo [2026-08-26T07:04:18.708Z] JOB test-sanity-ansible-os: waiting for dependecy [deploy-tf-ansible-os:NOT_BUILT] [Pipeline] } [Pipeline] // waitUntil [Pipeline] echo [2026-08-26T07:04:18.839Z] JOB test-sanity-ansible-os: wait finished. overall result = false [Pipeline] } [Pipeline] // stage [Pipeline] } [Pipeline] { [Pipeline] { [Pipeline] { [Pipeline] echo [2026-08-26T07:04:30.095Z] JOB freeze-rocky9: waiting for dependecy [package-rocky9:NOT_BUILT] [Pipeline] } [Pipeline] echo [2026-08-26T07:04:30.127Z] JOB cleanup-rocky9: waiting for dependecy [freeze-rocky9:null] [Pipeline] echo [2026-08-26T07:04:30.150Z] JOB cleanup-rocky9: waiting for all = false [Pipeline] } [Pipeline] // waitUntil [2026-08-26T07:04:30.482Z] Will try again after 15 sec [Pipeline] echo [2026-08-26T07:04:30.514Z] JOB freeze-rocky9: wait finished. overall result = false [Pipeline] } [Pipeline] echo [2026-08-26T07:04:30.536Z] JOB collect-logs-and-cleanup-ansible-os: waiting for dependecy [deploy-platform-ansible-os:NOT_BUILT, deploy-tf-ansible-os:NOT_BUILT, test-sanity-ansible-os:NOT_BUILT] [Pipeline] echo [2026-08-26T07:04:30.554Z] JOB collect-logs-and-cleanup-ansible-os: waiting for all = true [Pipeline] } [Pipeline] // stage [Pipeline] } [Pipeline] // waitUntil [Pipeline] echo [2026-08-26T07:04:31.015Z] JOB collect-logs-and-cleanup-ansible-os: wait finished. overall result = true [Pipeline] echo [2026-08-26T07:04:31.120Z] JOB collect-logs-and-cleanup-ansible-os: entering run_job [Pipeline] writeFile [Pipeline] archiveArtifacts [2026-08-26T07:04:31.379Z] Archiving artifacts [Pipeline] echo [2026-08-26T07:04:33.085Z] JOB collect-logs-and-cleanup-ansible-os / stream ansible-os : deps: [deploy-platform-ansible-os, deploy-tf-ansible-os, test-sanity-ansible-os] [Pipeline] dir [2026-08-26T07:04:33.236Z] Running in /home/jenkins/workspace/pipeline-check@2 [Pipeline] { [Pipeline] findFiles [Pipeline] echo [2026-08-26T07:04:33.651Z] JOB collect-logs-and-cleanup-ansible-os: files found = 0 [Pipeline] } [Pipeline] // dir [Pipeline] dir [2026-08-26T07:04:33.842Z] Running in /home/jenkins/workspace/pipeline-check@2 [Pipeline] { [Pipeline] findFiles [Pipeline] echo [2026-08-26T07:04:34.085Z] JOB collect-logs-and-cleanup-ansible-os: files found = 0 [Pipeline] } [Pipeline] // dir [Pipeline] dir [2026-08-26T07:04:34.447Z] Running in /home/jenkins/workspace/pipeline-check@2 [Pipeline] { [Pipeline] findFiles [Pipeline] echo [2026-08-26T07:04:34.653Z] JOB collect-logs-and-cleanup-ansible-os: files found = 0 [Pipeline] } [Pipeline] // dir [Pipeline] echo [2026-08-26T07:04:35.001Z] JOB collect-logs-and-cleanup-ansible-os: content of deps file is empty [Pipeline] echo [2026-08-26T07:04:35.059Z] JOB collect-logs-and-cleanup-ansible-os: Starting job: collect-logs-and-cleanup rnd: #41721 [Pipeline] timeout [2026-08-26T07:04:35.095Z] Timeout set to expire in 3 hr 0 min [Pipeline] { [Pipeline] build [2026-08-26T07:04:35.361Z] Scheduling project: collect-logs-and-cleanup [Pipeline] { [2026-08-26T07:04:46.501Z] Starting building: collect-logs-and-cleanup #12124 [Pipeline] echo [2026-08-26T07:04:46.675Z] JOB cleanup-rocky9: waiting for dependecy [freeze-rocky9:NOT_BUILT] [Pipeline] echo [2026-08-26T07:04:46.703Z] JOB cleanup-rocky9: waiting for all = true [Pipeline] } [Pipeline] // waitUntil [Pipeline] echo [2026-08-26T07:04:46.936Z] JOB cleanup-rocky9: wait finished. overall result = true [Pipeline] echo [2026-08-26T07:04:47.007Z] JOB cleanup-rocky9: entering run_job [Pipeline] writeFile [Pipeline] archiveArtifacts [2026-08-26T07:04:47.336Z] Archiving artifacts [Pipeline] echo [2026-08-26T07:04:47.440Z] JOB cleanup-rocky9 / stream build-rocky9 : deps: [freeze-rocky9] [Pipeline] dir [2026-08-26T07:04:47.687Z] Running in /home/jenkins/workspace/pipeline-check@2 [Pipeline] { [Pipeline] findFiles [Pipeline] echo [2026-08-26T07:04:48.231Z] JOB cleanup-rocky9: files found = 0 [Pipeline] } [Pipeline] // dir [Pipeline] echo [2026-08-26T07:04:48.974Z] JOB cleanup-rocky9: content of deps file is empty [Pipeline] echo [2026-08-26T07:04:49.076Z] JOB cleanup-rocky9: Starting job: build-cleanup rnd: #72939 [Pipeline] timeout [2026-08-26T07:04:49.163Z] Timeout set to expire in 3 hr 0 min [Pipeline] { [Pipeline] build [2026-08-26T07:04:49.328Z] Scheduling project: build-cleanup [2026-08-26T07:04:56.583Z] Starting building: build-cleanup #14144 [2026-08-26T07:05:13.684Z] Build collect-logs-and-cleanup #12124 completed: SUCCESS [Pipeline] echo [2026-08-26T07:05:13.748Z] JOB collect-logs-and-cleanup-ansible-os: Finished with SUCCESS [Pipeline] } [Pipeline] // timeout [Pipeline] copyArtifacts [2026-08-26T07:05:14.184Z] Copied 0 artifacts from "collect-logs-and-cleanup" build number 12124 [Pipeline] echo [2026-08-26T07:05:14.304Z] JOB collect-logs-and-cleanup-ansible-os: Collected artifacts: [Pipeline] sh [2026-08-26T07:05:14.773Z] Build build-cleanup #14144 completed: SUCCESS [Pipeline] echo [2026-08-26T07:05:15.261Z] JOB cleanup-rocky9: Finished with SUCCESS [Pipeline] } [2026-08-26T07:05:15.606Z] + ls -la collect-logs-and-cleanup-41721 [2026-08-26T07:05:15.606Z] total 16 [2026-08-26T07:05:15.606Z] drwxr-xr-x 2 jenkins docker 4096 Aug 26 07:05 . [2026-08-26T07:05:15.606Z] drwxr-xr-x 4 jenkins docker 12288 Aug 26 07:05 .. [Pipeline] // timeout [Pipeline] copyArtifacts [2026-08-26T07:05:15.922Z] Copied 0 artifacts from "build-cleanup" build number 14144 [Pipeline] withCredentials [Pipeline] echo [2026-08-26T07:05:15.987Z] JOB cleanup-rocky9: Collected artifacts: [Pipeline] sh [2026-08-26T07:05:16.176Z] Masking supported pattern matches of $LOGS_HOST_SSH_KEY [Pipeline] { [Pipeline] sh [2026-08-26T07:05:16.483Z] + ls -la build-cleanup-72939 [2026-08-26T07:05:16.483Z] total 16 [2026-08-26T07:05:16.483Z] drwxr-xr-x 2 jenkins docker 4096 Aug 26 07:05 . [2026-08-26T07:05:16.483Z] drwxr-xr-x 5 jenkins docker 12288 Aug 26 07:05 .. [Pipeline] withCredentials [2026-08-26T07:05:17.792Z] Masking supported pattern matches of $LOGS_HOST_SSH_KEY [Pipeline] { [Pipeline] sh [2026-08-26T07:05:33.022Z] Warning: Permanently added 'nexus.gz1.opensdn.io,212.233.90.199' (ECDSA) to the list of known hosts. [2026-08-26T07:05:33.238Z] Warning: Permanently added 'nexus.gz1.opensdn.io,212.233.90.199' (ECDSA) to the list of known hosts. [2026-08-26T07:05:33.507Z] Warning: Permanently added 'nexus.gz1.opensdn.io,212.233.90.199' (ECDSA) to the list of known hosts. [2026-08-26T07:05:33.650Z] Warning: Permanently added 'nexus.gz1.opensdn.io,212.233.90.199' (ECDSA) to the list of known hosts. [Pipeline] sh [Pipeline] sh [Pipeline] } [Pipeline] // withCredentials [Pipeline] } [Pipeline] // stage [Pipeline] } [Pipeline] } [Pipeline] // parallel [Pipeline] // withCredentials [Pipeline] } [2026-08-26T07:05:36.331Z] Failed in branch stream-build-rocky9 [Pipeline] } [Pipeline] // stage [Pipeline] } [Pipeline] // parallel [Pipeline] } [Pipeline] // parallel [Pipeline] echo [2026-08-26T07:05:38.040Z] ERROR: Failed to run Unable to create live FilePath for slave-openstack-gz1_1; slave-openstack-gz1_1 was marked offline: Connection was broken [Pipeline] echo [2026-08-26T07:05:38.093Z] Stacktrace - [PluginClassLoader for workflow-durable-task-step//org.jenkinsci.plugins.workflow.support.steps.ExecutorStepDynamicContext$FilePathTranslator.get(ExecutorStepDynamicContext.java:188), PluginClassLoader for workflow-durable-task-step//org.jenkinsci.plugins.workflow.support.steps.ExecutorStepDynamicContext$FilePathTranslator.get(ExecutorStepDynamicContext.java:160), PluginClassLoader for workflow-durable-task-step//org.jenkinsci.plugins.workflow.support.steps.ExecutorStepDynamicContext$Translator.get(ExecutorStepDynamicContext.java:153), PluginClassLoader for workflow-durable-task-step//org.jenkinsci.plugins.workflow.support.steps.ExecutorStepDynamicContext$FilePathTranslator.get(ExecutorStepDynamicContext.java:170), PluginClassLoader for workflow-durable-task-step//org.jenkinsci.plugins.workflow.support.steps.ExecutorStepDynamicContext$FilePathTranslator.get(ExecutorStepDynamicContext.java:160), PluginClassLoader for workflow-step-api//org.jenkinsci.plugins.workflow.steps.DynamicContext$Typed.get(DynamicContext.java:95), PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.ContextVariableSet.get(ContextVariableSet.java:139), PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsThread.getContextVariable(CpsThread.java:135), PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsStepContext.doGet(CpsStepContext.java:297), PluginClassLoader for workflow-support//org.jenkinsci.plugins.workflow.support.DefaultStepContext.get(DefaultStepContext.java:75), PluginClassLoader for workflow-basic-steps//org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:83), PluginClassLoader for workflow-basic-steps//org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:71), PluginClassLoader for workflow-step-api//org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47), java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source), java.base/java.util.concurrent.FutureTask.run(Unknown Source), java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source), java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source), java.base/java.lang.Thread.run(Unknown Source)] [Pipeline] echo [2026-08-26T07:05:38.164Z] Jobs results: [test-unit-agent:[job-rnd:99295, number:-1, duration:0, result:NOT_BUILT], test-unit-opserver:[job-rnd:26307, number:-1, duration:0, result:NOT_BUILT], test-unit-bgp:[job-rnd:71971, number:-1, duration:0, result:NOT_BUILT], test-unit-group-one:[job-rnd:97784, number:-1, duration:0, result:NOT_BUILT], test-unit-group-two:[job-rnd:81792, number:-1, duration:0, result:NOT_BUILT], test-unit-ungrouped:[job-rnd:4182, number:-1, duration:0, result:NOT_BUILT], test-code-coverage:[job-rnd:83676, number:-1, duration:0, result:NOT_BUILT], prepare-sandbox-ut-sandbox:[job-rnd:21070, result:FAILURE, number:15886, started:1787727781652, duration:5802], fetch-sources-ut-sandbox:[job-rnd:39215, number:-1, duration:0, result:NOT_BUILT], configure-ut-sandbox:[job-rnd:6549, number:-1, duration:0, result:NOT_BUILT], cleanup-worker-ut-sandbox:[job-rnd:63114, result:FAILURE], deploy-platform-ansible-os:[job-rnd:47023, number:-1, duration:0, result:NOT_BUILT], deploy-tf-ansible-os:[job-rnd:13813, number:-1, duration:0, result:NOT_BUILT], test-sanity-ansible-os:[job-rnd:56751, number:-1, duration:0, result:NOT_BUILT], collect-logs-and-cleanup-ansible-os:[job-rnd:41721, number:12124, started:1787727886484, duration:16490, result:SUCCESS], prepare-sandbox-rocky9:[job-rnd:78533, result:FAILURE, number:15885, started:1787727776484, duration:10982], fetch-sources-rocky9:[job-rnd:74646, number:-1, duration:0, result:NOT_BUILT], configure-rocky9:[job-rnd:39515, number:-1, duration:0, result:NOT_BUILT], package-deployers-rocky9:[job-rnd:97939, number:-1, duration:0, result:NOT_BUILT], compile-rocky9:[job-rnd:88918, number:-1, duration:0, result:NOT_BUILT], package-rocky9:[job-rnd:12794, number:-1, duration:0, result:NOT_BUILT], freeze-rocky9:[job-rnd:1548, number:-1, duration:0, result:NOT_BUILT], cleanup-rocky9:[job-rnd:72939, number:14144, started:1787727896582, duration:7059, result:SUCCESS]] [Pipeline] stage [Pipeline] { (gerrit vote) [Pipeline] echo [2026-08-26T07:05:38.982Z] Evaluated results for test-unit = [results:[NOT_BUILT, NOT_BUILT, NOT_BUILT, NOT_BUILT, NOT_BUILT, NOT_BUILT, NOT_BUILT], duration:0] [Pipeline] echo [2026-08-26T07:05:39.140Z] Evaluated results for prepare-ut-sandbox = [results:[FAILURE, NOT_BUILT, NOT_BUILT, FAILURE], duration:5802] [Pipeline] echo [2026-08-26T07:05:39.191Z] Evaluated results for ansible-os = [results:[NOT_BUILT, NOT_BUILT, NOT_BUILT, SUCCESS], duration:16490] [Pipeline] echo [2026-08-26T07:05:39.303Z] Evaluated results for build-rocky9 = [results:[FAILURE, NOT_BUILT, NOT_BUILT, NOT_BUILT, NOT_BUILT, NOT_BUILT, NOT_BUILT, SUCCESS], duration:18041] [Pipeline] echo [2026-08-26T07:05:39.338Z] Notify gerrit verified=-1, submit=false, msg= [2026-08-26T07:05:39.338Z] TF CI Build Failed (check) in 0h 3m 44s [2026-08-26T07:05:39.338Z] [2026-08-26T07:05:39.338Z] Failed checks: [2026-08-26T07:05:39.338Z] - test-unit : NOT_BUILT [2026-08-26T07:05:39.338Z] - prepare-ut-sandbox http://nexus.gz1.opensdn.io:8082/jenkins_logs/gerrit/34/3034/3/check_4729/prepare-ut-sandbox : FAILURE in 0h 0m 5s [2026-08-26T07:05:39.338Z] - ansible-os http://nexus.gz1.opensdn.io:8082/jenkins_logs/gerrit/34/3034/3/check_4729/ansible-os : FAILURE in 0h 0m 16s [2026-08-26T07:05:39.338Z] - build-rocky9 http://nexus.gz1.opensdn.io:8082/jenkins_logs/gerrit/34/3034/3/check_4729/build-rocky9 : FAILURE in 0h 0m 18s [2026-08-26T07:05:39.338Z] [2026-08-26T07:05:39.338Z] Succeeded checks: [Pipeline] withCredentials [2026-08-26T07:05:39.442Z] Masking supported pattern matches of $GERRIT_API_PASSWORD [Pipeline] { [Pipeline] echo [2026-08-26T07:05:39.613Z] INFO: resolved gerrit URL is https://gerrit.opensdn.io/ [Pipeline] sh [2026-08-26T07:05:40.240Z] + /home/jenkins/workspace/pipeline-check@2/src/opensdn-io/tf-jenkins/infra/gerrit/notify.py --gerrit https://gerrit.opensdn.io/ --user tf-gerrit --password **** --review I48b6aa7a1c5904c6b68afa3ae8ec4e7d367c3615 --patchset 3 --branch master --labels Verified=-1 --message 'TF CI Build Failed (check) in 0h 3m 44s [2026-08-26T07:05:40.240Z] [2026-08-26T07:05:40.240Z] Failed checks: [2026-08-26T07:05:40.240Z] - test-unit : NOT_BUILT [2026-08-26T07:05:40.240Z] - prepare-ut-sandbox http://nexus.gz1.opensdn.io:8082/jenkins_logs/gerrit/34/3034/3/check_4729/prepare-ut-sandbox : FAILURE in 0h 0m 5s [2026-08-26T07:05:40.240Z] - ansible-os http://nexus.gz1.opensdn.io:8082/jenkins_logs/gerrit/34/3034/3/check_4729/ansible-os : FAILURE in 0h 0m 16s [2026-08-26T07:05:40.241Z] - build-rocky9 http://nexus.gz1.opensdn.io:8082/jenkins_logs/gerrit/34/3034/3/check_4729/build-rocky9 : FAILURE in 0h 0m 18s [2026-08-26T07:05:40.241Z] [2026-08-26T07:05:40.241Z] Succeeded checks:' [2026-08-26T07:05:40.241Z] DEBUG: Change: {'id': 'opensdn-io%2Ftf-controller~master~I48b6aa7a1c5904c6b68afa3ae8ec4e7d367c3615', 'project': 'opensdn-io/tf-controller', 'branch': 'master', 'topic': 'route-target-lifecycle', 'attention_set': {'1000021': {'account': {'_account_id': 1000021}, 'last_update': '2026-08-26 04:01:56.000000000', 'reason': 'Someone else replied on the change'}}, 'removed_from_attention_set': {'1000003': {'account': {'_account_id': 1000003}, 'last_update': '2026-08-26 07:01:32.000000000', 'reason': ' replied on the change', 'reason_account': {'_account_id': 1000003}}, '1000001': {'account': {'_account_id': 1000001}, 'last_update': '2026-08-20 08:23:10.000000000', 'reason': 'removed on reply'}}, 'hashtags': [], 'change_id': 'I48b6aa7a1c5904c6b68afa3ae8ec4e7d367c3615', 'subject': 'ST: always release the logical-router route-target ID on delete', 'status': 'NEW', 'created': '2026-08-06 06:10:38.000000000', 'updated': '2026-08-26 07:02:19.000000000', 'submit_type': 'MERGE_IF_NECESSARY', 'insertions': 38, 'deletions': 6, 'total_comment_count': 9, 'unresolved_comment_count': 1, 'has_review_started': True, 'meta_rev_id': '81713c4e6c0be7d7778e0c4806c65caf1d6aa8b1', '_number': 3034, 'owner': {'_account_id': 1000021}, 'labels': {'Code-Review': {'all': [{'value': 0, 'permitted_voting_range': {'min': -2, 'max': 2}, '_account_id': 1000003}, {'value': 0, 'permitted_voting_range': {'min': -2, 'max': 2}, '_account_id': 1000001}], 'values': {'-2': 'This shall not be submitted', '-1': 'I would prefer this is not submitted as is', ' 0': 'No score', '+1': 'Looks good to me, but someone else must approve', '+2': 'Looks good to me, approved'}, 'description': '', 'default_value': 0}, 'Verified': {'all': [{'value': 0, 'permitted_voting_range': {'min': -2, 'max': 2}, '_account_id': 1000003}, {'value': 0, 'date': '2026-08-26 07:02:19.000000000', 'permitted_voting_range': {'min': -2, 'max': 2}, '_account_id': 1000001}], 'values': {'-2': 'Fails', '-1': 'Fails', ' 0': 'No score', '+1': 'Verified', '+2': 'Gating'}, 'description': '', 'default_value': 0}}, 'permitted_labels': {'Code-Review': ['-2', '-1', ' 0', '+1', '+2'], 'Verified': ['-2', '-1', ' 0', '+1', '+2']}, 'removable_labels': {'Verified': {' 0': [{'_account_id': 1000001}]}}, 'removable_reviewers': [{'_account_id': 1000003}, {'_account_id': 1000002}, {'_account_id': 1000001}], 'reviewers': {'REVIEWER': [{'_account_id': 1000001}, {'_account_id': 1000003}], 'CC': [{'_account_id': 1000002}]}, 'pending_reviewers': {}, 'current_revision': 'bf640c0e985af7951b386684af01e66c5de7e548', 'revisions': {'bf640c0e985af7951b386684af01e66c5de7e548': {'kind': 'REWORK', '_number': 3, 'created': '2026-08-20 10:54:43.000000000', 'uploader': {'_account_id': 1000021}, 'ref': 'refs/changes/34/3034/3', 'fetch': {'ssh': {'url': 'ssh://tf-gerrit@gerrit.opensdn.io:29418/opensdn-io/tf-controller', 'ref': 'refs/changes/34/3034/3'}, 'anonymous http': {'url': 'https://gerrit.opensdn.io/opensdn-io/tf-controller', 'ref': 'refs/changes/34/3034/3'}, 'http': {'url': 'https://tf-gerrit@gerrit.opensdn.io/a/opensdn-io/tf-controller', 'ref': 'refs/changes/34/3034/3'}}, 'commit': {'parents': [{'commit': '9112407ed0a26aaec3ee49b7513d267bf7e93597', 'subject': 'Switching to oneTBB: old includes removal'}], 'author': {'name': 'Charnsilp Chinprasert(Nipa)', 'email': 'justice.form@gmail.com', 'date': '2026-08-06 06:05:31.000000000', 'tz': 420}, 'committer': {'name': 'Charnsilp Chinprasert(Nipa)', 'email': 'justice.form@gmail.com', 'date': '2026-08-20 10:54:21.000000000', 'tz': 420}, 'subject': 'ST: always release the logical-router route-target ID on delete', 'message': "ST: always release the logical-router route-target ID on delete\n\nLogicalRouterST.delete_obj() runs five steps before it releases the\nroute-target ID, and every one of them is an API or datastore call that can\nraise:\n\n update_multiple_refs('virtual_machine_interface', {})\n update_multiple_refs('route_table', {})\n update_multiple_refs('bgpvpn', {})\n update_virtual_networks()\n delete_route_targets([self.route_target])\n asn = ...get_autonomous_system()\n self._object_db.free_route_target(self.name, asn)\n\nIf any of them raises, free_route_target() never runs and the ID stays\nallocated. Nothing recovers it: the logical router is gone, so delete_obj()\nis never called again, and a Zookeeper index with no row in route_target_table\nhas no reader - alloc_route_target() starts from the row, so an index without\none is never examined.\n\nA production config DB shows six route_target_table bindings naming logical\nrouters that no longer exist. Three of them still have their route-target\nobject and three do not, which places the failure between delete_route_targets()\nand free_route_target() in the first case and at or before\ndelete_route_targets() in the second.\n\nRead the ASN before the steps that can raise, and release the ID in a finally\nblock, so it is released on every path. The ASN lookup is deliberately outside\nthe try: if it stayed inside and raised - which is one of the two things that\nexplains the three-case split above - the finally clause would have no asn to\nuse.\n\nEach teardown step is attempted independently rather than under one try, so a\nfailure in an early step does not skip the later ones and the log names the\nstep that actually failed. Raised in review on patchset 1.\n\nThe per-step handling is about diagnosis rather than damage, and it is worth\nbeing precise about why. None of the five steps leaves unrecoverable state on\nits own: RouteTargetST.reinit() already deletes route-target objects that have\nno routing-instance or logical-router back-refs, and walks _RT_CF freeing any\nrow whose object is gone (resources/route_target.py:20-38); the ref sets are\nrebuilt from the DB at the same reinit. The ID release is the one step with no\nsuch collector - alloc_route_target() starts from the _RT_CF row, so a\nZookeeper index whose row is gone is never examined again - which is why it,\nand only it, sits in finally.\n\nOne consequence to note: update_virtual_networks() reads the ref sets that the\nearlier steps clear, so if an earlier step fails it works from stale membership\nand may leave a route target on a VN. That is now logged instead of being\nhidden behind an abandoned try, and reinit recomputes it. It does not justify\nskipping the steps after it.\n\nThe catch is deliberately broad. The goal is that the release below always\nruns, and narrowing it to the vnc_api exception family would let anything\nunexpected skip the remaining steps again. If the project would rather see it\nnarrowed, say so and I will.\n\nThis does not make delete_obj() idempotent, and it does not help if\nfree_route_target() itself raises partway - that is handled separately.\n\nTested on this patchset: tox -e pep8 passes clean. tox -e py39 runs 83 tests\nwith 16 skipped and one failure, test_security_group.test_stale_acl, which\nfails the same way on an unpatched tree - it hashes a dict in\n_access_control_list_update() and is unrelated to this change.\n\nChange-Id: I48b6aa7a1c5904c6b68afa3ae8ec4e7d367c3615\nSigned-off-by: Charnsilp Chinprasert(Nipa) \n"}}}, 'requirements': [], 'submit_records': [{'rule_name': 'gerrit~DefaultSubmitRule', 'status': 'NOT_READY', 'labels': [{'label': 'Code-Review', 'status': 'NEED'}, {'label': 'Verified', 'status': 'NEED'}]}]} [2026-08-26T07:05:40.242Z] DEBUG: push message data: {'labels': {'Verified': '-1'}, 'message': 'TF CI Build Failed (check) in 0h 3m 44s\n\nFailed checks:\n- test-unit : NOT_BUILT\n- prepare-ut-sandbox http://nexus.gz1.opensdn.io:8082/jenkins_logs/gerrit/34/3034/3/check_4729/prepare-ut-sandbox : FAILURE in 0h 0m 5s\n- ansible-os http://nexus.gz1.opensdn.io:8082/jenkins_logs/gerrit/34/3034/3/check_4729/ansible-os : FAILURE in 0h 0m 16s\n- build-rocky9 http://nexus.gz1.opensdn.io:8082/jenkins_logs/gerrit/34/3034/3/check_4729/build-rocky9 : FAILURE in 0h 0m 18s\n\nSucceeded checks:'} [Pipeline] } [Pipeline] // withCredentials [Pipeline] sh [Pipeline] archiveArtifacts [2026-08-26T07:05:44.790Z] Archiving artifacts [Pipeline] withCredentials [2026-08-26T07:05:45.311Z] Masking supported pattern matches of $LOGS_HOST_SSH_KEY [Pipeline] { [Pipeline] writeFile [Pipeline] sh [2026-08-26T07:05:45.795Z] Warning: Permanently added 'nexus.gz1.opensdn.io,212.233.90.199' (ECDSA) to the list of known hosts. [2026-08-26T07:05:46.405Z] Warning: Permanently added 'nexus.gz1.opensdn.io,212.233.90.199' (ECDSA) to the list of known hosts. [Pipeline] } [Pipeline] // withCredentials [Pipeline] } [Pipeline] // stage [Pipeline] parallel [Pipeline] { (Branch: job-cleanup-workers) [Pipeline] stage [Pipeline] { (cleanup-workers) [Pipeline] echo [2026-08-26T07:05:48.045Z] JOB cleanup-workers: entering run_job [Pipeline] echo [2026-08-26T07:05:48.105Z] JOB cleanup-workers: vars empty. do not store vars file. [Pipeline] echo [2026-08-26T07:05:48.336Z] JOB cleanup-workers: Starting job: cleanup-workers rnd: #50485 [Pipeline] timeout [2026-08-26T07:05:48.383Z] Timeout set to expire in 3 hr 0 min [Pipeline] { [Pipeline] build [2026-08-26T07:05:48.521Z] Scheduling project: cleanup-workers [2026-08-26T07:06:00.344Z] Starting building: cleanup-workers #7410 [2026-08-26T07:06:27.816Z] Build cleanup-workers #7410 completed: FAILURE [Pipeline] } [Pipeline] // timeout [Pipeline] echo [2026-08-26T07:06:28.301Z] JOB cleanup-workers: Failed [Pipeline] copyArtifacts [Pipeline] echo [2026-08-26T07:06:28.936Z] JOB cleanup-workers: Collected artifacts: [Pipeline] sh [2026-08-26T07:06:29.425Z] + ls -la cleanup-workers-50485 [2026-08-26T07:06:29.425Z] ls: cannot access 'cleanup-workers-50485': No such file or directory [2026-08-26T07:06:29.425Z] + /bin/true [Pipeline] withCredentials [2026-08-26T07:06:29.775Z] Masking supported pattern matches of $LOGS_HOST_SSH_KEY [Pipeline] { [Pipeline] sh [2026-08-26T07:06:32.988Z] Warning: Permanently added 'nexus.gz1.opensdn.io,212.233.90.199' (ECDSA) to the list of known hosts. [2026-08-26T07:06:34.914Z] Warning: Permanently added 'nexus.gz1.opensdn.io,212.233.90.199' (ECDSA) to the list of known hosts. [Pipeline] sh [Pipeline] } [Pipeline] // withCredentials [Pipeline] echo [2026-08-26T07:06:36.471Z] JOB cleanup-workers: error in job!!! [Pipeline] echo [2026-08-26T07:06:36.499Z] JOB cleanup-workers: Err - org.jenkinsci.plugins.workflow.steps.FlowInterruptedException [Pipeline] echo [2026-08-26T07:06:36.515Z] JOB cleanup-workers: Message - null [Pipeline] echo [2026-08-26T07:06:36.528Z] JOB cleanup-workers: Cause - null [Pipeline] echo [2026-08-26T07:06:36.546Z] JOB cleanup-workers: Stacktrace - [PluginClassLoader for pipeline-build-step//org.jenkinsci.plugins.workflow.support.steps.build.BuildTriggerListener.onFinalized(BuildTriggerListener.java:76), hudson.model.listeners.RunListener.lambda$fireFinalized$3(RunListener.java:260), jenkins.util.Listeners.lambda$notify$0(Listeners.java:59), jenkins.util.Listeners.notify(Listeners.java:67), hudson.model.listeners.RunListener.fireFinalized(RunListener.java:258), hudson.model.Run.onEndBuilding(Run.java:2052), hudson.model.Run.execute(Run.java:1959), hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:44), hudson.model.ResourceController.execute(ResourceController.java:101), hudson.model.Executor.run(Executor.java:446)] [Pipeline] } [Pipeline] // stage [Pipeline] } [2026-08-26T07:06:36.662Z] Failed in branch job-cleanup-workers [Pipeline] // parallel [Pipeline] echo [2026-08-26T07:06:36.971Z] URL of console output = https://jenkins.opensdn.io/job/pipeline-check/4729/consoleText [Pipeline] withCredentials [2026-08-26T07:06:37.052Z] Masking supported pattern matches of $LOGS_HOST_SSH_KEY [Pipeline] { [Pipeline] sh