2025-10-20 05:03:56.701 [info] <0.8.0> Log file opened with Lager 2025-10-20 05:03:59.403 [info] <0.8.0> Feature flags: list of feature flags found: 2025-10-20 05:03:59.403 [info] <0.8.0> Feature flags: feature flag states written to disk: yes 2025-10-20 05:03:59.437 [info] <0.238.0> Starting RabbitMQ 3.7.28 on Erlang 21.3.8.21 Copyright (c) 2007-2020 Pivotal Software, Inc. Licensed under the MPL. See https://www.rabbitmq.com/ 2025-10-20 05:03:59.438 [info] <0.238.0> node : contrail@cn-jenkins-deploy-platform-ansible-os-4371-3 home dir : /var/lib/rabbitmq config file(s) : /etc/rabbitmq/rabbitmq.config cookie hash : esZsDxSN6VGbi9JkMSxNZA== log(s) : /var/log/contrail/config-database-rabbitmq/rabbitmq.log : /var/log/rabbitmq/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3_upgrade.log database dir : /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3 2025-10-20 05:03:59.450 [info] <0.238.0> Running boot step pre_boot defined by app rabbit 2025-10-20 05:03:59.450 [info] <0.238.0> Running boot step rabbit_core_metrics defined by app rabbit 2025-10-20 05:03:59.451 [info] <0.238.0> Running boot step rabbit_alarm defined by app rabbit 2025-10-20 05:03:59.455 [info] <0.244.0> Memory high watermark set to 12637 MiB (13251084288 bytes) of 15796 MiB (16563855360 bytes) total 2025-10-20 05:03:59.464 [info] <0.246.0> Enabling free disk space monitoring 2025-10-20 05:03:59.464 [info] <0.246.0> Disk free limit set to 50MB 2025-10-20 05:03:59.470 [info] <0.238.0> Running boot step code_server_cache defined by app rabbit 2025-10-20 05:03:59.470 [info] <0.238.0> Running boot step file_handle_cache defined by app rabbit 2025-10-20 05:03:59.470 [info] <0.249.0> Limiting to approx 65436 file handles (58890 sockets) 2025-10-20 05:03:59.470 [info] <0.250.0> FHC read buffering: OFF 2025-10-20 05:03:59.471 [info] <0.250.0> FHC write buffering: ON 2025-10-20 05:03:59.471 [info] <0.238.0> Running boot step worker_pool defined by app rabbit 2025-10-20 05:03:59.471 [info] <0.239.0> Will use 4 processes for default worker pool 2025-10-20 05:03:59.471 [info] <0.239.0> Starting worker pool 'worker_pool' with 4 processes in it 2025-10-20 05:03:59.471 [info] <0.238.0> Running boot step database defined by app rabbit 2025-10-20 05:03:59.471 [info] <0.238.0> Node database directory at /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3 is empty. Assuming we need to join an existing cluster or initialise from scratch... 2025-10-20 05:03:59.471 [info] <0.238.0> Configured peer discovery backend: rabbit_peer_discovery_classic_config 2025-10-20 05:03:59.471 [info] <0.238.0> Will try to lock with peer discovery backend rabbit_peer_discovery_classic_config 2025-10-20 05:03:59.471 [info] <0.238.0> Peer discovery backend does not support locking, falling back to randomized delay 2025-10-20 05:03:59.471 [info] <0.238.0> Peer discovery backend rabbit_peer_discovery_classic_config supports registration. 2025-10-20 05:03:59.471 [info] <0.238.0> Will wait for 58919 milliseconds before proceeding with registration... 2025-10-20 05:04:53.667 [info] <0.316.0> RabbitMQ hasn't finished starting yet. Waiting for startup to finish before stopping... 2025-10-20 05:04:58.392 [info] <0.238.0> All discovered existing cluster peers: contrail@cn-jenkins-deploy-platform-ansible-os-4371-1, contrail@cn-jenkins-deploy-platform-ansible-os-4371-2, contrail@cn-jenkins-deploy-platform-ansible-os-4371-3 2025-10-20 05:04:58.392 [info] <0.238.0> Peer nodes we can cluster with: contrail@cn-jenkins-deploy-platform-ansible-os-4371-1, contrail@cn-jenkins-deploy-platform-ansible-os-4371-2 2025-10-20 05:04:58.393 [warning] <0.238.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:04:58.394 [warning] <0.238.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:04:58.394 [error] <0.238.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 9 retries left... 2025-10-20 05:04:58.896 [warning] <0.238.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:04:58.896 [warning] <0.238.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:04:58.896 [error] <0.238.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 8 retries left... 2025-10-20 05:04:59.460 [warning] <0.238.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {error,mnesia_not_running} 2025-10-20 05:04:59.460 [warning] <0.238.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:04:59.461 [error] <0.238.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 7 retries left... 2025-10-20 05:04:59.966 [warning] <0.238.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {error,mnesia_not_running} 2025-10-20 05:04:59.967 [warning] <0.238.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:04:59.967 [error] <0.238.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 6 retries left... 2025-10-20 05:05:00.469 [warning] <0.238.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {error,mnesia_not_running} 2025-10-20 05:05:00.470 [warning] <0.238.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:05:00.470 [error] <0.238.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 5 retries left... 2025-10-20 05:05:00.983 [warning] <0.238.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {error,mnesia_not_running} 2025-10-20 05:05:00.984 [warning] <0.238.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:05:00.984 [error] <0.238.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 4 retries left... 2025-10-20 05:05:01.495 [warning] <0.238.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {error,mnesia_not_running} 2025-10-20 05:05:01.496 [warning] <0.238.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:05:01.496 [error] <0.238.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 3 retries left... 2025-10-20 05:05:02.001 [warning] <0.238.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {error,mnesia_not_running} 2025-10-20 05:05:02.002 [warning] <0.238.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:05:02.002 [error] <0.238.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 2 retries left... 2025-10-20 05:05:02.509 [warning] <0.238.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {error,mnesia_not_running} 2025-10-20 05:05:02.510 [warning] <0.238.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:05:02.510 [error] <0.238.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 1 retries left... 2025-10-20 05:05:03.013 [info] <0.238.0> Node 'contrail@cn-jenkins-deploy-platform-ansible-os-4371-1' selected for auto-clustering 2025-10-20 05:05:03.047 [info] <0.238.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-10-20 05:05:33.048 [warning] <0.238.0> Error while waiting for Mnesia tables: {timeout_waiting_for_tables,[rabbit_durable_exchange,rabbit_exchange,rabbit_exchange_serial,rabbit_runtime_parameters,rabbit_durable_queue,rabbit_queue,gm_group,mirrored_sup_childspec]} 2025-10-20 05:05:33.048 [info] <0.238.0> Waiting for Mnesia tables for 30000 ms, 8 retries left 2025-10-20 05:06:03.049 [warning] <0.238.0> Error while waiting for Mnesia tables: {timeout_waiting_for_tables,[rabbit_durable_exchange,rabbit_exchange,rabbit_exchange_serial,rabbit_runtime_parameters,rabbit_durable_queue,rabbit_queue,gm_group,mirrored_sup_childspec]} 2025-10-20 05:06:03.049 [info] <0.238.0> Waiting for Mnesia tables for 30000 ms, 7 retries left 2025-10-20 05:06:33.050 [warning] <0.238.0> Error while waiting for Mnesia tables: {timeout_waiting_for_tables,[rabbit_durable_exchange,rabbit_exchange,rabbit_exchange_serial,rabbit_runtime_parameters,rabbit_durable_queue,rabbit_queue,gm_group,mirrored_sup_childspec]} 2025-10-20 05:06:33.050 [info] <0.238.0> Waiting for Mnesia tables for 30000 ms, 6 retries left 2025-10-20 05:07:03.051 [warning] <0.238.0> Error while waiting for Mnesia tables: {timeout_waiting_for_tables,[rabbit_durable_exchange,rabbit_exchange,rabbit_exchange_serial,rabbit_runtime_parameters,rabbit_durable_queue,rabbit_queue,gm_group,mirrored_sup_childspec]} 2025-10-20 05:07:03.051 [info] <0.238.0> Waiting for Mnesia tables for 30000 ms, 5 retries left 2025-10-20 05:07:33.052 [warning] <0.238.0> Error while waiting for Mnesia tables: {timeout_waiting_for_tables,[rabbit_durable_exchange,rabbit_exchange,rabbit_exchange_serial,rabbit_runtime_parameters,rabbit_durable_queue,rabbit_queue,gm_group,mirrored_sup_childspec]} 2025-10-20 05:07:33.052 [info] <0.238.0> Waiting for Mnesia tables for 30000 ms, 4 retries left 2025-10-20 05:08:03.053 [warning] <0.238.0> Error while waiting for Mnesia tables: {timeout_waiting_for_tables,[rabbit_durable_exchange,rabbit_exchange,rabbit_exchange_serial,rabbit_runtime_parameters,rabbit_durable_queue,rabbit_queue,gm_group,mirrored_sup_childspec]} 2025-10-20 05:08:03.053 [info] <0.238.0> Waiting for Mnesia tables for 30000 ms, 3 retries left 2025-10-20 05:08:33.054 [warning] <0.238.0> Error while waiting for Mnesia tables: {timeout_waiting_for_tables,[rabbit_durable_exchange,rabbit_exchange,rabbit_exchange_serial,rabbit_runtime_parameters,rabbit_durable_queue,rabbit_queue,gm_group,mirrored_sup_childspec]} 2025-10-20 05:08:33.054 [info] <0.238.0> Waiting for Mnesia tables for 30000 ms, 2 retries left 2025-10-20 05:09:03.055 [warning] <0.238.0> Error while waiting for Mnesia tables: {timeout_waiting_for_tables,[rabbit_durable_exchange,rabbit_exchange,rabbit_exchange_serial,rabbit_runtime_parameters,rabbit_durable_queue,rabbit_queue,gm_group,mirrored_sup_childspec]} 2025-10-20 05:09:03.055 [info] <0.238.0> Waiting for Mnesia tables for 30000 ms, 1 retries left 2025-10-20 05:09:33.056 [warning] <0.238.0> Error while waiting for Mnesia tables: {timeout_waiting_for_tables,[rabbit_durable_exchange,rabbit_exchange,rabbit_exchange_serial,rabbit_runtime_parameters,rabbit_durable_queue,rabbit_queue,gm_group,mirrored_sup_childspec]} 2025-10-20 05:09:33.056 [info] <0.238.0> Waiting for Mnesia tables for 30000 ms, 0 retries left 2025-10-20 05:10:03.057 [error] <0.237.0> CRASH REPORT Process <0.237.0> with 0 neighbours exited with reason: {{timeout_waiting_for_tables,[rabbit_user,rabbit_user_permission,rabbit_topic_permission,rabbit_vhost,rabbit_listener,rabbit_durable_route,rabbit_semi_durable_route,rabbit_route,rabbit_reverse_route,rabbit_topic_trie_node,rabbit_topic_trie_edge,rabbit_topic_trie_binding,rabbit_durable_exchange,rabbit_exchange,rabbit_exchange_serial,rabbit_runtime_parameters,rabbit_durable_queue,rabbit_queue,gm_group,mirrored_sup_childspec]},{rabbit,start,[normal,[]]}} in application_master:init/4 line 138 2025-10-20 05:10:03.058 [info] <0.43.0> Application rabbit exited with reason: {{timeout_waiting_for_tables,[rabbit_user,rabbit_user_permission,rabbit_topic_permission,rabbit_vhost,rabbit_listener,rabbit_durable_route,rabbit_semi_durable_route,rabbit_route,rabbit_reverse_route,rabbit_topic_trie_node,rabbit_topic_trie_edge,rabbit_topic_trie_binding,rabbit_durable_exchange,rabbit_exchange,rabbit_exchange_serial,rabbit_runtime_parameters,rabbit_durable_queue,rabbit_queue,gm_group,mirrored_sup_childspec]},{rabbit,start,[normal,[]]}} 2025-10-20 05:10:34.531 [info] <0.8.0> Log file opened with Lager 2025-10-20 05:10:39.440 [info] <0.8.0> Feature flags: list of feature flags found: 2025-10-20 05:10:39.440 [info] <0.8.0> Feature flags: feature flag states written to disk: yes 2025-10-20 05:10:39.464 [info] <0.240.0> Starting RabbitMQ 3.7.28 on Erlang 21.3.8.21 Copyright (c) 2007-2020 Pivotal Software, Inc. Licensed under the MPL. See https://www.rabbitmq.com/ 2025-10-20 05:10:39.464 [info] <0.240.0> node : contrail@cn-jenkins-deploy-platform-ansible-os-4371-3 home dir : /var/lib/rabbitmq config file(s) : /etc/rabbitmq/rabbitmq.config cookie hash : esZsDxSN6VGbi9JkMSxNZA== log(s) : /var/log/contrail/config-database-rabbitmq/rabbitmq.log : /var/log/rabbitmq/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3_upgrade.log database dir : /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3 2025-10-20 05:10:39.480 [info] <0.240.0> Running boot step pre_boot defined by app rabbit 2025-10-20 05:10:39.480 [info] <0.240.0> Running boot step rabbit_core_metrics defined by app rabbit 2025-10-20 05:10:39.481 [info] <0.240.0> Running boot step rabbit_alarm defined by app rabbit 2025-10-20 05:10:39.488 [info] <0.246.0> Memory high watermark set to 12637 MiB (13251084288 bytes) of 15796 MiB (16563855360 bytes) total 2025-10-20 05:10:39.494 [info] <0.248.0> Enabling free disk space monitoring 2025-10-20 05:10:39.494 [info] <0.248.0> Disk free limit set to 50MB 2025-10-20 05:10:39.501 [info] <0.240.0> Running boot step code_server_cache defined by app rabbit 2025-10-20 05:10:39.501 [info] <0.240.0> Running boot step file_handle_cache defined by app rabbit 2025-10-20 05:10:39.501 [info] <0.251.0> Limiting to approx 65436 file handles (58890 sockets) 2025-10-20 05:10:39.502 [info] <0.252.0> FHC read buffering: OFF 2025-10-20 05:10:39.502 [info] <0.252.0> FHC write buffering: ON 2025-10-20 05:10:39.502 [info] <0.240.0> Running boot step worker_pool defined by app rabbit 2025-10-20 05:10:39.502 [info] <0.241.0> Will use 4 processes for default worker pool 2025-10-20 05:10:39.502 [info] <0.241.0> Starting worker pool 'worker_pool' with 4 processes in it 2025-10-20 05:10:39.502 [info] <0.240.0> Running boot step database defined by app rabbit 2025-10-20 05:10:39.503 [info] <0.240.0> Node database directory at /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3 is empty. Assuming we need to join an existing cluster or initialise from scratch... 2025-10-20 05:10:39.503 [info] <0.240.0> Configured peer discovery backend: rabbit_peer_discovery_classic_config 2025-10-20 05:10:39.504 [info] <0.240.0> Will try to lock with peer discovery backend rabbit_peer_discovery_classic_config 2025-10-20 05:10:39.504 [info] <0.240.0> Peer discovery backend does not support locking, falling back to randomized delay 2025-10-20 05:10:39.504 [info] <0.240.0> Peer discovery backend rabbit_peer_discovery_classic_config supports registration. 2025-10-20 05:10:39.504 [info] <0.240.0> Will wait for 26804 milliseconds before proceeding with registration... 2025-10-20 05:11:06.308 [info] <0.240.0> All discovered existing cluster peers: contrail@cn-jenkins-deploy-platform-ansible-os-4371-1, contrail@cn-jenkins-deploy-platform-ansible-os-4371-2, contrail@cn-jenkins-deploy-platform-ansible-os-4371-3 2025-10-20 05:11:06.309 [info] <0.240.0> Peer nodes we can cluster with: contrail@cn-jenkins-deploy-platform-ansible-os-4371-1, contrail@cn-jenkins-deploy-platform-ansible-os-4371-2 2025-10-20 05:11:06.530 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,{'EXIT',{{badmatch,undefined},[{rabbit_misc,version,0,[{file,"src/rabbit_misc.erl"},{line,1100}]},{rabbit_mnesia,node_info,0,[{file,"src/rabbit_mnesia.erl"},{line,516}]},{rpc,'-handle_call_call/6-fun-0-',5,[{file,"rpc.erl"},{line,197}]}]}}} 2025-10-20 05:11:06.553 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,tables_not_present} 2025-10-20 05:11:06.553 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 9 retries left... 2025-10-20 05:11:07.119 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {error,mnesia_not_running} 2025-10-20 05:11:07.120 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,tables_not_present} 2025-10-20 05:11:07.120 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 8 retries left... 2025-10-20 05:11:07.623 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {error,mnesia_not_running} 2025-10-20 05:11:07.623 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,tables_not_present} 2025-10-20 05:11:07.623 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 7 retries left... 2025-10-20 05:11:08.128 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {error,mnesia_not_running} 2025-10-20 05:11:08.129 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,tables_not_present} 2025-10-20 05:11:08.129 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 6 retries left... 2025-10-20 05:11:08.630 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {error,mnesia_not_running} 2025-10-20 05:11:08.631 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,tables_not_present} 2025-10-20 05:11:08.631 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 5 retries left... 2025-10-20 05:11:09.137 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {error,mnesia_not_running} 2025-10-20 05:11:09.138 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,tables_not_present} 2025-10-20 05:11:09.138 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 4 retries left... 2025-10-20 05:11:09.645 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {error,mnesia_not_running} 2025-10-20 05:11:09.645 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,tables_not_present} 2025-10-20 05:11:09.646 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 3 retries left... 2025-10-20 05:11:10.151 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {error,mnesia_not_running} 2025-10-20 05:11:10.152 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,tables_not_present} 2025-10-20 05:11:10.152 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 2 retries left... 2025-10-20 05:11:10.653 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {error,mnesia_not_running} 2025-10-20 05:11:10.656 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,tables_not_present} 2025-10-20 05:11:10.656 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 1 retries left... 2025-10-20 05:11:11.157 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {error,mnesia_not_running} 2025-10-20 05:11:11.158 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,tables_not_present} 2025-10-20 05:11:11.158 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 0 retries left... 2025-10-20 05:11:11.659 [warning] <0.240.0> Could not successfully contact any node of: contrail@cn-jenkins-deploy-platform-ansible-os-4371-1,contrail@cn-jenkins-deploy-platform-ansible-os-4371-2 (as in Erlang distribution). Starting as a blank standalone node... 2025-10-20 05:11:11.662 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-10-20 05:11:11.822 [info] <0.240.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-10-20 05:11:11.822 [info] <0.240.0> Successfully synced tables from a peer 2025-10-20 05:11:11.868 [info] <0.240.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-10-20 05:11:11.869 [info] <0.240.0> Successfully synced tables from a peer 2025-10-20 05:11:11.956 [info] <0.240.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-10-20 05:11:11.956 [info] <0.240.0> Successfully synced tables from a peer 2025-10-20 05:11:11.957 [info] <0.240.0> Will register with peer discovery backend rabbit_peer_discovery_classic_config 2025-10-20 05:11:11.957 [info] <0.240.0> Running boot step database_sync defined by app rabbit 2025-10-20 05:11:11.960 [info] <0.240.0> Running boot step feature_flags defined by app rabbit 2025-10-20 05:11:11.962 [info] <0.240.0> Running boot step codec_correctness_check defined by app rabbit 2025-10-20 05:11:11.962 [info] <0.240.0> Running boot step external_infrastructure defined by app rabbit 2025-10-20 05:11:11.962 [info] <0.240.0> Running boot step rabbit_registry defined by app rabbit 2025-10-20 05:11:11.962 [info] <0.240.0> Running boot step rabbit_auth_mechanism_cr_demo defined by app rabbit 2025-10-20 05:11:11.963 [info] <0.240.0> Running boot step rabbit_queue_location_random defined by app rabbit 2025-10-20 05:11:11.964 [info] <0.240.0> Running boot step rabbit_event defined by app rabbit 2025-10-20 05:11:11.964 [info] <0.240.0> Running boot step rabbit_auth_mechanism_amqplain defined by app rabbit 2025-10-20 05:11:11.964 [info] <0.240.0> Running boot step rabbit_auth_mechanism_plain defined by app rabbit 2025-10-20 05:11:11.964 [info] <0.240.0> Running boot step rabbit_exchange_type_direct defined by app rabbit 2025-10-20 05:11:11.964 [info] <0.240.0> Running boot step rabbit_exchange_type_fanout defined by app rabbit 2025-10-20 05:11:11.964 [info] <0.240.0> Running boot step rabbit_exchange_type_headers defined by app rabbit 2025-10-20 05:11:11.964 [info] <0.240.0> Running boot step rabbit_exchange_type_topic defined by app rabbit 2025-10-20 05:11:11.964 [info] <0.240.0> Running boot step rabbit_mirror_queue_mode_all defined by app rabbit 2025-10-20 05:11:11.964 [info] <0.240.0> Running boot step rabbit_mirror_queue_mode_exactly defined by app rabbit 2025-10-20 05:11:11.964 [info] <0.240.0> Running boot step rabbit_mirror_queue_mode_nodes defined by app rabbit 2025-10-20 05:11:11.965 [info] <0.240.0> Running boot step rabbit_priority_queue defined by app rabbit 2025-10-20 05:11:11.965 [info] <0.240.0> Priority queues enabled, real BQ is rabbit_variable_queue 2025-10-20 05:11:11.965 [info] <0.240.0> Running boot step rabbit_queue_location_client_local defined by app rabbit 2025-10-20 05:11:11.965 [info] <0.240.0> Running boot step rabbit_queue_location_min_masters defined by app rabbit 2025-10-20 05:11:11.965 [info] <0.240.0> Running boot step kernel_ready defined by app rabbit 2025-10-20 05:11:11.965 [info] <0.240.0> Running boot step rabbit_sysmon_minder defined by app rabbit 2025-10-20 05:11:11.965 [info] <0.240.0> Running boot step rabbit_epmd_monitor defined by app rabbit 2025-10-20 05:11:11.968 [info] <0.472.0> epmd monitor knows us, inter-node communication (distribution) port: 25673 2025-10-20 05:11:11.968 [info] <0.240.0> Running boot step guid_generator defined by app rabbit 2025-10-20 05:11:11.980 [info] <0.240.0> Running boot step rabbit_node_monitor defined by app rabbit 2025-10-20 05:11:11.981 [info] <0.476.0> Starting rabbit_node_monitor 2025-10-20 05:11:11.981 [info] <0.240.0> Running boot step delegate_sup defined by app rabbit 2025-10-20 05:11:11.982 [info] <0.240.0> Running boot step rabbit_memory_monitor defined by app rabbit 2025-10-20 05:11:11.982 [info] <0.240.0> Running boot step core_initialized defined by app rabbit 2025-10-20 05:11:11.982 [info] <0.240.0> Running boot step upgrade_queues defined by app rabbit 2025-10-20 05:11:12.041 [info] <0.240.0> message_store upgrades: 1 to apply 2025-10-20 05:11:12.041 [info] <0.240.0> message_store upgrades: Applying rabbit_variable_queue:move_messages_to_vhost_store 2025-10-20 05:11:12.041 [info] <0.240.0> message_store upgrades: No durable queues found. Skipping message store migration 2025-10-20 05:11:12.041 [info] <0.240.0> message_store upgrades: Removing the old message store data 2025-10-20 05:11:12.047 [info] <0.240.0> message_store upgrades: All upgrades applied successfully 2025-10-20 05:11:12.080 [info] <0.240.0> Running boot step rabbit_connection_tracking defined by app rabbit 2025-10-20 05:11:12.081 [info] <0.240.0> Running boot step rabbit_connection_tracking_handler defined by app rabbit 2025-10-20 05:11:12.081 [info] <0.240.0> Running boot step rabbit_exchange_parameters defined by app rabbit 2025-10-20 05:11:12.081 [info] <0.240.0> Running boot step rabbit_mirror_queue_misc defined by app rabbit 2025-10-20 05:11:12.082 [info] <0.240.0> Running boot step rabbit_policies defined by app rabbit 2025-10-20 05:11:12.083 [info] <0.240.0> Running boot step rabbit_policy defined by app rabbit 2025-10-20 05:11:12.083 [info] <0.240.0> Running boot step rabbit_queue_location_validator defined by app rabbit 2025-10-20 05:11:12.083 [info] <0.240.0> Running boot step rabbit_vhost_limit defined by app rabbit 2025-10-20 05:11:12.084 [info] <0.240.0> Running boot step rabbit_mgmt_reset_handler defined by app rabbitmq_management 2025-10-20 05:11:12.084 [info] <0.240.0> Running boot step rabbit_mgmt_db_handler defined by app rabbitmq_management_agent 2025-10-20 05:11:12.084 [warning] <0.240.0> force_fine_statistics set to true; ignored. Replaced by {rates_mode, none} in the rabbitmq_management application. 2025-10-20 05:11:12.085 [info] <0.240.0> Management plugin: using rates mode 'basic' 2025-10-20 05:11:12.085 [info] <0.240.0> Running boot step recovery defined by app rabbit 2025-10-20 05:11:12.087 [info] <0.240.0> Running boot step load_definitions defined by app rabbitmq_management 2025-10-20 05:11:12.088 [info] <0.240.0> Applying definitions from /etc/rabbitmq/definitions.json 2025-10-20 05:11:12.088 [info] <0.240.0> Asked to import definitions. Acting user: rmq-internal 2025-10-20 05:11:12.088 [info] <0.240.0> Importing users... 2025-10-20 05:11:12.089 [info] <0.240.0> Creating user 'guest' 2025-10-20 05:11:12.099 [info] <0.240.0> Setting user tags for user 'guest' to [administrator] 2025-10-20 05:11:12.102 [info] <0.240.0> Importing vhosts... 2025-10-20 05:11:12.102 [info] <0.240.0> Adding vhost '/' 2025-10-20 05:11:12.143 [info] <0.524.0> Making sure data directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists 2025-10-20 05:11:12.151 [info] <0.524.0> Starting message stores for vhost '/' 2025-10-20 05:11:12.151 [info] <0.528.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index 2025-10-20 05:11:12.152 [info] <0.524.0> Started message store of type transient for vhost '/' 2025-10-20 05:11:12.152 [info] <0.531.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index 2025-10-20 05:11:12.153 [warning] <0.531.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": rebuilding indices from scratch 2025-10-20 05:11:12.154 [info] <0.524.0> Started message store of type persistent for vhost '/' 2025-10-20 05:11:12.156 [info] <0.240.0> Importing user permissions... 2025-10-20 05:11:12.156 [info] <0.240.0> Setting permissions for 'guest' in '/' to '.*', '.*', '.*' 2025-10-20 05:11:12.160 [info] <0.240.0> Importing topic permissions... 2025-10-20 05:11:12.160 [info] <0.240.0> Importing parameters... 2025-10-20 05:11:12.160 [info] <0.240.0> Importing global parameters... 2025-10-20 05:11:12.160 [info] <0.240.0> Importing policies... 2025-10-20 05:11:12.178 [info] <0.240.0> Importing queues... 2025-10-20 05:11:12.178 [info] <0.240.0> Importing exchanges... 2025-10-20 05:11:12.178 [info] <0.240.0> Importing bindings... 2025-10-20 05:11:12.178 [info] <0.240.0> Running boot step empty_db_check defined by app rabbit 2025-10-20 05:11:12.179 [info] <0.240.0> Running boot step rabbit_looking_glass defined by app rabbit 2025-10-20 05:11:12.179 [info] <0.240.0> Running boot step rabbit_core_metrics_gc defined by app rabbit 2025-10-20 05:11:12.179 [info] <0.240.0> Running boot step background_gc defined by app rabbit 2025-10-20 05:11:12.179 [info] <0.240.0> Running boot step connection_tracking defined by app rabbit 2025-10-20 05:11:12.187 [info] <0.240.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-4371-3' 2025-10-20 05:11:12.194 [info] <0.240.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-4371-3' 2025-10-20 05:11:12.195 [info] <0.240.0> Running boot step routing_ready defined by app rabbit 2025-10-20 05:11:12.195 [info] <0.240.0> Running boot step pre_flight defined by app rabbit 2025-10-20 05:11:12.196 [info] <0.240.0> Running boot step notify_cluster defined by app rabbit 2025-10-20 05:11:12.196 [info] <0.240.0> Running boot step networking defined by app rabbit 2025-10-20 05:11:12.200 [info] <0.577.0> started TCP listener on [::]:5673 2025-10-20 05:11:12.200 [info] <0.240.0> Running boot step cluster_name defined by app rabbit 2025-10-20 05:11:12.200 [info] <0.240.0> Running boot step direct_client defined by app rabbit 2025-10-20 05:11:12.200 [info] <0.240.0> Running boot step os_signal_handler defined by app rabbit 2025-10-20 05:11:12.200 [info] <0.579.0> Swapping OS signal event handler (erl_signal_server) for our own 2025-10-20 05:11:12.236 [info] <0.633.0> Management plugin: HTTP (non-TLS) listener started on port 15673 2025-10-20 05:11:12.236 [info] <0.739.0> Statistics database started. 2025-10-20 05:11:12.236 [info] <0.738.0> Starting worker pool 'management_worker_pool' with 3 processes in it 2025-10-20 05:11:12.237 [notice] <0.106.0> Changed loghwm of /var/log/contrail/config-database-rabbitmq/rabbitmq.log to 50 2025-10-20 05:11:12.355 [info] <0.8.0> Server startup complete; 3 plugins started. * rabbitmq_management * rabbitmq_management_agent * rabbitmq_web_dispatch 2025-10-20 05:11:13.048 [info] <0.749.0> accepting AMQP connection <0.749.0> (10.0.0.48:52746 -> 10.0.0.49:5673) 2025-10-20 05:11:13.048 [info] <0.752.0> accepting AMQP connection <0.752.0> (10.0.0.48:52748 -> 10.0.0.49:5673) 2025-10-20 05:11:13.055 [info] <0.752.0> connection <0.752.0> (10.0.0.48:52748 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:11:13.056 [info] <0.758.0> accepting AMQP connection <0.758.0> (10.0.0.49:55224 -> 10.0.0.49:5673) 2025-10-20 05:11:13.056 [info] <0.749.0> connection <0.749.0> (10.0.0.48:52746 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:11:13.060 [info] <0.758.0> connection <0.758.0> (10.0.0.49:55224 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:11:13.076 [info] <0.787.0> accepting AMQP connection <0.787.0> (10.0.0.254:53626 -> 10.0.0.49:5673) 2025-10-20 05:11:13.086 [info] <0.787.0> connection <0.787.0> (10.0.0.254:53626 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:11:13.098 [info] <0.819.0> accepting AMQP connection <0.819.0> (10.0.0.49:55230 -> 10.0.0.49:5673) 2025-10-20 05:11:13.114 [info] <0.819.0> connection <0.819.0> (10.0.0.49:55230 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:11:13.797 [info] <0.843.0> accepting AMQP connection <0.843.0> (10.0.0.48:52762 -> 10.0.0.49:5673) 2025-10-20 05:11:13.804 [info] <0.843.0> connection <0.843.0> (10.0.0.48:52762 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:11:13.843 [info] <0.863.0> accepting AMQP connection <0.863.0> (10.0.0.48:52764 -> 10.0.0.49:5673) 2025-10-20 05:11:13.873 [info] <0.863.0> connection <0.863.0> (10.0.0.48:52764 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:11:13.878 [info] <0.863.0> closing AMQP connection <0.863.0> (10.0.0.48:52764 -> 10.0.0.49:5673, vhost: '/', user: 'guest') 2025-10-20 05:11:13.879 [info] <0.876.0> accepting AMQP connection <0.876.0> (10.0.0.48:52770 -> 10.0.0.49:5673) 2025-10-20 05:11:13.899 [info] <0.876.0> connection <0.876.0> (10.0.0.48:52770 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:11:14.053 [info] <0.887.0> accepting AMQP connection <0.887.0> (10.0.0.254:53634 -> 10.0.0.49:5673) 2025-10-20 05:11:14.059 [info] <0.887.0> connection <0.887.0> (10.0.0.254:53634 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:11:14.095 [info] <0.907.0> accepting AMQP connection <0.907.0> (10.0.0.254:53648 -> 10.0.0.49:5673) 2025-10-20 05:11:14.106 [info] <0.910.0> accepting AMQP connection <0.910.0> (10.0.0.49:55234 -> 10.0.0.49:5673) 2025-10-20 05:11:14.108 [info] <0.907.0> connection <0.907.0> (10.0.0.254:53648 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:11:14.114 [info] <0.910.0> connection <0.910.0> (10.0.0.49:55234 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:11:14.116 [info] <0.907.0> closing AMQP connection <0.907.0> (10.0.0.254:53648 -> 10.0.0.49:5673, vhost: '/', user: 'guest') 2025-10-20 05:11:14.132 [info] <0.931.0> accepting AMQP connection <0.931.0> (10.0.0.254:53652 -> 10.0.0.49:5673) 2025-10-20 05:11:14.167 [info] <0.931.0> connection <0.931.0> (10.0.0.254:53652 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:11:14.194 [info] <0.951.0> accepting AMQP connection <0.951.0> (10.0.0.49:55246 -> 10.0.0.49:5673) 2025-10-20 05:11:14.222 [info] <0.951.0> connection <0.951.0> (10.0.0.49:55246 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:11:14.240 [info] <0.951.0> closing AMQP connection <0.951.0> (10.0.0.49:55246 -> 10.0.0.49:5673, vhost: '/', user: 'guest') 2025-10-20 05:11:14.245 [info] <0.964.0> accepting AMQP connection <0.964.0> (10.0.0.49:55248 -> 10.0.0.49:5673) 2025-10-20 05:11:14.281 [info] <0.964.0> connection <0.964.0> (10.0.0.49:55248 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:11:14.640 [warning] <0.887.0> closing AMQP connection <0.887.0> (10.0.0.254:53634 -> 10.0.0.49:5673, vhost: '/', user: 'guest'): client unexpectedly closed TCP connection 2025-10-20 05:11:14.640 [warning] <0.931.0> closing AMQP connection <0.931.0> (10.0.0.254:53652 -> 10.0.0.49:5673, vhost: '/', user: 'guest'): client unexpectedly closed TCP connection 2025-10-20 05:11:14.719 [warning] <0.876.0> closing AMQP connection <0.876.0> (10.0.0.48:52770 -> 10.0.0.49:5673, vhost: '/', user: 'guest'): client unexpectedly closed TCP connection 2025-10-20 05:11:14.719 [warning] <0.843.0> closing AMQP connection <0.843.0> (10.0.0.48:52762 -> 10.0.0.49:5673, vhost: '/', user: 'guest'): client unexpectedly closed TCP connection 2025-10-20 05:11:15.987 [info] <0.995.0> RabbitMQ is asked to stop... 2025-10-20 05:11:16.006 [info] <0.998.0> accepting AMQP connection <0.998.0> (10.0.0.48:52778 -> 10.0.0.49:5673) 2025-10-20 05:11:16.006 [info] <0.1001.0> accepting AMQP connection <0.1001.0> (10.0.0.254:53666 -> 10.0.0.49:5673) 2025-10-20 05:11:16.008 [info] <0.1004.0> accepting AMQP connection <0.1004.0> (10.0.0.254:53680 -> 10.0.0.49:5673) 2025-10-20 05:11:16.009 [info] <0.1007.0> accepting AMQP connection <0.1007.0> (10.0.0.49:55250 -> 10.0.0.49:5673) 2025-10-20 05:11:16.012 [info] <0.1010.0> accepting AMQP connection <0.1010.0> (10.0.0.48:52784 -> 10.0.0.49:5673) 2025-10-20 05:11:16.012 [info] <0.1013.0> accepting AMQP connection <0.1013.0> (10.0.0.49:55254 -> 10.0.0.49:5673) 2025-10-20 05:11:16.013 [info] <0.1019.0> accepting AMQP connection <0.1019.0> (10.0.0.49:55270 -> 10.0.0.49:5673) 2025-10-20 05:11:16.014 [info] <0.1022.0> accepting AMQP connection <0.1022.0> (10.0.0.254:53696 -> 10.0.0.49:5673) 2025-10-20 05:11:16.020 [info] <0.1016.0> accepting AMQP connection <0.1016.0> (10.0.0.48:52788 -> 10.0.0.49:5673) 2025-10-20 05:11:16.023 [info] <0.1010.0> connection <0.1010.0> (10.0.0.48:52784 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:11:16.025 [info] <0.1013.0> connection <0.1013.0> (10.0.0.49:55254 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:11:16.027 [info] <0.998.0> connection <0.998.0> (10.0.0.48:52778 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:11:16.040 [info] <0.1004.0> connection <0.1004.0> (10.0.0.254:53680 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:11:16.040 [info] <0.1007.0> connection <0.1007.0> (10.0.0.49:55250 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:11:16.040 [info] <0.1001.0> connection <0.1001.0> (10.0.0.254:53666 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:11:16.051 [info] <0.1022.0> connection <0.1022.0> (10.0.0.254:53696 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:11:16.052 [info] <0.1016.0> connection <0.1016.0> (10.0.0.48:52788 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:11:16.071 [info] <0.1019.0> connection <0.1019.0> (10.0.0.49:55270 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:11:16.177 [info] <0.1163.0> accepting AMQP connection <0.1163.0> (10.0.0.49:55298 -> 10.0.0.49:5673) 2025-10-20 05:11:16.182 [info] <0.1160.0> accepting AMQP connection <0.1160.0> (10.0.0.49:55282 -> 10.0.0.49:5673) 2025-10-20 05:11:16.183 [info] <0.1163.0> connection <0.1163.0> (10.0.0.49:55298 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:11:16.202 [info] <0.1160.0> connection <0.1160.0> (10.0.0.49:55282 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:11:16.269 [info] <0.1160.0> closing AMQP connection <0.1160.0> (10.0.0.49:55282 -> 10.0.0.49:5673, vhost: '/', user: 'guest') 2025-10-20 05:11:16.272 [info] <0.1192.0> accepting AMQP connection <0.1192.0> (10.0.0.49:55300 -> 10.0.0.49:5673) 2025-10-20 05:11:16.278 [info] <0.1192.0> connection <0.1192.0> (10.0.0.49:55300 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:11:16.376 [info] <0.995.0> Stopping RabbitMQ applications and their dependencies in the following order: rabbitmq_management amqp_client rabbitmq_web_dispatch cowboy cowlib rabbitmq_management_agent rabbit mnesia rabbit_common sysmon_handler os_mon 2025-10-20 05:11:16.376 [info] <0.995.0> Stopping application 'rabbitmq_management' 2025-10-20 05:11:16.378 [warning] <0.585.0> RabbitMQ HTTP listener registry could not find context rabbitmq_management_tls 2025-10-20 05:11:16.380 [info] <0.43.0> Application rabbitmq_management exited with reason: stopped 2025-10-20 05:11:16.381 [info] <0.995.0> Stopping application 'amqp_client' 2025-10-20 05:11:16.382 [info] <0.43.0> Application amqp_client exited with reason: stopped 2025-10-20 05:11:16.382 [info] <0.995.0> Stopping application 'rabbitmq_web_dispatch' 2025-10-20 05:11:16.384 [info] <0.995.0> Stopping application 'cowboy' 2025-10-20 05:11:16.384 [info] <0.43.0> Application rabbitmq_web_dispatch exited with reason: stopped 2025-10-20 05:11:16.385 [info] <0.995.0> Stopping application 'cowlib' 2025-10-20 05:11:16.385 [info] <0.995.0> Stopping application 'rabbitmq_management_agent' 2025-10-20 05:11:16.385 [info] <0.43.0> Application cowboy exited with reason: stopped 2025-10-20 05:11:16.385 [info] <0.43.0> Application cowlib exited with reason: stopped 2025-10-20 05:11:16.387 [info] <0.995.0> Stopping application 'rabbit' 2025-10-20 05:11:16.387 [info] <0.43.0> Application rabbitmq_management_agent exited with reason: stopped 2025-10-20 05:11:16.388 [info] <0.240.0> Will unregister with peer discovery backend rabbit_peer_discovery_classic_config 2025-10-20 05:11:16.389 [info] <0.577.0> stopped TCP listener on [::]:5673 2025-10-20 05:11:16.390 [error] <0.1022.0> Error on AMQP connection <0.1022.0> (10.0.0.254:53696 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:11:16.390 [error] <0.1007.0> Error on AMQP connection <0.1007.0> (10.0.0.49:55250 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:11:16.390 [error] <0.1013.0> Error on AMQP connection <0.1013.0> (10.0.0.49:55254 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:11:16.391 [error] <0.910.0> Error on AMQP connection <0.910.0> (10.0.0.49:55234 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:11:16.391 [error] <0.758.0> Error on AMQP connection <0.758.0> (10.0.0.49:55224 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:11:16.391 [error] <0.964.0> Error on AMQP connection <0.964.0> (10.0.0.49:55248 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:11:16.391 [error] <0.1192.0> Error on AMQP connection <0.1192.0> (10.0.0.49:55300 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:11:16.391 [error] <0.998.0> Error on AMQP connection <0.998.0> (10.0.0.48:52778 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:11:17.423 [warning] lager_file_backend dropped 43 messages in the last second that exceeded the limit of 50 messages/sec 2025-10-20 05:11:45.416 [info] <0.1312.0> Resetting Rabbit forcefully 2025-10-20 05:11:47.590 [info] <0.1327.0> RabbitMQ is asked to stop... 2025-10-20 05:11:47.792 [info] <0.1327.0> Stopping RabbitMQ applications and their dependencies in the following order: rabbitmq_management amqp_client rabbitmq_management_agent rabbitmq_web_dispatch rabbit mnesia rabbit_common sysmon_handler os_mon 2025-10-20 05:11:47.792 [info] <0.1327.0> Stopping application 'rabbitmq_management' 2025-10-20 05:11:47.792 [info] <0.1327.0> Stopping application 'amqp_client' 2025-10-20 05:11:47.792 [info] <0.1327.0> Stopping application 'rabbitmq_management_agent' 2025-10-20 05:11:47.792 [info] <0.1327.0> Stopping application 'rabbitmq_web_dispatch' 2025-10-20 05:11:47.792 [info] <0.1327.0> Stopping application 'rabbit' 2025-10-20 05:11:47.792 [info] <0.1327.0> Stopping application 'mnesia' 2025-10-20 05:11:47.792 [info] <0.1327.0> Stopping application 'rabbit_common' 2025-10-20 05:11:47.792 [info] <0.1327.0> Stopping application 'sysmon_handler' 2025-10-20 05:11:47.792 [info] <0.1327.0> Stopping application 'os_mon' 2025-10-20 05:11:47.792 [info] <0.1327.0> Successfully stopped RabbitMQ and its dependencies 2025-10-20 05:11:47.792 [info] <0.1327.0> Halting Erlang VM with the following applications: ranch ssl public_key asn1 credentials_obfuscation observer_cli jsx tools xmerl crypto stdout_formatter recon inets lager goldrush compiler syntax_tools sasl stdlib kernel 2025-10-20 05:11:57.249 [info] <0.8.0> Log file opened with Lager 2025-10-20 05:12:00.451 [info] <0.8.0> Feature flags: list of feature flags found: 2025-10-20 05:12:00.451 [info] <0.8.0> Feature flags: feature flag states written to disk: yes 2025-10-20 05:12:00.476 [info] <0.240.0> Starting RabbitMQ 3.7.28 on Erlang 21.3.8.21 Copyright (c) 2007-2020 Pivotal Software, Inc. Licensed under the MPL. See https://www.rabbitmq.com/ 2025-10-20 05:12:00.477 [info] <0.240.0> node : contrail@cn-jenkins-deploy-platform-ansible-os-4371-3 home dir : /var/lib/rabbitmq config file(s) : /etc/rabbitmq/rabbitmq.config cookie hash : esZsDxSN6VGbi9JkMSxNZA== log(s) : /var/log/contrail/config-database-rabbitmq/rabbitmq.log : /var/log/rabbitmq/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3_upgrade.log database dir : /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3 2025-10-20 05:12:00.493 [info] <0.240.0> Running boot step pre_boot defined by app rabbit 2025-10-20 05:12:00.493 [info] <0.240.0> Running boot step rabbit_core_metrics defined by app rabbit 2025-10-20 05:12:00.493 [info] <0.240.0> Running boot step rabbit_alarm defined by app rabbit 2025-10-20 05:12:00.502 [info] <0.246.0> Memory high watermark set to 12637 MiB (13251084288 bytes) of 15796 MiB (16563855360 bytes) total 2025-10-20 05:12:00.514 [info] <0.248.0> Enabling free disk space monitoring 2025-10-20 05:12:00.514 [info] <0.248.0> Disk free limit set to 50MB 2025-10-20 05:12:00.521 [info] <0.240.0> Running boot step code_server_cache defined by app rabbit 2025-10-20 05:12:00.521 [info] <0.240.0> Running boot step file_handle_cache defined by app rabbit 2025-10-20 05:12:00.521 [info] <0.251.0> Limiting to approx 65436 file handles (58890 sockets) 2025-10-20 05:12:00.521 [info] <0.252.0> FHC read buffering: OFF 2025-10-20 05:12:00.521 [info] <0.252.0> FHC write buffering: ON 2025-10-20 05:12:00.525 [info] <0.240.0> Running boot step worker_pool defined by app rabbit 2025-10-20 05:12:00.525 [info] <0.241.0> Will use 4 processes for default worker pool 2025-10-20 05:12:00.525 [info] <0.241.0> Starting worker pool 'worker_pool' with 4 processes in it 2025-10-20 05:12:00.526 [info] <0.240.0> Running boot step database defined by app rabbit 2025-10-20 05:12:00.526 [info] <0.240.0> Node database directory at /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3 is empty. Assuming we need to join an existing cluster or initialise from scratch... 2025-10-20 05:12:00.526 [info] <0.240.0> Configured peer discovery backend: rabbit_peer_discovery_classic_config 2025-10-20 05:12:00.526 [info] <0.240.0> Will try to lock with peer discovery backend rabbit_peer_discovery_classic_config 2025-10-20 05:12:00.526 [info] <0.240.0> Peer discovery backend does not support locking, falling back to randomized delay 2025-10-20 05:12:00.526 [info] <0.240.0> Peer discovery backend rabbit_peer_discovery_classic_config supports registration. 2025-10-20 05:12:00.526 [info] <0.240.0> Will wait for 16546 milliseconds before proceeding with registration... 2025-10-20 05:12:17.073 [info] <0.240.0> All discovered existing cluster peers: contrail@cn-jenkins-deploy-platform-ansible-os-4371-1, contrail@cn-jenkins-deploy-platform-ansible-os-4371-2, contrail@cn-jenkins-deploy-platform-ansible-os-4371-3 2025-10-20 05:12:17.073 [info] <0.240.0> Peer nodes we can cluster with: contrail@cn-jenkins-deploy-platform-ansible-os-4371-1, contrail@cn-jenkins-deploy-platform-ansible-os-4371-2 2025-10-20 05:12:17.119 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {error,mnesia_not_running} 2025-10-20 05:12:17.121 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {badrpc,nodedown} 2025-10-20 05:12:17.121 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 9 retries left... 2025-10-20 05:12:17.625 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {error,mnesia_not_running} 2025-10-20 05:12:17.699 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:12:17.700 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 8 retries left... 2025-10-20 05:12:18.202 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {error,mnesia_not_running} 2025-10-20 05:12:18.203 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:12:18.203 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 7 retries left... 2025-10-20 05:12:18.705 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {error,mnesia_not_running} 2025-10-20 05:12:18.707 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:12:18.707 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 6 retries left... 2025-10-20 05:12:19.208 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {error,mnesia_not_running} 2025-10-20 05:12:19.212 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:12:19.212 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 5 retries left... 2025-10-20 05:12:19.715 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {error,mnesia_not_running} 2025-10-20 05:12:19.718 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:12:19.718 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 4 retries left... 2025-10-20 05:12:20.223 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:12:20.223 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,tables_not_present} 2025-10-20 05:12:20.224 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 3 retries left... 2025-10-20 05:12:20.727 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:12:20.728 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,tables_not_present} 2025-10-20 05:12:20.728 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 2 retries left... 2025-10-20 05:12:21.229 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:12:21.230 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,tables_not_present} 2025-10-20 05:12:21.230 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 1 retries left... 2025-10-20 05:12:21.734 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:12:21.734 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,tables_not_present} 2025-10-20 05:12:21.735 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 0 retries left... 2025-10-20 05:12:22.235 [warning] <0.240.0> Could not successfully contact any node of: contrail@cn-jenkins-deploy-platform-ansible-os-4371-1,contrail@cn-jenkins-deploy-platform-ansible-os-4371-2 (as in Erlang distribution). Starting as a blank standalone node... 2025-10-20 05:12:22.239 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-10-20 05:12:22.517 [info] <0.240.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-10-20 05:12:22.517 [info] <0.240.0> Successfully synced tables from a peer 2025-10-20 05:12:22.558 [info] <0.240.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-10-20 05:12:22.559 [info] <0.240.0> Successfully synced tables from a peer 2025-10-20 05:12:22.595 [info] <0.240.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-10-20 05:12:22.595 [info] <0.240.0> Successfully synced tables from a peer 2025-10-20 05:12:22.595 [info] <0.240.0> Will register with peer discovery backend rabbit_peer_discovery_classic_config 2025-10-20 05:12:22.596 [info] <0.240.0> Running boot step database_sync defined by app rabbit 2025-10-20 05:12:22.596 [info] <0.240.0> Running boot step feature_flags defined by app rabbit 2025-10-20 05:12:22.596 [info] <0.240.0> Running boot step codec_correctness_check defined by app rabbit 2025-10-20 05:12:22.596 [info] <0.240.0> Running boot step external_infrastructure defined by app rabbit 2025-10-20 05:12:22.596 [info] <0.240.0> Running boot step rabbit_registry defined by app rabbit 2025-10-20 05:12:22.596 [info] <0.240.0> Running boot step rabbit_auth_mechanism_cr_demo defined by app rabbit 2025-10-20 05:12:22.596 [info] <0.240.0> Running boot step rabbit_queue_location_random defined by app rabbit 2025-10-20 05:12:22.596 [info] <0.240.0> Running boot step rabbit_event defined by app rabbit 2025-10-20 05:12:22.596 [info] <0.240.0> Running boot step rabbit_auth_mechanism_amqplain defined by app rabbit 2025-10-20 05:12:22.596 [info] <0.240.0> Running boot step rabbit_auth_mechanism_plain defined by app rabbit 2025-10-20 05:12:22.596 [info] <0.240.0> Running boot step rabbit_exchange_type_direct defined by app rabbit 2025-10-20 05:12:22.596 [info] <0.240.0> Running boot step rabbit_exchange_type_fanout defined by app rabbit 2025-10-20 05:12:22.596 [info] <0.240.0> Running boot step rabbit_exchange_type_headers defined by app rabbit 2025-10-20 05:12:22.597 [info] <0.240.0> Running boot step rabbit_exchange_type_topic defined by app rabbit 2025-10-20 05:12:22.597 [info] <0.240.0> Running boot step rabbit_mirror_queue_mode_all defined by app rabbit 2025-10-20 05:12:22.597 [info] <0.240.0> Running boot step rabbit_mirror_queue_mode_exactly defined by app rabbit 2025-10-20 05:12:22.597 [info] <0.240.0> Running boot step rabbit_mirror_queue_mode_nodes defined by app rabbit 2025-10-20 05:12:22.597 [info] <0.240.0> Running boot step rabbit_priority_queue defined by app rabbit 2025-10-20 05:12:22.597 [info] <0.240.0> Priority queues enabled, real BQ is rabbit_variable_queue 2025-10-20 05:12:22.597 [info] <0.240.0> Running boot step rabbit_queue_location_client_local defined by app rabbit 2025-10-20 05:12:22.597 [info] <0.240.0> Running boot step rabbit_queue_location_min_masters defined by app rabbit 2025-10-20 05:12:22.597 [info] <0.240.0> Running boot step kernel_ready defined by app rabbit 2025-10-20 05:12:22.597 [info] <0.240.0> Running boot step rabbit_sysmon_minder defined by app rabbit 2025-10-20 05:12:22.597 [info] <0.240.0> Running boot step rabbit_epmd_monitor defined by app rabbit 2025-10-20 05:12:22.598 [info] <0.478.0> epmd monitor knows us, inter-node communication (distribution) port: 25673 2025-10-20 05:12:22.598 [info] <0.240.0> Running boot step guid_generator defined by app rabbit 2025-10-20 05:12:22.602 [info] <0.240.0> Running boot step rabbit_node_monitor defined by app rabbit 2025-10-20 05:12:22.602 [info] <0.482.0> Starting rabbit_node_monitor 2025-10-20 05:12:22.602 [info] <0.240.0> Running boot step delegate_sup defined by app rabbit 2025-10-20 05:12:22.602 [info] <0.240.0> Running boot step rabbit_memory_monitor defined by app rabbit 2025-10-20 05:12:22.603 [info] <0.240.0> Running boot step core_initialized defined by app rabbit 2025-10-20 05:12:22.603 [info] <0.240.0> Running boot step upgrade_queues defined by app rabbit 2025-10-20 05:12:22.626 [info] <0.240.0> message_store upgrades: 1 to apply 2025-10-20 05:12:22.627 [info] <0.240.0> message_store upgrades: Applying rabbit_variable_queue:move_messages_to_vhost_store 2025-10-20 05:12:22.627 [info] <0.240.0> message_store upgrades: No durable queues found. Skipping message store migration 2025-10-20 05:12:22.627 [info] <0.240.0> message_store upgrades: Removing the old message store data 2025-10-20 05:12:22.627 [info] <0.240.0> message_store upgrades: All upgrades applied successfully 2025-10-20 05:12:22.655 [info] <0.240.0> Running boot step rabbit_connection_tracking defined by app rabbit 2025-10-20 05:12:22.655 [info] <0.240.0> Running boot step rabbit_connection_tracking_handler defined by app rabbit 2025-10-20 05:12:22.655 [info] <0.240.0> Running boot step rabbit_exchange_parameters defined by app rabbit 2025-10-20 05:12:22.655 [info] <0.240.0> Running boot step rabbit_mirror_queue_misc defined by app rabbit 2025-10-20 05:12:22.655 [info] <0.240.0> Running boot step rabbit_policies defined by app rabbit 2025-10-20 05:12:22.656 [info] <0.240.0> Running boot step rabbit_policy defined by app rabbit 2025-10-20 05:12:22.656 [info] <0.240.0> Running boot step rabbit_queue_location_validator defined by app rabbit 2025-10-20 05:12:22.656 [info] <0.240.0> Running boot step rabbit_vhost_limit defined by app rabbit 2025-10-20 05:12:22.656 [info] <0.240.0> Running boot step rabbit_mgmt_reset_handler defined by app rabbitmq_management 2025-10-20 05:12:22.656 [info] <0.240.0> Running boot step rabbit_mgmt_db_handler defined by app rabbitmq_management_agent 2025-10-20 05:12:22.656 [warning] <0.240.0> force_fine_statistics set to true; ignored. Replaced by {rates_mode, none} in the rabbitmq_management application. 2025-10-20 05:12:22.656 [info] <0.240.0> Management plugin: using rates mode 'basic' 2025-10-20 05:12:22.656 [info] <0.240.0> Running boot step recovery defined by app rabbit 2025-10-20 05:12:22.657 [info] <0.240.0> Running boot step load_definitions defined by app rabbitmq_management 2025-10-20 05:12:22.657 [info] <0.240.0> Applying definitions from /etc/rabbitmq/definitions.json 2025-10-20 05:12:22.657 [info] <0.240.0> Asked to import definitions. Acting user: rmq-internal 2025-10-20 05:12:22.657 [info] <0.240.0> Importing users... 2025-10-20 05:12:22.657 [info] <0.240.0> Creating user 'guest' 2025-10-20 05:12:22.665 [info] <0.240.0> Setting user tags for user 'guest' to [administrator] 2025-10-20 05:12:22.668 [info] <0.240.0> Importing vhosts... 2025-10-20 05:12:22.668 [info] <0.240.0> Adding vhost '/' 2025-10-20 05:12:22.695 [info] <0.530.0> Making sure data directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists 2025-10-20 05:12:22.698 [info] <0.530.0> Starting message stores for vhost '/' 2025-10-20 05:12:22.699 [info] <0.534.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index 2025-10-20 05:12:22.703 [info] <0.530.0> Started message store of type transient for vhost '/' 2025-10-20 05:12:22.704 [info] <0.537.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index 2025-10-20 05:12:22.706 [warning] <0.537.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": rebuilding indices from scratch 2025-10-20 05:12:22.707 [info] <0.530.0> Started message store of type persistent for vhost '/' 2025-10-20 05:12:22.708 [info] <0.240.0> Importing user permissions... 2025-10-20 05:12:22.708 [info] <0.240.0> Setting permissions for 'guest' in '/' to '.*', '.*', '.*' 2025-10-20 05:12:22.712 [info] <0.240.0> Importing topic permissions... 2025-10-20 05:12:22.712 [info] <0.240.0> Importing parameters... 2025-10-20 05:12:22.712 [info] <0.240.0> Importing global parameters... 2025-10-20 05:12:22.712 [info] <0.240.0> Importing policies... 2025-10-20 05:12:22.718 [info] <0.240.0> Importing queues... 2025-10-20 05:12:22.718 [info] <0.240.0> Importing exchanges... 2025-10-20 05:12:22.718 [info] <0.240.0> Importing bindings... 2025-10-20 05:12:22.718 [info] <0.240.0> Running boot step empty_db_check defined by app rabbit 2025-10-20 05:12:22.718 [info] <0.240.0> Running boot step rabbit_looking_glass defined by app rabbit 2025-10-20 05:12:22.718 [info] <0.240.0> Running boot step rabbit_core_metrics_gc defined by app rabbit 2025-10-20 05:12:22.719 [info] <0.240.0> Running boot step background_gc defined by app rabbit 2025-10-20 05:12:22.719 [info] <0.240.0> Running boot step connection_tracking defined by app rabbit 2025-10-20 05:12:22.726 [info] <0.240.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-4371-3' 2025-10-20 05:12:22.742 [info] <0.240.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-4371-3' 2025-10-20 05:12:22.742 [info] <0.240.0> Running boot step routing_ready defined by app rabbit 2025-10-20 05:12:22.742 [info] <0.240.0> Running boot step pre_flight defined by app rabbit 2025-10-20 05:12:22.742 [info] <0.240.0> Running boot step notify_cluster defined by app rabbit 2025-10-20 05:12:22.742 [info] <0.240.0> Running boot step networking defined by app rabbit 2025-10-20 05:12:22.746 [info] <0.583.0> started TCP listener on [::]:5673 2025-10-20 05:12:22.749 [info] <0.240.0> Running boot step cluster_name defined by app rabbit 2025-10-20 05:12:22.750 [info] <0.240.0> Running boot step direct_client defined by app rabbit 2025-10-20 05:12:22.750 [info] <0.240.0> Running boot step os_signal_handler defined by app rabbit 2025-10-20 05:12:22.750 [info] <0.585.0> Swapping OS signal event handler (erl_signal_server) for our own 2025-10-20 05:12:22.783 [info] <0.635.0> Management plugin: HTTP (non-TLS) listener started on port 15673 2025-10-20 05:12:22.784 [info] <0.741.0> Statistics database started. 2025-10-20 05:12:22.784 [info] <0.740.0> Starting worker pool 'management_worker_pool' with 3 processes in it 2025-10-20 05:12:22.785 [notice] <0.106.0> Changed loghwm of /var/log/contrail/config-database-rabbitmq/rabbitmq.log to 50 2025-10-20 05:12:22.816 [info] <0.751.0> accepting AMQP connection <0.751.0> (10.0.0.254:60084 -> 10.0.0.49:5673) 2025-10-20 05:12:22.816 [info] <0.760.0> accepting AMQP connection <0.760.0> (10.0.0.48:44802 -> 10.0.0.49:5673) 2025-10-20 05:12:22.816 [info] <0.754.0> accepting AMQP connection <0.754.0> (10.0.0.48:44784 -> 10.0.0.49:5673) 2025-10-20 05:12:22.816 [info] <0.757.0> accepting AMQP connection <0.757.0> (10.0.0.48:44788 -> 10.0.0.49:5673) 2025-10-20 05:12:22.817 [info] <0.763.0> accepting AMQP connection <0.763.0> (10.0.0.254:60112 -> 10.0.0.49:5673) 2025-10-20 05:12:22.817 [info] <0.769.0> accepting AMQP connection <0.769.0> (10.0.0.254:60096 -> 10.0.0.49:5673) 2025-10-20 05:12:22.818 [info] <0.772.0> accepting AMQP connection <0.772.0> (10.0.0.49:44900 -> 10.0.0.49:5673) 2025-10-20 05:12:22.818 [info] <0.766.0> accepting AMQP connection <0.766.0> (10.0.0.49:44886 -> 10.0.0.49:5673) 2025-10-20 05:12:22.819 [info] <0.779.0> accepting AMQP connection <0.779.0> (10.0.0.49:44914 -> 10.0.0.49:5673) 2025-10-20 05:12:22.821 [info] <0.772.0> connection <0.772.0> (10.0.0.49:44900 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:12:22.822 [info] <0.779.0> connection <0.779.0> (10.0.0.49:44914 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:12:22.822 [info] <0.766.0> connection <0.766.0> (10.0.0.49:44886 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:12:22.823 [info] <0.769.0> connection <0.769.0> (10.0.0.254:60096 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:12:22.824 [info] <0.751.0> connection <0.751.0> (10.0.0.254:60084 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:12:22.825 [info] <0.757.0> connection <0.757.0> (10.0.0.48:44788 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:12:22.830 [info] <0.754.0> connection <0.754.0> (10.0.0.48:44784 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:12:22.832 [info] <0.763.0> connection <0.763.0> (10.0.0.254:60112 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:12:22.833 [info] <0.760.0> connection <0.760.0> (10.0.0.48:44802 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:12:22.962 [info] <0.8.0> Server startup complete; 3 plugins started. * rabbitmq_management * rabbitmq_management_agent * rabbitmq_web_dispatch 2025-10-20 05:12:24.860 [info] <0.921.0> accepting AMQP connection <0.921.0> (10.0.0.48:44804 -> 10.0.0.49:5673) 2025-10-20 05:12:24.863 [info] <0.921.0> connection <0.921.0> (10.0.0.48:44804 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:12:24.864 [info] <0.926.0> accepting AMQP connection <0.926.0> (10.0.0.49:44916 -> 10.0.0.49:5673) 2025-10-20 05:12:24.865 [info] <0.926.0> connection <0.926.0> (10.0.0.49:44916 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:12:24.889 [info] <0.957.0> accepting AMQP connection <0.957.0> (10.0.0.48:44814 -> 10.0.0.49:5673) 2025-10-20 05:12:24.893 [info] <0.957.0> connection <0.957.0> (10.0.0.48:44814 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:12:24.908 [info] <0.975.0> accepting AMQP connection <0.975.0> (10.0.0.49:44928 -> 10.0.0.49:5673) 2025-10-20 05:12:24.911 [info] <0.975.0> connection <0.975.0> (10.0.0.49:44928 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:12:24.968 [info] <0.993.0> accepting AMQP connection <0.993.0> (10.0.0.48:44824 -> 10.0.0.49:5673) 2025-10-20 05:12:24.970 [info] <0.993.0> connection <0.993.0> (10.0.0.48:44824 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:12:28.474 [info] <0.1059.0> accepting AMQP connection <0.1059.0> (10.0.0.254:60114 -> 10.0.0.49:5673) 2025-10-20 05:12:28.478 [info] <0.1062.0> accepting AMQP connection <0.1062.0> (10.0.0.49:44940 -> 10.0.0.49:5673) 2025-10-20 05:12:28.478 [info] <0.1065.0> accepting AMQP connection <0.1065.0> (10.0.0.48:44828 -> 10.0.0.49:5673) 2025-10-20 05:12:28.479 [info] <0.1059.0> connection <0.1059.0> (10.0.0.254:60114 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:12:28.480 [info] <0.1065.0> connection <0.1065.0> (10.0.0.48:44828 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:12:28.483 [info] <0.1076.0> accepting AMQP connection <0.1076.0> (10.0.0.49:44946 -> 10.0.0.49:5673) 2025-10-20 05:12:28.486 [info] <0.1084.0> accepting AMQP connection <0.1084.0> (10.0.0.48:44842 -> 10.0.0.49:5673) 2025-10-20 05:12:28.494 [info] <0.1076.0> connection <0.1076.0> (10.0.0.49:44946 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:12:28.496 [info] <0.1062.0> connection <0.1062.0> (10.0.0.49:44940 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:12:28.499 [info] <0.1084.0> connection <0.1084.0> (10.0.0.48:44842 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:12:28.501 [info] <0.1124.0> accepting AMQP connection <0.1124.0> (10.0.0.49:44958 -> 10.0.0.49:5673) 2025-10-20 05:12:28.505 [info] <0.1124.0> connection <0.1124.0> (10.0.0.49:44958 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:12:28.582 [info] <0.1169.0> accepting AMQP connection <0.1169.0> (10.0.0.49:44962 -> 10.0.0.49:5673) 2025-10-20 05:12:28.585 [info] <0.1169.0> connection <0.1169.0> (10.0.0.49:44962 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:12:30.999 [info] <0.1190.0> accepting AMQP connection <0.1190.0> (10.0.0.49:44976 -> 10.0.0.49:5673) 2025-10-20 05:12:31.002 [info] <0.1190.0> connection <0.1190.0> (10.0.0.49:44976 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:12:33.290 [info] <0.1201.0> accepting AMQP connection <0.1201.0> (10.0.0.254:59592 -> 10.0.0.49:5673) 2025-10-20 05:12:33.305 [info] <0.1201.0> connection <0.1201.0> (10.0.0.254:59592 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:12:33.356 [info] <0.1219.0> accepting AMQP connection <0.1219.0> (10.0.0.254:59608 -> 10.0.0.49:5673) 2025-10-20 05:12:33.359 [info] <0.1219.0> connection <0.1219.0> (10.0.0.254:59608 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:12:33.370 [info] <0.1219.0> closing AMQP connection <0.1219.0> (10.0.0.254:59608 -> 10.0.0.49:5673, vhost: '/', user: 'guest') 2025-10-20 05:12:33.376 [info] <0.1230.0> accepting AMQP connection <0.1230.0> (10.0.0.254:59618 -> 10.0.0.49:5673) 2025-10-20 05:12:33.383 [info] <0.1230.0> connection <0.1230.0> (10.0.0.254:59618 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:12:33.577 [info] <0.482.0> node 'contrail@cn-jenkins-deploy-platform-ansible-os-4371-1' up 2025-10-20 05:12:35.653 [info] <0.1257.0> accepting AMQP connection <0.1257.0> (10.0.0.48:43268 -> 10.0.0.49:5673) 2025-10-20 05:12:35.657 [info] <0.1257.0> connection <0.1257.0> (10.0.0.48:43268 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:12:35.708 [info] <0.1277.0> accepting AMQP connection <0.1277.0> (10.0.0.48:43276 -> 10.0.0.49:5673) 2025-10-20 05:12:35.712 [info] <0.1277.0> connection <0.1277.0> (10.0.0.48:43276 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:12:35.718 [info] <0.1277.0> closing AMQP connection <0.1277.0> (10.0.0.48:43276 -> 10.0.0.49:5673, vhost: '/', user: 'guest') 2025-10-20 05:12:35.720 [info] <0.1290.0> accepting AMQP connection <0.1290.0> (10.0.0.48:43286 -> 10.0.0.49:5673) 2025-10-20 05:12:35.725 [info] <0.1290.0> connection <0.1290.0> (10.0.0.48:43286 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:12:36.021 [info] <0.1302.0> RabbitMQ is asked to stop... 2025-10-20 05:12:36.183 [info] <0.1302.0> Stopping RabbitMQ applications and their dependencies in the following order: rabbitmq_management amqp_client rabbitmq_web_dispatch cowboy cowlib rabbitmq_management_agent rabbit mnesia rabbit_common sysmon_handler os_mon 2025-10-20 05:12:36.183 [info] <0.1302.0> Stopping application 'rabbitmq_management' 2025-10-20 05:12:36.185 [warning] <0.591.0> RabbitMQ HTTP listener registry could not find context rabbitmq_management_tls 2025-10-20 05:12:36.188 [info] <0.43.0> Application rabbitmq_management exited with reason: stopped 2025-10-20 05:12:36.188 [info] <0.1302.0> Stopping application 'amqp_client' 2025-10-20 05:12:36.190 [info] <0.43.0> Application amqp_client exited with reason: stopped 2025-10-20 05:12:36.190 [info] <0.1302.0> Stopping application 'rabbitmq_web_dispatch' 2025-10-20 05:12:36.193 [info] <0.43.0> Application rabbitmq_web_dispatch exited with reason: stopped 2025-10-20 05:12:36.193 [info] <0.1302.0> Stopping application 'cowboy' 2025-10-20 05:12:36.196 [info] <0.43.0> Application cowboy exited with reason: stopped 2025-10-20 05:12:36.196 [info] <0.1302.0> Stopping application 'cowlib' 2025-10-20 05:12:36.196 [info] <0.43.0> Application cowlib exited with reason: stopped 2025-10-20 05:12:36.196 [info] <0.1302.0> Stopping application 'rabbitmq_management_agent' 2025-10-20 05:12:36.199 [info] <0.43.0> Application rabbitmq_management_agent exited with reason: stopped 2025-10-20 05:12:36.199 [info] <0.1302.0> Stopping application 'rabbit' 2025-10-20 05:12:36.199 [info] <0.240.0> Will unregister with peer discovery backend rabbit_peer_discovery_classic_config 2025-10-20 05:12:36.199 [info] <0.583.0> stopped TCP listener on [::]:5673 2025-10-20 05:12:36.200 [error] <0.1059.0> Error on AMQP connection <0.1059.0> (10.0.0.254:60114 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:12:36.200 [error] <0.760.0> Error on AMQP connection <0.760.0> (10.0.0.48:44802 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:12:36.200 [error] <0.1169.0> Error on AMQP connection <0.1169.0> (10.0.0.49:44962 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:12:36.200 [error] <0.993.0> Error on AMQP connection <0.993.0> (10.0.0.48:44824 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:12:36.200 [error] <0.1124.0> Error on AMQP connection <0.1124.0> (10.0.0.49:44958 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:12:36.200 [error] <0.1065.0> Error on AMQP connection <0.1065.0> (10.0.0.48:44828 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:12:36.200 [error] <0.763.0> Error on AMQP connection <0.763.0> (10.0.0.254:60112 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:12:36.200 [error] <0.1084.0> Error on AMQP connection <0.1084.0> (10.0.0.48:44842 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:12:36.200 [error] <0.751.0> Error on AMQP connection <0.751.0> (10.0.0.254:60084 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:12:36.200 [error] <0.772.0> Error on AMQP connection <0.772.0> (10.0.0.49:44900 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:12:36.200 [error] <0.1230.0> Error on AMQP connection <0.1230.0> (10.0.0.254:59618 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:12:36.200 [error] <0.754.0> Error on AMQP connection <0.754.0> (10.0.0.48:44784 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:12:36.200 [error] <0.926.0> Error on AMQP connection <0.926.0> (10.0.0.49:44916 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:12:36.201 [error] <0.1257.0> Error on AMQP connection <0.1257.0> (10.0.0.48:43268 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:12:36.201 [error] <0.769.0> Error on AMQP connection <0.769.0> (10.0.0.254:60096 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:12:36.201 [error] <0.1076.0> Error on AMQP connection <0.1076.0> (10.0.0.49:44946 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:12:36.200 [error] <0.1201.0> Error on AMQP connection <0.1201.0> (10.0.0.254:59592 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:12:36.201 [error] <0.766.0> Error on AMQP connection <0.766.0> (10.0.0.49:44886 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:12:36.200 [error] <0.1190.0> Error on AMQP connection <0.1190.0> (10.0.0.49:44976 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:12:36.200 [error] <0.957.0> Error on AMQP connection <0.957.0> (10.0.0.48:44814 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:12:36.200 [error] <0.1062.0> Error on AMQP connection <0.1062.0> (10.0.0.49:44940 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:12:36.200 [error] <0.1290.0> Error on AMQP connection <0.1290.0> (10.0.0.48:43286 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:12:36.201 [error] <0.921.0> Error on AMQP connection <0.921.0> (10.0.0.48:44804 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:12:36.201 [error] <0.975.0> Error on AMQP connection <0.975.0> (10.0.0.49:44928 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:12:36.200 [error] <0.757.0> Error on AMQP connection <0.757.0> (10.0.0.48:44788 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:12:36.201 [error] <0.779.0> Error on AMQP connection <0.779.0> (10.0.0.49:44914 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:12:36.224 [info] <0.507.0> Closing all connections in vhost '/' on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4371-3' because the vhost is stopping 2025-10-20 05:12:36.225 [warning] <0.1150.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-4371-2.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-4371-2.' in vhost '/': Stopping all nodes on master shutdown since no synchronised slave is available 2025-10-20 05:12:36.225 [warning] <0.1013.0> Mirrored queue 'job_request_consumer' in vhost '/': Stopping all nodes on master shutdown since no synchronised slave is available 2025-10-20 05:12:36.226 [warning] <0.984.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-4371-3.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-4371-3.' in vhost '/': Stopping all nodes on master shutdown since no synchronised slave is available 2025-10-20 05:12:36.226 [warning] <0.816.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-4371-3' in vhost '/': Stopping all nodes on master shutdown since no synchronised slave is available 2025-10-20 05:12:36.226 [warning] <0.1050.0> Mirrored queue 'device_manager_ztp.ztp_queue' in vhost '/': Stopping all nodes on master shutdown since no synchronised slave is available 2025-10-20 05:12:37.273 [warning] lager_file_backend dropped 35 messages in the last second that exceeded the limit of 50 messages/sec 2025-10-20 05:12:46.265 [info] <0.1426.0> Resetting Rabbit forcefully 2025-10-20 05:12:47.922 [info] <0.1440.0> RabbitMQ is asked to stop... 2025-10-20 05:12:47.996 [info] <0.1440.0> Stopping RabbitMQ applications and their dependencies in the following order: rabbitmq_management amqp_client rabbitmq_management_agent rabbitmq_web_dispatch rabbit mnesia rabbit_common sysmon_handler os_mon 2025-10-20 05:12:47.996 [info] <0.1440.0> Stopping application 'rabbitmq_management' 2025-10-20 05:12:47.997 [info] <0.1440.0> Stopping application 'amqp_client' 2025-10-20 05:12:47.997 [info] <0.1440.0> Stopping application 'rabbitmq_management_agent' 2025-10-20 05:12:47.997 [info] <0.1440.0> Stopping application 'rabbitmq_web_dispatch' 2025-10-20 05:12:47.997 [info] <0.1440.0> Stopping application 'rabbit' 2025-10-20 05:12:47.997 [info] <0.1440.0> Stopping application 'mnesia' 2025-10-20 05:12:47.997 [info] <0.1440.0> Stopping application 'rabbit_common' 2025-10-20 05:12:47.997 [info] <0.1440.0> Stopping application 'sysmon_handler' 2025-10-20 05:12:47.997 [info] <0.1440.0> Stopping application 'os_mon' 2025-10-20 05:12:47.997 [info] <0.1440.0> Successfully stopped RabbitMQ and its dependencies 2025-10-20 05:12:47.997 [info] <0.1440.0> Halting Erlang VM with the following applications: ranch ssl public_key asn1 observer_cli tools credentials_obfuscation crypto stdout_formatter recon inets jsx xmerl lager goldrush compiler syntax_tools sasl stdlib kernel 2025-10-20 05:12:56.660 [info] <0.8.0> Log file opened with Lager 2025-10-20 05:12:59.270 [info] <0.8.0> Feature flags: list of feature flags found: 2025-10-20 05:12:59.270 [info] <0.8.0> Feature flags: feature flag states written to disk: yes 2025-10-20 05:12:59.293 [info] <0.253.0> Starting RabbitMQ 3.7.28 on Erlang 21.3.8.21 Copyright (c) 2007-2020 Pivotal Software, Inc. Licensed under the MPL. See https://www.rabbitmq.com/ 2025-10-20 05:12:59.294 [info] <0.253.0> node : contrail@cn-jenkins-deploy-platform-ansible-os-4371-3 home dir : /var/lib/rabbitmq config file(s) : /etc/rabbitmq/rabbitmq.config cookie hash : esZsDxSN6VGbi9JkMSxNZA== log(s) : /var/log/contrail/config-database-rabbitmq/rabbitmq.log : /var/log/rabbitmq/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3_upgrade.log database dir : /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3 2025-10-20 05:12:59.308 [info] <0.253.0> Running boot step pre_boot defined by app rabbit 2025-10-20 05:12:59.309 [info] <0.253.0> Running boot step rabbit_core_metrics defined by app rabbit 2025-10-20 05:12:59.309 [info] <0.253.0> Running boot step rabbit_alarm defined by app rabbit 2025-10-20 05:12:59.318 [info] <0.259.0> Memory high watermark set to 12637 MiB (13251084288 bytes) of 15796 MiB (16563855360 bytes) total 2025-10-20 05:12:59.329 [info] <0.261.0> Enabling free disk space monitoring 2025-10-20 05:12:59.330 [info] <0.261.0> Disk free limit set to 50MB 2025-10-20 05:12:59.337 [info] <0.253.0> Running boot step code_server_cache defined by app rabbit 2025-10-20 05:12:59.337 [info] <0.253.0> Running boot step file_handle_cache defined by app rabbit 2025-10-20 05:12:59.338 [info] <0.264.0> Limiting to approx 65436 file handles (58890 sockets) 2025-10-20 05:12:59.338 [info] <0.265.0> FHC read buffering: OFF 2025-10-20 05:12:59.338 [info] <0.265.0> FHC write buffering: ON 2025-10-20 05:12:59.338 [info] <0.253.0> Running boot step worker_pool defined by app rabbit 2025-10-20 05:12:59.339 [info] <0.254.0> Will use 4 processes for default worker pool 2025-10-20 05:12:59.339 [info] <0.254.0> Starting worker pool 'worker_pool' with 4 processes in it 2025-10-20 05:12:59.339 [info] <0.253.0> Running boot step database defined by app rabbit 2025-10-20 05:12:59.339 [info] <0.253.0> Node database directory at /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3 is empty. Assuming we need to join an existing cluster or initialise from scratch... 2025-10-20 05:12:59.339 [info] <0.253.0> Configured peer discovery backend: rabbit_peer_discovery_classic_config 2025-10-20 05:12:59.339 [info] <0.253.0> Will try to lock with peer discovery backend rabbit_peer_discovery_classic_config 2025-10-20 05:12:59.339 [info] <0.253.0> Peer discovery backend does not support locking, falling back to randomized delay 2025-10-20 05:12:59.339 [info] <0.253.0> Peer discovery backend rabbit_peer_discovery_classic_config supports registration. 2025-10-20 05:12:59.339 [info] <0.253.0> Will wait for 5000 milliseconds before proceeding with registration... 2025-10-20 05:13:04.340 [info] <0.253.0> All discovered existing cluster peers: contrail@cn-jenkins-deploy-platform-ansible-os-4371-1, contrail@cn-jenkins-deploy-platform-ansible-os-4371-2, contrail@cn-jenkins-deploy-platform-ansible-os-4371-3 2025-10-20 05:13:04.340 [info] <0.253.0> Peer nodes we can cluster with: contrail@cn-jenkins-deploy-platform-ansible-os-4371-1, contrail@cn-jenkins-deploy-platform-ansible-os-4371-2 2025-10-20 05:13:04.341 [warning] <0.253.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {error,mnesia_not_running} 2025-10-20 05:13:04.352 [info] <0.253.0> Node 'contrail@cn-jenkins-deploy-platform-ansible-os-4371-2' selected for auto-clustering 2025-10-20 05:13:04.365 [error] <0.252.0> CRASH REPORT Process <0.252.0> with 0 neighbours exited with reason: no case clause matching {{error,{merge_schema_failed,"Bad cookie in table definition rabbit_topic_trie_binding: 'contrail@cn-jenkins-deploy-platform-ansible-os-4371-3' = {cstruct,rabbit_topic_trie_binding,ordered_set,['contrail@cn-jenkins-deploy-platform-ansible-os-4371-1'],[],[],[],0,read_write,false,[],[],false,topic_trie_binding,[trie_binding,value],[],[],[],{{1760936694524366637,-576460752303421535,1},'contrail@cn-jenkins-deploy-platform-ansible-os-4371-2'},{{3,2},{'contrail@cn-jenkins-deploy-platform-ansible-..."}},...} in rabbit_mnesia:change_extra_db_nodes/2 line 883 in application_master:init/4 line 138 2025-10-20 05:13:04.366 [info] <0.43.0> Application rabbit exited with reason: no case clause matching {{error,{merge_schema_failed,"Bad cookie in table definition rabbit_topic_trie_binding: 'contrail@cn-jenkins-deploy-platform-ansible-os-4371-3' = {cstruct,rabbit_topic_trie_binding,ordered_set,['contrail@cn-jenkins-deploy-platform-ansible-os-4371-1'],[],[],[],0,read_write,false,[],[],false,topic_trie_binding,[trie_binding,value],[],[],[],{{1760936694524366637,-576460752303421535,1},'contrail@cn-jenkins-deploy-platform-ansible-os-4371-2'},{{3,2},{'contrail@cn-jenkins-deploy-platform-ansible-..."}},...} in rabbit_mnesia:change_extra_db_nodes/2 line 883 2025-10-20 05:13:52.634 [info] <0.8.0> Log file opened with Lager 2025-10-20 05:13:56.722 [info] <0.8.0> Feature flags: list of feature flags found: 2025-10-20 05:13:56.722 [info] <0.8.0> Feature flags: feature flag states written to disk: yes 2025-10-20 05:13:56.747 [info] <0.240.0> Starting RabbitMQ 3.7.28 on Erlang 21.3.8.21 Copyright (c) 2007-2020 Pivotal Software, Inc. Licensed under the MPL. See https://www.rabbitmq.com/ 2025-10-20 05:13:56.747 [info] <0.240.0> node : contrail@cn-jenkins-deploy-platform-ansible-os-4371-3 home dir : /var/lib/rabbitmq config file(s) : /etc/rabbitmq/rabbitmq.config cookie hash : esZsDxSN6VGbi9JkMSxNZA== log(s) : /var/log/contrail/config-database-rabbitmq/rabbitmq.log : /var/log/rabbitmq/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3_upgrade.log database dir : /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3 2025-10-20 05:13:56.761 [info] <0.240.0> Running boot step pre_boot defined by app rabbit 2025-10-20 05:13:56.761 [info] <0.240.0> Running boot step rabbit_core_metrics defined by app rabbit 2025-10-20 05:13:56.761 [info] <0.240.0> Running boot step rabbit_alarm defined by app rabbit 2025-10-20 05:13:56.768 [info] <0.246.0> Memory high watermark set to 12637 MiB (13251084288 bytes) of 15796 MiB (16563855360 bytes) total 2025-10-20 05:13:56.778 [info] <0.248.0> Enabling free disk space monitoring 2025-10-20 05:13:56.778 [info] <0.248.0> Disk free limit set to 50MB 2025-10-20 05:13:56.784 [info] <0.240.0> Running boot step code_server_cache defined by app rabbit 2025-10-20 05:13:56.784 [info] <0.240.0> Running boot step file_handle_cache defined by app rabbit 2025-10-20 05:13:56.785 [info] <0.251.0> Limiting to approx 65436 file handles (58890 sockets) 2025-10-20 05:13:56.785 [info] <0.252.0> FHC read buffering: OFF 2025-10-20 05:13:56.785 [info] <0.252.0> FHC write buffering: ON 2025-10-20 05:13:56.786 [info] <0.240.0> Running boot step worker_pool defined by app rabbit 2025-10-20 05:13:56.786 [info] <0.241.0> Will use 4 processes for default worker pool 2025-10-20 05:13:56.786 [info] <0.241.0> Starting worker pool 'worker_pool' with 4 processes in it 2025-10-20 05:13:56.786 [info] <0.240.0> Running boot step database defined by app rabbit 2025-10-20 05:13:56.787 [info] <0.240.0> Node database directory at /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3 is empty. Assuming we need to join an existing cluster or initialise from scratch... 2025-10-20 05:13:56.787 [info] <0.240.0> Configured peer discovery backend: rabbit_peer_discovery_classic_config 2025-10-20 05:13:56.787 [info] <0.240.0> Will try to lock with peer discovery backend rabbit_peer_discovery_classic_config 2025-10-20 05:13:56.787 [info] <0.240.0> Peer discovery backend does not support locking, falling back to randomized delay 2025-10-20 05:13:56.787 [info] <0.240.0> Peer discovery backend rabbit_peer_discovery_classic_config supports registration. 2025-10-20 05:13:56.787 [info] <0.240.0> Will wait for 12512 milliseconds before proceeding with registration... 2025-10-20 05:14:09.300 [info] <0.240.0> All discovered existing cluster peers: contrail@cn-jenkins-deploy-platform-ansible-os-4371-1, contrail@cn-jenkins-deploy-platform-ansible-os-4371-2, contrail@cn-jenkins-deploy-platform-ansible-os-4371-3 2025-10-20 05:14:09.300 [info] <0.240.0> Peer nodes we can cluster with: contrail@cn-jenkins-deploy-platform-ansible-os-4371-1, contrail@cn-jenkins-deploy-platform-ansible-os-4371-2 2025-10-20 05:14:09.301 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:14:09.303 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {badrpc,nodedown} 2025-10-20 05:14:09.303 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 9 retries left... 2025-10-20 05:14:09.805 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:14:09.806 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {badrpc,nodedown} 2025-10-20 05:14:09.806 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 8 retries left... 2025-10-20 05:14:10.308 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:14:10.310 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {badrpc,nodedown} 2025-10-20 05:14:10.310 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 7 retries left... 2025-10-20 05:14:10.811 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:14:10.812 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {badrpc,nodedown} 2025-10-20 05:14:10.812 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 6 retries left... 2025-10-20 05:14:11.314 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:14:11.315 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {badrpc,nodedown} 2025-10-20 05:14:11.315 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 5 retries left... 2025-10-20 05:14:11.817 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:14:11.818 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {badrpc,nodedown} 2025-10-20 05:14:11.818 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 4 retries left... 2025-10-20 05:14:12.320 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:14:12.321 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {badrpc,nodedown} 2025-10-20 05:14:12.322 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 3 retries left... 2025-10-20 05:14:12.823 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:14:12.888 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:14:12.888 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 2 retries left... 2025-10-20 05:14:13.390 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:14:13.393 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:14:13.393 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 1 retries left... 2025-10-20 05:14:13.896 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:14:13.898 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:14:13.898 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 0 retries left... 2025-10-20 05:14:14.399 [warning] <0.240.0> Could not successfully contact any node of: contrail@cn-jenkins-deploy-platform-ansible-os-4371-1,contrail@cn-jenkins-deploy-platform-ansible-os-4371-2 (as in Erlang distribution). Starting as a blank standalone node... 2025-10-20 05:14:14.401 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-10-20 05:14:14.612 [info] <0.240.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-10-20 05:14:14.612 [info] <0.240.0> Successfully synced tables from a peer 2025-10-20 05:14:14.687 [info] <0.240.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-10-20 05:14:14.687 [info] <0.240.0> Successfully synced tables from a peer 2025-10-20 05:14:14.832 [info] <0.240.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-10-20 05:14:14.832 [info] <0.240.0> Successfully synced tables from a peer 2025-10-20 05:14:14.832 [info] <0.240.0> Will register with peer discovery backend rabbit_peer_discovery_classic_config 2025-10-20 05:14:14.832 [info] <0.240.0> Running boot step database_sync defined by app rabbit 2025-10-20 05:14:14.832 [info] <0.240.0> Running boot step feature_flags defined by app rabbit 2025-10-20 05:14:14.832 [info] <0.240.0> Running boot step codec_correctness_check defined by app rabbit 2025-10-20 05:14:14.832 [info] <0.240.0> Running boot step external_infrastructure defined by app rabbit 2025-10-20 05:14:14.833 [info] <0.240.0> Running boot step rabbit_registry defined by app rabbit 2025-10-20 05:14:14.834 [info] <0.240.0> Running boot step rabbit_auth_mechanism_cr_demo defined by app rabbit 2025-10-20 05:14:14.834 [info] <0.240.0> Running boot step rabbit_queue_location_random defined by app rabbit 2025-10-20 05:14:14.834 [info] <0.240.0> Running boot step rabbit_event defined by app rabbit 2025-10-20 05:14:14.835 [info] <0.240.0> Running boot step rabbit_auth_mechanism_amqplain defined by app rabbit 2025-10-20 05:14:14.835 [info] <0.240.0> Running boot step rabbit_auth_mechanism_plain defined by app rabbit 2025-10-20 05:14:14.835 [info] <0.240.0> Running boot step rabbit_exchange_type_direct defined by app rabbit 2025-10-20 05:14:14.835 [info] <0.240.0> Running boot step rabbit_exchange_type_fanout defined by app rabbit 2025-10-20 05:14:14.835 [info] <0.240.0> Running boot step rabbit_exchange_type_headers defined by app rabbit 2025-10-20 05:14:14.835 [info] <0.240.0> Running boot step rabbit_exchange_type_topic defined by app rabbit 2025-10-20 05:14:14.835 [info] <0.240.0> Running boot step rabbit_mirror_queue_mode_all defined by app rabbit 2025-10-20 05:14:14.835 [info] <0.240.0> Running boot step rabbit_mirror_queue_mode_exactly defined by app rabbit 2025-10-20 05:14:14.835 [info] <0.240.0> Running boot step rabbit_mirror_queue_mode_nodes defined by app rabbit 2025-10-20 05:14:14.836 [info] <0.240.0> Running boot step rabbit_priority_queue defined by app rabbit 2025-10-20 05:14:14.836 [info] <0.240.0> Priority queues enabled, real BQ is rabbit_variable_queue 2025-10-20 05:14:14.836 [info] <0.240.0> Running boot step rabbit_queue_location_client_local defined by app rabbit 2025-10-20 05:14:14.836 [info] <0.240.0> Running boot step rabbit_queue_location_min_masters defined by app rabbit 2025-10-20 05:14:14.836 [info] <0.240.0> Running boot step kernel_ready defined by app rabbit 2025-10-20 05:14:14.836 [info] <0.240.0> Running boot step rabbit_sysmon_minder defined by app rabbit 2025-10-20 05:14:14.837 [info] <0.240.0> Running boot step rabbit_epmd_monitor defined by app rabbit 2025-10-20 05:14:14.838 [info] <0.485.0> epmd monitor knows us, inter-node communication (distribution) port: 25673 2025-10-20 05:14:14.838 [info] <0.240.0> Running boot step guid_generator defined by app rabbit 2025-10-20 05:14:14.846 [info] <0.240.0> Running boot step rabbit_node_monitor defined by app rabbit 2025-10-20 05:14:14.846 [info] <0.489.0> Starting rabbit_node_monitor 2025-10-20 05:14:14.846 [info] <0.240.0> Running boot step delegate_sup defined by app rabbit 2025-10-20 05:14:14.848 [info] <0.240.0> Running boot step rabbit_memory_monitor defined by app rabbit 2025-10-20 05:14:14.849 [info] <0.240.0> Running boot step core_initialized defined by app rabbit 2025-10-20 05:14:14.849 [info] <0.240.0> Running boot step upgrade_queues defined by app rabbit 2025-10-20 05:14:14.924 [info] <0.240.0> message_store upgrades: 1 to apply 2025-10-20 05:14:14.925 [info] <0.240.0> message_store upgrades: Applying rabbit_variable_queue:move_messages_to_vhost_store 2025-10-20 05:14:14.925 [info] <0.240.0> message_store upgrades: No durable queues found. Skipping message store migration 2025-10-20 05:14:14.925 [info] <0.240.0> message_store upgrades: Removing the old message store data 2025-10-20 05:14:14.929 [info] <0.240.0> message_store upgrades: All upgrades applied successfully 2025-10-20 05:14:14.988 [info] <0.240.0> Running boot step rabbit_connection_tracking defined by app rabbit 2025-10-20 05:14:14.988 [info] <0.240.0> Running boot step rabbit_connection_tracking_handler defined by app rabbit 2025-10-20 05:14:14.989 [info] <0.240.0> Running boot step rabbit_exchange_parameters defined by app rabbit 2025-10-20 05:14:14.989 [info] <0.240.0> Running boot step rabbit_mirror_queue_misc defined by app rabbit 2025-10-20 05:14:14.989 [info] <0.240.0> Running boot step rabbit_policies defined by app rabbit 2025-10-20 05:14:14.990 [info] <0.240.0> Running boot step rabbit_policy defined by app rabbit 2025-10-20 05:14:14.991 [info] <0.240.0> Running boot step rabbit_queue_location_validator defined by app rabbit 2025-10-20 05:14:14.991 [info] <0.240.0> Running boot step rabbit_vhost_limit defined by app rabbit 2025-10-20 05:14:14.991 [info] <0.240.0> Running boot step rabbit_mgmt_reset_handler defined by app rabbitmq_management 2025-10-20 05:14:14.991 [info] <0.240.0> Running boot step rabbit_mgmt_db_handler defined by app rabbitmq_management_agent 2025-10-20 05:14:14.992 [warning] <0.240.0> force_fine_statistics set to true; ignored. Replaced by {rates_mode, none} in the rabbitmq_management application. 2025-10-20 05:14:14.992 [info] <0.240.0> Management plugin: using rates mode 'basic' 2025-10-20 05:14:14.993 [info] <0.240.0> Running boot step recovery defined by app rabbit 2025-10-20 05:14:14.994 [info] <0.240.0> Running boot step load_definitions defined by app rabbitmq_management 2025-10-20 05:14:14.997 [info] <0.240.0> Applying definitions from /etc/rabbitmq/definitions.json 2025-10-20 05:14:14.997 [info] <0.240.0> Asked to import definitions. Acting user: rmq-internal 2025-10-20 05:14:14.997 [info] <0.240.0> Importing users... 2025-10-20 05:14:14.997 [info] <0.240.0> Creating user 'guest' 2025-10-20 05:14:15.008 [info] <0.240.0> Setting user tags for user 'guest' to [administrator] 2025-10-20 05:14:15.010 [info] <0.240.0> Importing vhosts... 2025-10-20 05:14:15.010 [info] <0.240.0> Adding vhost '/' 2025-10-20 05:14:15.032 [info] <0.537.0> Making sure data directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists 2025-10-20 05:14:15.038 [info] <0.537.0> Starting message stores for vhost '/' 2025-10-20 05:14:15.038 [info] <0.541.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index 2025-10-20 05:14:15.041 [info] <0.537.0> Started message store of type transient for vhost '/' 2025-10-20 05:14:15.041 [info] <0.544.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index 2025-10-20 05:14:15.043 [warning] <0.544.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": rebuilding indices from scratch 2025-10-20 05:14:15.050 [info] <0.537.0> Started message store of type persistent for vhost '/' 2025-10-20 05:14:15.051 [info] <0.240.0> Importing user permissions... 2025-10-20 05:14:15.051 [info] <0.240.0> Setting permissions for 'guest' in '/' to '.*', '.*', '.*' 2025-10-20 05:14:15.056 [info] <0.240.0> Importing topic permissions... 2025-10-20 05:14:15.056 [info] <0.240.0> Importing parameters... 2025-10-20 05:14:15.056 [info] <0.240.0> Importing global parameters... 2025-10-20 05:14:15.056 [info] <0.240.0> Importing policies... 2025-10-20 05:14:15.066 [info] <0.240.0> Importing queues... 2025-10-20 05:14:15.066 [info] <0.240.0> Importing exchanges... 2025-10-20 05:14:15.066 [info] <0.240.0> Importing bindings... 2025-10-20 05:14:15.066 [info] <0.240.0> Running boot step empty_db_check defined by app rabbit 2025-10-20 05:14:15.066 [info] <0.240.0> Running boot step rabbit_looking_glass defined by app rabbit 2025-10-20 05:14:15.066 [info] <0.240.0> Running boot step rabbit_core_metrics_gc defined by app rabbit 2025-10-20 05:14:15.066 [info] <0.240.0> Running boot step background_gc defined by app rabbit 2025-10-20 05:14:15.066 [info] <0.240.0> Running boot step connection_tracking defined by app rabbit 2025-10-20 05:14:15.080 [info] <0.240.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-4371-3' 2025-10-20 05:14:15.091 [info] <0.240.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-4371-3' 2025-10-20 05:14:15.091 [info] <0.240.0> Running boot step routing_ready defined by app rabbit 2025-10-20 05:14:15.091 [info] <0.240.0> Running boot step pre_flight defined by app rabbit 2025-10-20 05:14:15.091 [info] <0.240.0> Running boot step notify_cluster defined by app rabbit 2025-10-20 05:14:15.091 [info] <0.240.0> Running boot step networking defined by app rabbit 2025-10-20 05:14:15.096 [info] <0.590.0> started TCP listener on [::]:5673 2025-10-20 05:14:15.097 [info] <0.240.0> Running boot step cluster_name defined by app rabbit 2025-10-20 05:14:15.097 [info] <0.240.0> Running boot step direct_client defined by app rabbit 2025-10-20 05:14:15.097 [info] <0.240.0> Running boot step os_signal_handler defined by app rabbit 2025-10-20 05:14:15.097 [info] <0.592.0> Swapping OS signal event handler (erl_signal_server) for our own 2025-10-20 05:14:15.215 [info] <0.642.0> Management plugin: HTTP (non-TLS) listener started on port 15673 2025-10-20 05:14:15.216 [info] <0.748.0> Statistics database started. 2025-10-20 05:14:15.216 [info] <0.747.0> Starting worker pool 'management_worker_pool' with 3 processes in it 2025-10-20 05:14:15.218 [notice] <0.106.0> Changed loghwm of /var/log/contrail/config-database-rabbitmq/rabbitmq.log to 50 2025-10-20 05:14:15.522 [info] <0.8.0> Server startup complete; 3 plugins started. * rabbitmq_management * rabbitmq_management_agent * rabbitmq_web_dispatch 2025-10-20 05:14:15.584 [info] <0.759.0> RabbitMQ is asked to stop... 2025-10-20 05:14:15.816 [info] <0.759.0> Stopping RabbitMQ applications and their dependencies in the following order: rabbitmq_management amqp_client rabbitmq_web_dispatch cowboy cowlib rabbitmq_management_agent rabbit mnesia rabbit_common sysmon_handler os_mon 2025-10-20 05:14:15.816 [info] <0.759.0> Stopping application 'rabbitmq_management' 2025-10-20 05:14:15.818 [warning] <0.598.0> RabbitMQ HTTP listener registry could not find context rabbitmq_management_tls 2025-10-20 05:14:15.820 [info] <0.43.0> Application rabbitmq_management exited with reason: stopped 2025-10-20 05:14:15.820 [info] <0.759.0> Stopping application 'amqp_client' 2025-10-20 05:14:15.821 [info] <0.759.0> Stopping application 'rabbitmq_web_dispatch' 2025-10-20 05:14:15.822 [info] <0.43.0> Application amqp_client exited with reason: stopped 2025-10-20 05:14:15.826 [info] <0.759.0> Stopping application 'cowboy' 2025-10-20 05:14:15.826 [info] <0.43.0> Application rabbitmq_web_dispatch exited with reason: stopped 2025-10-20 05:14:15.828 [info] <0.43.0> Application cowboy exited with reason: stopped 2025-10-20 05:14:15.828 [info] <0.759.0> Stopping application 'cowlib' 2025-10-20 05:14:15.828 [info] <0.43.0> Application cowlib exited with reason: stopped 2025-10-20 05:14:15.828 [info] <0.759.0> Stopping application 'rabbitmq_management_agent' 2025-10-20 05:14:15.830 [info] <0.43.0> Application rabbitmq_management_agent exited with reason: stopped 2025-10-20 05:14:15.830 [info] <0.759.0> Stopping application 'rabbit' 2025-10-20 05:14:15.830 [info] <0.240.0> Will unregister with peer discovery backend rabbit_peer_discovery_classic_config 2025-10-20 05:14:15.830 [info] <0.590.0> stopped TCP listener on [::]:5673 2025-10-20 05:14:15.831 [info] <0.544.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' 2025-10-20 05:14:15.831 [info] <0.514.0> Closing all connections in vhost '/' on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4371-3' because the vhost is stopping 2025-10-20 05:14:15.848 [info] <0.544.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' is stopped 2025-10-20 05:14:15.848 [info] <0.541.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' 2025-10-20 05:14:15.853 [info] <0.541.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' is stopped 2025-10-20 05:14:15.856 [info] <0.43.0> Application rabbit exited with reason: stopped 2025-10-20 05:14:15.856 [info] <0.759.0> Stopping application 'mnesia' 2025-10-20 05:14:15.861 [info] <0.759.0> Stopping application 'rabbit_common' 2025-10-20 05:14:15.861 [info] <0.759.0> Stopping application 'sysmon_handler' 2025-10-20 05:14:15.861 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-10-20 05:14:15.862 [info] <0.43.0> Application rabbit_common exited with reason: stopped 2025-10-20 05:14:15.864 [info] <0.43.0> Application sysmon_handler exited with reason: stopped 2025-10-20 05:14:15.864 [info] <0.759.0> Stopping application 'os_mon' 2025-10-20 05:14:15.866 [info] <0.759.0> Successfully stopped RabbitMQ and its dependencies 2025-10-20 05:14:15.866 [info] <0.43.0> Application os_mon exited with reason: stopped 2025-10-20 05:14:39.981 [info] <0.778.0> Resetting Rabbit forcefully 2025-10-20 05:14:41.738 [info] <0.792.0> RabbitMQ is asked to stop... 2025-10-20 05:14:41.840 [info] <0.792.0> Stopping RabbitMQ applications and their dependencies in the following order: rabbitmq_management amqp_client rabbitmq_management_agent rabbitmq_web_dispatch rabbit mnesia rabbit_common sysmon_handler os_mon 2025-10-20 05:14:41.840 [info] <0.792.0> Stopping application 'rabbitmq_management' 2025-10-20 05:14:41.840 [info] <0.792.0> Stopping application 'amqp_client' 2025-10-20 05:14:41.840 [info] <0.792.0> Stopping application 'rabbitmq_management_agent' 2025-10-20 05:14:41.840 [info] <0.792.0> Stopping application 'rabbitmq_web_dispatch' 2025-10-20 05:14:41.840 [info] <0.792.0> Stopping application 'rabbit' 2025-10-20 05:14:41.840 [info] <0.792.0> Stopping application 'mnesia' 2025-10-20 05:14:41.840 [info] <0.792.0> Stopping application 'rabbit_common' 2025-10-20 05:14:41.841 [info] <0.792.0> Stopping application 'sysmon_handler' 2025-10-20 05:14:41.841 [info] <0.792.0> Stopping application 'os_mon' 2025-10-20 05:14:41.841 [info] <0.792.0> Successfully stopped RabbitMQ and its dependencies 2025-10-20 05:14:41.841 [info] <0.792.0> Halting Erlang VM with the following applications: ranch ssl public_key asn1 observer_cli tools credentials_obfuscation crypto stdout_formatter jsx recon inets xmerl lager goldrush compiler syntax_tools sasl stdlib kernel 2025-10-20 05:14:49.968 [info] <0.8.0> Log file opened with Lager 2025-10-20 05:14:52.893 [info] <0.8.0> Feature flags: list of feature flags found: 2025-10-20 05:14:52.893 [info] <0.8.0> Feature flags: feature flag states written to disk: yes 2025-10-20 05:14:52.921 [info] <0.250.0> Starting RabbitMQ 3.7.28 on Erlang 21.3.8.21 Copyright (c) 2007-2020 Pivotal Software, Inc. Licensed under the MPL. See https://www.rabbitmq.com/ 2025-10-20 05:14:52.923 [info] <0.250.0> node : contrail@cn-jenkins-deploy-platform-ansible-os-4371-3 home dir : /var/lib/rabbitmq config file(s) : /etc/rabbitmq/rabbitmq.config cookie hash : esZsDxSN6VGbi9JkMSxNZA== log(s) : /var/log/contrail/config-database-rabbitmq/rabbitmq.log : /var/log/rabbitmq/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3_upgrade.log database dir : /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3 2025-10-20 05:14:52.940 [info] <0.250.0> Running boot step pre_boot defined by app rabbit 2025-10-20 05:14:52.940 [info] <0.250.0> Running boot step rabbit_core_metrics defined by app rabbit 2025-10-20 05:14:52.940 [info] <0.250.0> Running boot step rabbit_alarm defined by app rabbit 2025-10-20 05:14:52.946 [info] <0.256.0> Memory high watermark set to 12637 MiB (13251084288 bytes) of 15796 MiB (16563855360 bytes) total 2025-10-20 05:14:52.951 [info] <0.258.0> Enabling free disk space monitoring 2025-10-20 05:14:52.951 [info] <0.258.0> Disk free limit set to 50MB 2025-10-20 05:14:52.957 [info] <0.250.0> Running boot step code_server_cache defined by app rabbit 2025-10-20 05:14:52.957 [info] <0.250.0> Running boot step file_handle_cache defined by app rabbit 2025-10-20 05:14:52.958 [info] <0.261.0> Limiting to approx 65436 file handles (58890 sockets) 2025-10-20 05:14:52.958 [info] <0.262.0> FHC read buffering: OFF 2025-10-20 05:14:52.958 [info] <0.262.0> FHC write buffering: ON 2025-10-20 05:14:52.958 [info] <0.250.0> Running boot step worker_pool defined by app rabbit 2025-10-20 05:14:52.958 [info] <0.251.0> Will use 4 processes for default worker pool 2025-10-20 05:14:52.958 [info] <0.251.0> Starting worker pool 'worker_pool' with 4 processes in it 2025-10-20 05:14:52.959 [info] <0.250.0> Running boot step database defined by app rabbit 2025-10-20 05:14:52.959 [info] <0.250.0> Node database directory at /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3 is empty. Assuming we need to join an existing cluster or initialise from scratch... 2025-10-20 05:14:52.959 [info] <0.250.0> Configured peer discovery backend: rabbit_peer_discovery_classic_config 2025-10-20 05:14:52.959 [info] <0.250.0> Will try to lock with peer discovery backend rabbit_peer_discovery_classic_config 2025-10-20 05:14:52.959 [info] <0.250.0> Peer discovery backend does not support locking, falling back to randomized delay 2025-10-20 05:14:52.959 [info] <0.250.0> Peer discovery backend rabbit_peer_discovery_classic_config supports registration. 2025-10-20 05:14:52.959 [info] <0.250.0> Will wait for 22473 milliseconds before proceeding with registration... 2025-10-20 05:15:15.433 [info] <0.250.0> All discovered existing cluster peers: contrail@cn-jenkins-deploy-platform-ansible-os-4371-1, contrail@cn-jenkins-deploy-platform-ansible-os-4371-2, contrail@cn-jenkins-deploy-platform-ansible-os-4371-3 2025-10-20 05:15:15.433 [info] <0.250.0> Peer nodes we can cluster with: contrail@cn-jenkins-deploy-platform-ansible-os-4371-1, contrail@cn-jenkins-deploy-platform-ansible-os-4371-2 2025-10-20 05:15:15.435 [warning] <0.250.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:15:15.436 [warning] <0.250.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:15:15.436 [error] <0.250.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 9 retries left... 2025-10-20 05:15:15.938 [warning] <0.250.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:15:15.939 [warning] <0.250.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:15:15.939 [error] <0.250.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 8 retries left... 2025-10-20 05:15:16.440 [warning] <0.250.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:15:16.441 [warning] <0.250.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:15:16.441 [error] <0.250.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 7 retries left... 2025-10-20 05:15:16.945 [warning] <0.250.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:15:16.946 [warning] <0.250.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:15:16.946 [error] <0.250.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 6 retries left... 2025-10-20 05:15:17.448 [warning] <0.250.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:15:17.449 [warning] <0.250.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:15:17.449 [error] <0.250.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 5 retries left... 2025-10-20 05:15:17.951 [warning] <0.250.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:15:17.952 [warning] <0.250.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:15:17.952 [error] <0.250.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 4 retries left... 2025-10-20 05:15:18.455 [warning] <0.250.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:15:18.455 [warning] <0.250.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:15:18.455 [error] <0.250.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 3 retries left... 2025-10-20 05:15:18.960 [warning] <0.250.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:15:18.960 [warning] <0.250.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:15:18.960 [error] <0.250.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 2 retries left... 2025-10-20 05:15:19.462 [warning] <0.250.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:15:19.463 [warning] <0.250.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:15:19.463 [error] <0.250.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 1 retries left... 2025-10-20 05:15:19.965 [warning] <0.250.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:15:19.966 [warning] <0.250.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:15:19.966 [error] <0.250.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 0 retries left... 2025-10-20 05:15:20.467 [warning] <0.250.0> Could not successfully contact any node of: contrail@cn-jenkins-deploy-platform-ansible-os-4371-1,contrail@cn-jenkins-deploy-platform-ansible-os-4371-2 (as in Erlang distribution). Starting as a blank standalone node... 2025-10-20 05:15:20.469 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-10-20 05:15:20.607 [info] <0.250.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-10-20 05:15:20.607 [info] <0.250.0> Successfully synced tables from a peer 2025-10-20 05:15:20.639 [info] <0.250.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-10-20 05:15:20.639 [info] <0.250.0> Successfully synced tables from a peer 2025-10-20 05:15:20.672 [info] <0.250.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-10-20 05:15:20.672 [info] <0.250.0> Successfully synced tables from a peer 2025-10-20 05:15:20.673 [info] <0.250.0> Will register with peer discovery backend rabbit_peer_discovery_classic_config 2025-10-20 05:15:20.673 [info] <0.250.0> Running boot step database_sync defined by app rabbit 2025-10-20 05:15:20.673 [info] <0.250.0> Running boot step feature_flags defined by app rabbit 2025-10-20 05:15:20.674 [info] <0.250.0> Running boot step codec_correctness_check defined by app rabbit 2025-10-20 05:15:20.675 [info] <0.250.0> Running boot step external_infrastructure defined by app rabbit 2025-10-20 05:15:20.675 [info] <0.250.0> Running boot step rabbit_registry defined by app rabbit 2025-10-20 05:15:20.676 [info] <0.250.0> Running boot step rabbit_auth_mechanism_cr_demo defined by app rabbit 2025-10-20 05:15:20.676 [info] <0.250.0> Running boot step rabbit_queue_location_random defined by app rabbit 2025-10-20 05:15:20.677 [info] <0.250.0> Running boot step rabbit_event defined by app rabbit 2025-10-20 05:15:20.677 [info] <0.250.0> Running boot step rabbit_auth_mechanism_amqplain defined by app rabbit 2025-10-20 05:15:20.678 [info] <0.250.0> Running boot step rabbit_auth_mechanism_plain defined by app rabbit 2025-10-20 05:15:20.679 [info] <0.250.0> Running boot step rabbit_exchange_type_direct defined by app rabbit 2025-10-20 05:15:20.679 [info] <0.250.0> Running boot step rabbit_exchange_type_fanout defined by app rabbit 2025-10-20 05:15:20.679 [info] <0.250.0> Running boot step rabbit_exchange_type_headers defined by app rabbit 2025-10-20 05:15:20.679 [info] <0.250.0> Running boot step rabbit_exchange_type_topic defined by app rabbit 2025-10-20 05:15:20.680 [info] <0.250.0> Running boot step rabbit_mirror_queue_mode_all defined by app rabbit 2025-10-20 05:15:20.680 [info] <0.250.0> Running boot step rabbit_mirror_queue_mode_exactly defined by app rabbit 2025-10-20 05:15:20.680 [info] <0.250.0> Running boot step rabbit_mirror_queue_mode_nodes defined by app rabbit 2025-10-20 05:15:20.680 [info] <0.250.0> Running boot step rabbit_priority_queue defined by app rabbit 2025-10-20 05:15:20.680 [info] <0.250.0> Priority queues enabled, real BQ is rabbit_variable_queue 2025-10-20 05:15:20.680 [info] <0.250.0> Running boot step rabbit_queue_location_client_local defined by app rabbit 2025-10-20 05:15:20.680 [info] <0.250.0> Running boot step rabbit_queue_location_min_masters defined by app rabbit 2025-10-20 05:15:20.680 [info] <0.250.0> Running boot step kernel_ready defined by app rabbit 2025-10-20 05:15:20.680 [info] <0.250.0> Running boot step rabbit_sysmon_minder defined by app rabbit 2025-10-20 05:15:20.680 [info] <0.250.0> Running boot step rabbit_epmd_monitor defined by app rabbit 2025-10-20 05:15:20.681 [info] <0.469.0> epmd monitor knows us, inter-node communication (distribution) port: 25673 2025-10-20 05:15:20.681 [info] <0.250.0> Running boot step guid_generator defined by app rabbit 2025-10-20 05:15:20.686 [info] <0.250.0> Running boot step rabbit_node_monitor defined by app rabbit 2025-10-20 05:15:20.686 [info] <0.473.0> Starting rabbit_node_monitor 2025-10-20 05:15:20.686 [info] <0.250.0> Running boot step delegate_sup defined by app rabbit 2025-10-20 05:15:20.686 [info] <0.250.0> Running boot step rabbit_memory_monitor defined by app rabbit 2025-10-20 05:15:20.687 [info] <0.250.0> Running boot step core_initialized defined by app rabbit 2025-10-20 05:15:20.687 [info] <0.250.0> Running boot step upgrade_queues defined by app rabbit 2025-10-20 05:15:20.713 [info] <0.250.0> message_store upgrades: 1 to apply 2025-10-20 05:15:20.714 [info] <0.250.0> message_store upgrades: Applying rabbit_variable_queue:move_messages_to_vhost_store 2025-10-20 05:15:20.714 [info] <0.250.0> message_store upgrades: No durable queues found. Skipping message store migration 2025-10-20 05:15:20.714 [info] <0.250.0> message_store upgrades: Removing the old message store data 2025-10-20 05:15:20.714 [info] <0.250.0> message_store upgrades: All upgrades applied successfully 2025-10-20 05:15:20.740 [info] <0.250.0> Running boot step rabbit_connection_tracking defined by app rabbit 2025-10-20 05:15:20.741 [info] <0.250.0> Running boot step rabbit_connection_tracking_handler defined by app rabbit 2025-10-20 05:15:20.741 [info] <0.250.0> Running boot step rabbit_exchange_parameters defined by app rabbit 2025-10-20 05:15:20.741 [info] <0.250.0> Running boot step rabbit_mirror_queue_misc defined by app rabbit 2025-10-20 05:15:20.741 [info] <0.250.0> Running boot step rabbit_policies defined by app rabbit 2025-10-20 05:15:20.742 [info] <0.250.0> Running boot step rabbit_policy defined by app rabbit 2025-10-20 05:15:20.742 [info] <0.250.0> Running boot step rabbit_queue_location_validator defined by app rabbit 2025-10-20 05:15:20.742 [info] <0.250.0> Running boot step rabbit_vhost_limit defined by app rabbit 2025-10-20 05:15:20.742 [info] <0.250.0> Running boot step rabbit_mgmt_reset_handler defined by app rabbitmq_management 2025-10-20 05:15:20.742 [info] <0.250.0> Running boot step rabbit_mgmt_db_handler defined by app rabbitmq_management_agent 2025-10-20 05:15:20.742 [warning] <0.250.0> force_fine_statistics set to true; ignored. Replaced by {rates_mode, none} in the rabbitmq_management application. 2025-10-20 05:15:20.743 [info] <0.250.0> Management plugin: using rates mode 'basic' 2025-10-20 05:15:20.743 [info] <0.250.0> Running boot step recovery defined by app rabbit 2025-10-20 05:15:20.744 [info] <0.250.0> Running boot step load_definitions defined by app rabbitmq_management 2025-10-20 05:15:20.744 [info] <0.250.0> Applying definitions from /etc/rabbitmq/definitions.json 2025-10-20 05:15:20.744 [info] <0.250.0> Asked to import definitions. Acting user: rmq-internal 2025-10-20 05:15:20.744 [info] <0.250.0> Importing users... 2025-10-20 05:15:20.744 [info] <0.250.0> Creating user 'guest' 2025-10-20 05:15:20.750 [info] <0.250.0> Setting user tags for user 'guest' to [administrator] 2025-10-20 05:15:20.752 [info] <0.250.0> Importing vhosts... 2025-10-20 05:15:20.752 [info] <0.250.0> Adding vhost '/' 2025-10-20 05:15:20.780 [info] <0.521.0> Making sure data directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists 2025-10-20 05:15:20.784 [info] <0.521.0> Starting message stores for vhost '/' 2025-10-20 05:15:20.784 [info] <0.525.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index 2025-10-20 05:15:20.785 [info] <0.521.0> Started message store of type transient for vhost '/' 2025-10-20 05:15:20.785 [info] <0.528.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index 2025-10-20 05:15:20.786 [warning] <0.528.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": rebuilding indices from scratch 2025-10-20 05:15:20.787 [info] <0.521.0> Started message store of type persistent for vhost '/' 2025-10-20 05:15:20.788 [info] <0.250.0> Importing user permissions... 2025-10-20 05:15:20.789 [info] <0.250.0> Setting permissions for 'guest' in '/' to '.*', '.*', '.*' 2025-10-20 05:15:20.792 [info] <0.250.0> Importing topic permissions... 2025-10-20 05:15:20.793 [info] <0.250.0> Importing parameters... 2025-10-20 05:15:20.793 [info] <0.250.0> Importing global parameters... 2025-10-20 05:15:20.793 [info] <0.250.0> Importing policies... 2025-10-20 05:15:20.800 [info] <0.250.0> Importing queues... 2025-10-20 05:15:20.800 [info] <0.250.0> Importing exchanges... 2025-10-20 05:15:20.800 [info] <0.250.0> Importing bindings... 2025-10-20 05:15:20.800 [info] <0.250.0> Running boot step empty_db_check defined by app rabbit 2025-10-20 05:15:20.800 [info] <0.250.0> Running boot step rabbit_looking_glass defined by app rabbit 2025-10-20 05:15:20.800 [info] <0.250.0> Running boot step rabbit_core_metrics_gc defined by app rabbit 2025-10-20 05:15:20.800 [info] <0.250.0> Running boot step background_gc defined by app rabbit 2025-10-20 05:15:20.801 [info] <0.250.0> Running boot step connection_tracking defined by app rabbit 2025-10-20 05:15:20.806 [info] <0.250.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-4371-3' 2025-10-20 05:15:20.812 [info] <0.250.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-4371-3' 2025-10-20 05:15:20.812 [info] <0.250.0> Running boot step routing_ready defined by app rabbit 2025-10-20 05:15:20.812 [info] <0.250.0> Running boot step pre_flight defined by app rabbit 2025-10-20 05:15:20.812 [info] <0.250.0> Running boot step notify_cluster defined by app rabbit 2025-10-20 05:15:20.812 [info] <0.250.0> Running boot step networking defined by app rabbit 2025-10-20 05:15:20.814 [info] <0.574.0> started TCP listener on [::]:5673 2025-10-20 05:15:20.814 [info] <0.250.0> Running boot step cluster_name defined by app rabbit 2025-10-20 05:15:20.814 [info] <0.250.0> Running boot step direct_client defined by app rabbit 2025-10-20 05:15:20.814 [info] <0.250.0> Running boot step os_signal_handler defined by app rabbit 2025-10-20 05:15:20.814 [info] <0.576.0> Swapping OS signal event handler (erl_signal_server) for our own 2025-10-20 05:15:20.855 [info] <0.626.0> Management plugin: HTTP (non-TLS) listener started on port 15673 2025-10-20 05:15:20.855 [info] <0.732.0> Statistics database started. 2025-10-20 05:15:20.855 [info] <0.731.0> Starting worker pool 'management_worker_pool' with 3 processes in it 2025-10-20 05:15:20.856 [notice] <0.106.0> Changed loghwm of /var/log/contrail/config-database-rabbitmq/rabbitmq.log to 50 2025-10-20 05:15:20.970 [info] <0.8.0> Server startup complete; 3 plugins started. * rabbitmq_management * rabbitmq_web_dispatch * rabbitmq_management_agent 2025-10-20 05:15:20.987 [info] <0.742.0> accepting AMQP connection <0.742.0> (10.0.0.49:38642 -> 10.0.0.49:5673) 2025-10-20 05:15:20.987 [info] <0.745.0> accepting AMQP connection <0.745.0> (10.0.0.48:49624 -> 10.0.0.49:5673) 2025-10-20 05:15:20.988 [info] <0.749.0> accepting AMQP connection <0.749.0> (10.0.0.254:40846 -> 10.0.0.49:5673) 2025-10-20 05:15:20.988 [info] <0.752.0> accepting AMQP connection <0.752.0> (10.0.0.254:40854 -> 10.0.0.49:5673) 2025-10-20 05:15:20.989 [info] <0.745.0> connection <0.745.0> (10.0.0.48:49624 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:15:20.990 [info] <0.760.0> accepting AMQP connection <0.760.0> (10.0.0.48:49636 -> 10.0.0.49:5673) 2025-10-20 05:15:20.991 [info] <0.764.0> accepting AMQP connection <0.764.0> (10.0.0.48:49638 -> 10.0.0.49:5673) 2025-10-20 05:15:20.991 [info] <0.752.0> connection <0.752.0> (10.0.0.254:40854 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:15:20.991 [info] <0.768.0> accepting AMQP connection <0.768.0> (10.0.0.254:40870 -> 10.0.0.49:5673) 2025-10-20 05:15:20.992 [info] <0.773.0> accepting AMQP connection <0.773.0> (10.0.0.49:38654 -> 10.0.0.49:5673) 2025-10-20 05:15:20.992 [info] <0.749.0> connection <0.749.0> (10.0.0.254:40846 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:15:20.992 [info] <0.780.0> accepting AMQP connection <0.780.0> (10.0.0.49:38658 -> 10.0.0.49:5673) 2025-10-20 05:15:20.993 [info] <0.760.0> connection <0.760.0> (10.0.0.48:49636 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:15:20.995 [info] <0.780.0> connection <0.780.0> (10.0.0.49:38658 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:15:20.995 [info] <0.768.0> connection <0.768.0> (10.0.0.254:40870 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:15:20.995 [info] <0.773.0> connection <0.773.0> (10.0.0.49:38654 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:15:20.996 [info] <0.764.0> connection <0.764.0> (10.0.0.48:49638 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:15:20.997 [info] <0.742.0> connection <0.742.0> (10.0.0.49:38642 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:15:21.017 [info] <0.902.0> accepting AMQP connection <0.902.0> (10.0.0.254:40882 -> 10.0.0.49:5673) 2025-10-20 05:15:21.025 [info] <0.902.0> connection <0.902.0> (10.0.0.254:40882 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:15:21.025 [info] <0.913.0> accepting AMQP connection <0.913.0> (10.0.0.48:49646 -> 10.0.0.49:5673) 2025-10-20 05:15:21.029 [info] <0.913.0> connection <0.913.0> (10.0.0.48:49646 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:15:21.111 [info] <0.944.0> accepting AMQP connection <0.944.0> (10.0.0.254:40898 -> 10.0.0.49:5673) 2025-10-20 05:15:21.113 [info] <0.944.0> connection <0.944.0> (10.0.0.254:40898 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:15:23.565 [info] <0.978.0> RabbitMQ is asked to stop... 2025-10-20 05:15:23.661 [info] <0.978.0> Stopping RabbitMQ applications and their dependencies in the following order: rabbitmq_management amqp_client rabbitmq_web_dispatch cowboy cowlib rabbitmq_management_agent rabbit mnesia rabbit_common sysmon_handler os_mon 2025-10-20 05:15:23.661 [info] <0.978.0> Stopping application 'rabbitmq_management' 2025-10-20 05:15:23.663 [warning] <0.622.0> RabbitMQ HTTP listener registry could not find context rabbitmq_management_tls 2025-10-20 05:15:23.666 [info] <0.978.0> Stopping application 'amqp_client' 2025-10-20 05:15:23.666 [info] <0.43.0> Application rabbitmq_management exited with reason: stopped 2025-10-20 05:15:23.668 [info] <0.978.0> Stopping application 'rabbitmq_web_dispatch' 2025-10-20 05:15:23.668 [info] <0.43.0> Application amqp_client exited with reason: stopped 2025-10-20 05:15:23.671 [info] <0.43.0> Application rabbitmq_web_dispatch exited with reason: stopped 2025-10-20 05:15:23.671 [info] <0.978.0> Stopping application 'cowboy' 2025-10-20 05:15:23.674 [info] <0.43.0> Application cowboy exited with reason: stopped 2025-10-20 05:15:23.674 [info] <0.978.0> Stopping application 'cowlib' 2025-10-20 05:15:23.674 [info] <0.43.0> Application cowlib exited with reason: stopped 2025-10-20 05:15:23.674 [info] <0.978.0> Stopping application 'rabbitmq_management_agent' 2025-10-20 05:15:23.678 [info] <0.978.0> Stopping application 'rabbit' 2025-10-20 05:15:23.678 [info] <0.250.0> Will unregister with peer discovery backend rabbit_peer_discovery_classic_config 2025-10-20 05:15:23.679 [info] <0.43.0> Application rabbitmq_management_agent exited with reason: stopped 2025-10-20 05:15:23.679 [info] <0.574.0> stopped TCP listener on [::]:5673 2025-10-20 05:15:23.680 [error] <0.768.0> Error on AMQP connection <0.768.0> (10.0.0.254:40870 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:15:23.680 [error] <0.764.0> Error on AMQP connection <0.764.0> (10.0.0.48:49638 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:15:23.680 [error] <0.944.0> Error on AMQP connection <0.944.0> (10.0.0.254:40898 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:15:23.680 [error] <0.902.0> Error on AMQP connection <0.902.0> (10.0.0.254:40882 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:15:23.680 [error] <0.745.0> Error on AMQP connection <0.745.0> (10.0.0.48:49624 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:15:23.680 [error] <0.742.0> Error on AMQP connection <0.742.0> (10.0.0.49:38642 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:15:23.680 [error] <0.752.0> Error on AMQP connection <0.752.0> (10.0.0.254:40854 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:15:23.680 [error] <0.913.0> Error on AMQP connection <0.913.0> (10.0.0.48:49646 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:15:23.680 [error] <0.780.0> Error on AMQP connection <0.780.0> (10.0.0.49:38658 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:15:23.680 [error] <0.773.0> Error on AMQP connection <0.773.0> (10.0.0.49:38654 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:15:23.680 [error] <0.760.0> Error on AMQP connection <0.760.0> (10.0.0.48:49636 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:15:23.680 [error] <0.749.0> Error on AMQP connection <0.749.0> (10.0.0.254:40846 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:15:23.682 [error] <0.910.0> Supervisor {<0.910.0>,rabbit_channel_sup_sup} had child channel_sup started with rabbit_channel_sup:start_link() at undefined exit with reason shutdown in context shutdown_error 2025-10-20 05:15:23.684 [error] <0.788.0> Supervisor {<0.788.0>,rabbit_channel_sup_sup} had child channel_sup started with rabbit_channel_sup:start_link() at undefined exit with reason shutdown in context shutdown_error 2025-10-20 05:15:23.684 [error] <0.822.0> Supervisor {<0.822.0>,rabbit_channel_sup_sup} had child channel_sup started with rabbit_channel_sup:start_link() at undefined exit with reason shutdown in context shutdown_error 2025-10-20 05:15:23.689 [info] <0.498.0> Closing all connections in vhost '/' on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4371-3' because the vhost is stopping 2025-10-20 05:15:23.688 [warning] <0.920.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-4371-1.-8082-0' in vhost '/': Stopping all nodes on master shutdown since no synchronised slave is available 2025-10-20 05:15:23.688 [warning] <0.935.0> Mirrored queue 'device_manager.cn-jenkins-deploy-platform-ansible-os-4371-2.' in vhost '/': Stopping all nodes on master shutdown since no synchronised slave is available 2025-10-20 05:15:23.688 [warning] <0.855.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-4371-2.' in vhost '/': Stopping all nodes on master shutdown since no synchronised slave is available 2025-10-20 05:15:23.689 [warning] <0.864.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-4371-3.' in vhost '/': Stopping all nodes on master shutdown since no synchronised slave is available 2025-10-20 05:15:23.689 [warning] <0.832.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-4371-1.' in vhost '/': Stopping all nodes on master shutdown since no synchronised slave is available 2025-10-20 05:15:23.690 [warning] <0.861.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-4371-1' in vhost '/': Stopping all nodes on master shutdown since no synchronised slave is available 2025-10-20 05:15:23.691 [warning] <0.841.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-4371-3' in vhost '/': Stopping all nodes on master shutdown since no synchronised slave is available 2025-10-20 05:15:23.691 [warning] <0.868.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-4371-2' in vhost '/': Stopping all nodes on master shutdown since no synchronised slave is available 2025-10-20 05:15:23.691 [warning] <0.828.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-4371-3' in vhost '/': Stopping all nodes on master shutdown since no synchronised slave is available 2025-10-20 05:15:23.691 [warning] <0.816.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-4371-1' in vhost '/': Stopping all nodes on master shutdown since no synchronised slave is available 2025-10-20 05:15:23.691 [warning] <0.811.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-4371-2' in vhost '/': Stopping all nodes on master shutdown since no synchronised slave is available 2025-10-20 05:15:23.694 [info] <0.528.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' 2025-10-20 05:15:23.750 [info] <0.528.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' is stopped 2025-10-20 05:15:23.750 [info] <0.525.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' 2025-10-20 05:15:23.755 [info] <0.525.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' is stopped 2025-10-20 05:15:23.762 [info] <0.978.0> Stopping application 'mnesia' 2025-10-20 05:15:24.197 [warning] lager_file_backend dropped 9 messages in the last second that exceeded the limit of 50 messages/sec 2025-10-20 05:15:40.022 [info] <0.1045.0> Resetting Rabbit forcefully 2025-10-20 05:15:41.694 [info] <0.1059.0> RabbitMQ is asked to stop... 2025-10-20 05:15:41.764 [info] <0.1059.0> Stopping RabbitMQ applications and their dependencies in the following order: rabbitmq_management amqp_client rabbitmq_web_dispatch rabbitmq_management_agent rabbit mnesia rabbit_common sysmon_handler os_mon 2025-10-20 05:15:41.764 [info] <0.1059.0> Stopping application 'rabbitmq_management' 2025-10-20 05:15:41.764 [info] <0.1059.0> Stopping application 'amqp_client' 2025-10-20 05:15:41.764 [info] <0.1059.0> Stopping application 'rabbitmq_web_dispatch' 2025-10-20 05:15:41.764 [info] <0.1059.0> Stopping application 'rabbitmq_management_agent' 2025-10-20 05:15:41.764 [info] <0.1059.0> Stopping application 'rabbit' 2025-10-20 05:15:41.764 [info] <0.1059.0> Stopping application 'mnesia' 2025-10-20 05:15:41.764 [info] <0.1059.0> Stopping application 'rabbit_common' 2025-10-20 05:15:41.764 [info] <0.1059.0> Stopping application 'sysmon_handler' 2025-10-20 05:15:41.764 [info] <0.1059.0> Stopping application 'os_mon' 2025-10-20 05:15:41.764 [info] <0.1059.0> Successfully stopped RabbitMQ and its dependencies 2025-10-20 05:15:41.764 [info] <0.1059.0> Halting Erlang VM with the following applications: ranch ssl public_key asn1 credentials_obfuscation observer_cli xmerl tools crypto stdout_formatter jsx recon inets lager goldrush compiler syntax_tools sasl stdlib kernel 2025-10-20 05:15:50.109 [info] <0.8.0> Log file opened with Lager 2025-10-20 05:15:52.891 [info] <0.8.0> Feature flags: list of feature flags found: 2025-10-20 05:15:52.892 [info] <0.8.0> Feature flags: feature flag states written to disk: yes 2025-10-20 05:15:52.912 [info] <0.240.0> Starting RabbitMQ 3.7.28 on Erlang 21.3.8.21 Copyright (c) 2007-2020 Pivotal Software, Inc. Licensed under the MPL. See https://www.rabbitmq.com/ 2025-10-20 05:15:52.913 [info] <0.240.0> node : contrail@cn-jenkins-deploy-platform-ansible-os-4371-3 home dir : /var/lib/rabbitmq config file(s) : /etc/rabbitmq/rabbitmq.config cookie hash : esZsDxSN6VGbi9JkMSxNZA== log(s) : /var/log/contrail/config-database-rabbitmq/rabbitmq.log : /var/log/rabbitmq/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3_upgrade.log database dir : /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3 2025-10-20 05:15:52.927 [info] <0.240.0> Running boot step pre_boot defined by app rabbit 2025-10-20 05:15:52.927 [info] <0.240.0> Running boot step rabbit_core_metrics defined by app rabbit 2025-10-20 05:15:52.927 [info] <0.240.0> Running boot step rabbit_alarm defined by app rabbit 2025-10-20 05:15:52.939 [info] <0.246.0> Memory high watermark set to 12637 MiB (13251084288 bytes) of 15796 MiB (16563855360 bytes) total 2025-10-20 05:15:52.945 [info] <0.248.0> Enabling free disk space monitoring 2025-10-20 05:15:52.945 [info] <0.248.0> Disk free limit set to 50MB 2025-10-20 05:15:52.951 [info] <0.240.0> Running boot step code_server_cache defined by app rabbit 2025-10-20 05:15:52.951 [info] <0.240.0> Running boot step file_handle_cache defined by app rabbit 2025-10-20 05:15:52.951 [info] <0.251.0> Limiting to approx 65436 file handles (58890 sockets) 2025-10-20 05:15:52.951 [info] <0.252.0> FHC read buffering: OFF 2025-10-20 05:15:52.951 [info] <0.252.0> FHC write buffering: ON 2025-10-20 05:15:52.952 [info] <0.240.0> Running boot step worker_pool defined by app rabbit 2025-10-20 05:15:52.952 [info] <0.241.0> Will use 4 processes for default worker pool 2025-10-20 05:15:52.952 [info] <0.241.0> Starting worker pool 'worker_pool' with 4 processes in it 2025-10-20 05:15:52.953 [info] <0.240.0> Running boot step database defined by app rabbit 2025-10-20 05:15:52.954 [info] <0.240.0> Node database directory at /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3 is empty. Assuming we need to join an existing cluster or initialise from scratch... 2025-10-20 05:15:52.954 [info] <0.240.0> Configured peer discovery backend: rabbit_peer_discovery_classic_config 2025-10-20 05:15:52.954 [info] <0.240.0> Will try to lock with peer discovery backend rabbit_peer_discovery_classic_config 2025-10-20 05:15:52.954 [info] <0.240.0> Peer discovery backend does not support locking, falling back to randomized delay 2025-10-20 05:15:52.954 [info] <0.240.0> Peer discovery backend rabbit_peer_discovery_classic_config supports registration. 2025-10-20 05:15:52.954 [info] <0.240.0> Will wait for 22308 milliseconds before proceeding with registration... 2025-10-20 05:16:15.262 [info] <0.240.0> All discovered existing cluster peers: contrail@cn-jenkins-deploy-platform-ansible-os-4371-1, contrail@cn-jenkins-deploy-platform-ansible-os-4371-2, contrail@cn-jenkins-deploy-platform-ansible-os-4371-3 2025-10-20 05:16:15.262 [info] <0.240.0> Peer nodes we can cluster with: contrail@cn-jenkins-deploy-platform-ansible-os-4371-1, contrail@cn-jenkins-deploy-platform-ansible-os-4371-2 2025-10-20 05:16:15.263 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:16:15.269 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:16:15.269 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 9 retries left... 2025-10-20 05:16:15.770 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:16:15.771 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:16:15.771 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 8 retries left... 2025-10-20 05:16:16.273 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:16:16.274 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:16:16.274 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 7 retries left... 2025-10-20 05:16:16.776 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:16:16.777 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:16:16.777 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 6 retries left... 2025-10-20 05:16:17.278 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:16:17.279 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:16:17.279 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 5 retries left... 2025-10-20 05:16:17.780 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:16:17.781 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:16:17.781 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 4 retries left... 2025-10-20 05:16:18.283 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:16:18.283 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:16:18.284 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 3 retries left... 2025-10-20 05:16:18.785 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:16:18.786 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:16:18.786 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 2 retries left... 2025-10-20 05:16:19.287 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:16:19.288 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:16:19.288 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 1 retries left... 2025-10-20 05:16:19.790 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:16:19.790 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:16:19.791 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 0 retries left... 2025-10-20 05:16:20.291 [warning] <0.240.0> Could not successfully contact any node of: contrail@cn-jenkins-deploy-platform-ansible-os-4371-1,contrail@cn-jenkins-deploy-platform-ansible-os-4371-2 (as in Erlang distribution). Starting as a blank standalone node... 2025-10-20 05:16:20.293 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-10-20 05:16:20.420 [info] <0.240.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-10-20 05:16:20.421 [info] <0.240.0> Successfully synced tables from a peer 2025-10-20 05:16:20.451 [info] <0.240.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-10-20 05:16:20.452 [info] <0.240.0> Successfully synced tables from a peer 2025-10-20 05:16:20.492 [info] <0.240.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-10-20 05:16:20.492 [info] <0.240.0> Successfully synced tables from a peer 2025-10-20 05:16:20.492 [info] <0.240.0> Will register with peer discovery backend rabbit_peer_discovery_classic_config 2025-10-20 05:16:20.492 [info] <0.240.0> Running boot step database_sync defined by app rabbit 2025-10-20 05:16:20.493 [info] <0.240.0> Running boot step feature_flags defined by app rabbit 2025-10-20 05:16:20.493 [info] <0.240.0> Running boot step codec_correctness_check defined by app rabbit 2025-10-20 05:16:20.493 [info] <0.240.0> Running boot step external_infrastructure defined by app rabbit 2025-10-20 05:16:20.493 [info] <0.240.0> Running boot step rabbit_registry defined by app rabbit 2025-10-20 05:16:20.493 [info] <0.240.0> Running boot step rabbit_auth_mechanism_cr_demo defined by app rabbit 2025-10-20 05:16:20.493 [info] <0.240.0> Running boot step rabbit_queue_location_random defined by app rabbit 2025-10-20 05:16:20.493 [info] <0.240.0> Running boot step rabbit_event defined by app rabbit 2025-10-20 05:16:20.493 [info] <0.240.0> Running boot step rabbit_auth_mechanism_amqplain defined by app rabbit 2025-10-20 05:16:20.493 [info] <0.240.0> Running boot step rabbit_auth_mechanism_plain defined by app rabbit 2025-10-20 05:16:20.493 [info] <0.240.0> Running boot step rabbit_exchange_type_direct defined by app rabbit 2025-10-20 05:16:20.493 [info] <0.240.0> Running boot step rabbit_exchange_type_fanout defined by app rabbit 2025-10-20 05:16:20.494 [info] <0.240.0> Running boot step rabbit_exchange_type_headers defined by app rabbit 2025-10-20 05:16:20.494 [info] <0.240.0> Running boot step rabbit_exchange_type_topic defined by app rabbit 2025-10-20 05:16:20.494 [info] <0.240.0> Running boot step rabbit_mirror_queue_mode_all defined by app rabbit 2025-10-20 05:16:20.494 [info] <0.240.0> Running boot step rabbit_mirror_queue_mode_exactly defined by app rabbit 2025-10-20 05:16:20.494 [info] <0.240.0> Running boot step rabbit_mirror_queue_mode_nodes defined by app rabbit 2025-10-20 05:16:20.494 [info] <0.240.0> Running boot step rabbit_priority_queue defined by app rabbit 2025-10-20 05:16:20.494 [info] <0.240.0> Priority queues enabled, real BQ is rabbit_variable_queue 2025-10-20 05:16:20.494 [info] <0.240.0> Running boot step rabbit_queue_location_client_local defined by app rabbit 2025-10-20 05:16:20.494 [info] <0.240.0> Running boot step rabbit_queue_location_min_masters defined by app rabbit 2025-10-20 05:16:20.494 [info] <0.240.0> Running boot step kernel_ready defined by app rabbit 2025-10-20 05:16:20.494 [info] <0.240.0> Running boot step rabbit_sysmon_minder defined by app rabbit 2025-10-20 05:16:20.494 [info] <0.240.0> Running boot step rabbit_epmd_monitor defined by app rabbit 2025-10-20 05:16:20.495 [info] <0.462.0> epmd monitor knows us, inter-node communication (distribution) port: 25673 2025-10-20 05:16:20.495 [info] <0.240.0> Running boot step guid_generator defined by app rabbit 2025-10-20 05:16:20.499 [info] <0.240.0> Running boot step rabbit_node_monitor defined by app rabbit 2025-10-20 05:16:20.500 [info] <0.466.0> Starting rabbit_node_monitor 2025-10-20 05:16:20.500 [info] <0.240.0> Running boot step delegate_sup defined by app rabbit 2025-10-20 05:16:20.500 [info] <0.240.0> Running boot step rabbit_memory_monitor defined by app rabbit 2025-10-20 05:16:20.501 [info] <0.240.0> Running boot step core_initialized defined by app rabbit 2025-10-20 05:16:20.501 [info] <0.240.0> Running boot step upgrade_queues defined by app rabbit 2025-10-20 05:16:20.530 [info] <0.240.0> message_store upgrades: 1 to apply 2025-10-20 05:16:20.530 [info] <0.240.0> message_store upgrades: Applying rabbit_variable_queue:move_messages_to_vhost_store 2025-10-20 05:16:20.530 [info] <0.240.0> message_store upgrades: No durable queues found. Skipping message store migration 2025-10-20 05:16:20.530 [info] <0.240.0> message_store upgrades: Removing the old message store data 2025-10-20 05:16:20.531 [info] <0.240.0> message_store upgrades: All upgrades applied successfully 2025-10-20 05:16:20.561 [info] <0.240.0> Running boot step rabbit_connection_tracking defined by app rabbit 2025-10-20 05:16:20.561 [info] <0.240.0> Running boot step rabbit_connection_tracking_handler defined by app rabbit 2025-10-20 05:16:20.561 [info] <0.240.0> Running boot step rabbit_exchange_parameters defined by app rabbit 2025-10-20 05:16:20.561 [info] <0.240.0> Running boot step rabbit_mirror_queue_misc defined by app rabbit 2025-10-20 05:16:20.562 [info] <0.240.0> Running boot step rabbit_policies defined by app rabbit 2025-10-20 05:16:20.563 [info] <0.240.0> Running boot step rabbit_policy defined by app rabbit 2025-10-20 05:16:20.563 [info] <0.240.0> Running boot step rabbit_queue_location_validator defined by app rabbit 2025-10-20 05:16:20.563 [info] <0.240.0> Running boot step rabbit_vhost_limit defined by app rabbit 2025-10-20 05:16:20.563 [info] <0.240.0> Running boot step rabbit_mgmt_reset_handler defined by app rabbitmq_management 2025-10-20 05:16:20.563 [info] <0.240.0> Running boot step rabbit_mgmt_db_handler defined by app rabbitmq_management_agent 2025-10-20 05:16:20.563 [warning] <0.240.0> force_fine_statistics set to true; ignored. Replaced by {rates_mode, none} in the rabbitmq_management application. 2025-10-20 05:16:20.563 [info] <0.240.0> Management plugin: using rates mode 'basic' 2025-10-20 05:16:20.564 [info] <0.240.0> Running boot step recovery defined by app rabbit 2025-10-20 05:16:20.564 [info] <0.240.0> Running boot step load_definitions defined by app rabbitmq_management 2025-10-20 05:16:20.565 [info] <0.240.0> Applying definitions from /etc/rabbitmq/definitions.json 2025-10-20 05:16:20.565 [info] <0.240.0> Asked to import definitions. Acting user: rmq-internal 2025-10-20 05:16:20.565 [info] <0.240.0> Importing users... 2025-10-20 05:16:20.565 [info] <0.240.0> Creating user 'guest' 2025-10-20 05:16:20.572 [info] <0.240.0> Setting user tags for user 'guest' to [administrator] 2025-10-20 05:16:20.574 [info] <0.240.0> Importing vhosts... 2025-10-20 05:16:20.574 [info] <0.240.0> Adding vhost '/' 2025-10-20 05:16:20.609 [info] <0.514.0> Making sure data directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists 2025-10-20 05:16:20.616 [info] <0.514.0> Starting message stores for vhost '/' 2025-10-20 05:16:20.616 [info] <0.518.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index 2025-10-20 05:16:20.619 [info] <0.514.0> Started message store of type transient for vhost '/' 2025-10-20 05:16:20.620 [info] <0.521.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index 2025-10-20 05:16:20.620 [warning] <0.521.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": rebuilding indices from scratch 2025-10-20 05:16:20.621 [info] <0.514.0> Started message store of type persistent for vhost '/' 2025-10-20 05:16:20.622 [info] <0.240.0> Importing user permissions... 2025-10-20 05:16:20.623 [info] <0.240.0> Setting permissions for 'guest' in '/' to '.*', '.*', '.*' 2025-10-20 05:16:20.626 [info] <0.240.0> Importing topic permissions... 2025-10-20 05:16:20.626 [info] <0.240.0> Importing parameters... 2025-10-20 05:16:20.626 [info] <0.240.0> Importing global parameters... 2025-10-20 05:16:20.626 [info] <0.240.0> Importing policies... 2025-10-20 05:16:20.635 [info] <0.240.0> Importing queues... 2025-10-20 05:16:20.635 [info] <0.240.0> Importing exchanges... 2025-10-20 05:16:20.635 [info] <0.240.0> Importing bindings... 2025-10-20 05:16:20.635 [info] <0.240.0> Running boot step empty_db_check defined by app rabbit 2025-10-20 05:16:20.635 [info] <0.240.0> Running boot step rabbit_looking_glass defined by app rabbit 2025-10-20 05:16:20.635 [info] <0.240.0> Running boot step rabbit_core_metrics_gc defined by app rabbit 2025-10-20 05:16:20.636 [info] <0.240.0> Running boot step background_gc defined by app rabbit 2025-10-20 05:16:20.636 [info] <0.240.0> Running boot step connection_tracking defined by app rabbit 2025-10-20 05:16:20.639 [info] <0.240.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-4371-3' 2025-10-20 05:16:20.645 [info] <0.240.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-4371-3' 2025-10-20 05:16:20.645 [info] <0.240.0> Running boot step routing_ready defined by app rabbit 2025-10-20 05:16:20.645 [info] <0.240.0> Running boot step pre_flight defined by app rabbit 2025-10-20 05:16:20.645 [info] <0.240.0> Running boot step notify_cluster defined by app rabbit 2025-10-20 05:16:20.645 [info] <0.240.0> Running boot step networking defined by app rabbit 2025-10-20 05:16:20.647 [info] <0.567.0> started TCP listener on [::]:5673 2025-10-20 05:16:20.647 [info] <0.240.0> Running boot step cluster_name defined by app rabbit 2025-10-20 05:16:20.647 [info] <0.240.0> Running boot step direct_client defined by app rabbit 2025-10-20 05:16:20.647 [info] <0.240.0> Running boot step os_signal_handler defined by app rabbit 2025-10-20 05:16:20.647 [info] <0.569.0> Swapping OS signal event handler (erl_signal_server) for our own 2025-10-20 05:16:20.683 [info] <0.619.0> Management plugin: HTTP (non-TLS) listener started on port 15673 2025-10-20 05:16:20.683 [info] <0.725.0> Statistics database started. 2025-10-20 05:16:20.683 [info] <0.724.0> Starting worker pool 'management_worker_pool' with 3 processes in it 2025-10-20 05:16:20.684 [notice] <0.106.0> Changed loghwm of /var/log/contrail/config-database-rabbitmq/rabbitmq.log to 50 2025-10-20 05:16:20.814 [info] <0.8.0> Server startup complete; 3 plugins started. * rabbitmq_management * rabbitmq_management_agent * rabbitmq_web_dispatch 2025-10-20 05:16:22.558 [info] <0.749.0> RabbitMQ is asked to stop... 2025-10-20 05:16:22.744 [info] <0.749.0> Stopping RabbitMQ applications and their dependencies in the following order: rabbitmq_management amqp_client rabbitmq_web_dispatch cowboy cowlib rabbitmq_management_agent rabbit mnesia rabbit_common sysmon_handler os_mon 2025-10-20 05:16:22.744 [info] <0.749.0> Stopping application 'rabbitmq_management' 2025-10-20 05:16:22.747 [warning] <0.575.0> RabbitMQ HTTP listener registry could not find context rabbitmq_management_tls 2025-10-20 05:16:22.751 [info] <0.43.0> Application rabbitmq_management exited with reason: stopped 2025-10-20 05:16:22.751 [info] <0.749.0> Stopping application 'amqp_client' 2025-10-20 05:16:22.754 [info] <0.43.0> Application amqp_client exited with reason: stopped 2025-10-20 05:16:22.754 [info] <0.749.0> Stopping application 'rabbitmq_web_dispatch' 2025-10-20 05:16:22.756 [info] <0.749.0> Stopping application 'cowboy' 2025-10-20 05:16:22.756 [info] <0.43.0> Application rabbitmq_web_dispatch exited with reason: stopped 2025-10-20 05:16:22.759 [info] <0.43.0> Application cowboy exited with reason: stopped 2025-10-20 05:16:22.759 [info] <0.749.0> Stopping application 'cowlib' 2025-10-20 05:16:22.759 [info] <0.43.0> Application cowlib exited with reason: stopped 2025-10-20 05:16:22.759 [info] <0.749.0> Stopping application 'rabbitmq_management_agent' 2025-10-20 05:16:22.762 [info] <0.43.0> Application rabbitmq_management_agent exited with reason: stopped 2025-10-20 05:16:22.762 [info] <0.749.0> Stopping application 'rabbit' 2025-10-20 05:16:22.762 [info] <0.240.0> Will unregister with peer discovery backend rabbit_peer_discovery_classic_config 2025-10-20 05:16:22.763 [info] <0.567.0> stopped TCP listener on [::]:5673 2025-10-20 05:16:22.764 [info] <0.491.0> Closing all connections in vhost '/' on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4371-3' because the vhost is stopping 2025-10-20 05:16:22.764 [info] <0.521.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' 2025-10-20 05:16:22.768 [info] <0.521.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' is stopped 2025-10-20 05:16:22.768 [info] <0.518.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' 2025-10-20 05:16:22.772 [info] <0.518.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' is stopped 2025-10-20 05:16:22.776 [info] <0.43.0> Application rabbit exited with reason: stopped 2025-10-20 05:16:22.776 [info] <0.749.0> Stopping application 'mnesia' 2025-10-20 05:16:22.779 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-10-20 05:16:22.779 [info] <0.749.0> Stopping application 'rabbit_common' 2025-10-20 05:16:22.779 [info] <0.43.0> Application rabbit_common exited with reason: stopped 2025-10-20 05:16:22.779 [info] <0.749.0> Stopping application 'sysmon_handler' 2025-10-20 05:16:22.781 [info] <0.43.0> Application sysmon_handler exited with reason: stopped 2025-10-20 05:16:22.781 [info] <0.749.0> Stopping application 'os_mon' 2025-10-20 05:16:22.782 [info] <0.43.0> Application os_mon exited with reason: stopped 2025-10-20 05:16:22.782 [info] <0.749.0> Successfully stopped RabbitMQ and its dependencies 2025-10-20 05:18:54.819 [info] <0.790.0> Resetting Rabbit forcefully 2025-10-20 05:18:56.901 [info] <0.804.0> RabbitMQ is asked to stop... 2025-10-20 05:18:57.035 [info] <0.804.0> Stopping RabbitMQ applications and their dependencies in the following order: rabbitmq_management amqp_client rabbitmq_management_agent rabbitmq_web_dispatch rabbit mnesia rabbit_common sysmon_handler os_mon 2025-10-20 05:18:57.035 [info] <0.804.0> Stopping application 'rabbitmq_management' 2025-10-20 05:18:57.036 [info] <0.804.0> Stopping application 'amqp_client' 2025-10-20 05:18:57.036 [info] <0.804.0> Stopping application 'rabbitmq_management_agent' 2025-10-20 05:18:57.036 [info] <0.804.0> Stopping application 'rabbitmq_web_dispatch' 2025-10-20 05:18:57.036 [info] <0.804.0> Stopping application 'rabbit' 2025-10-20 05:18:57.036 [info] <0.804.0> Stopping application 'mnesia' 2025-10-20 05:18:57.036 [info] <0.804.0> Stopping application 'rabbit_common' 2025-10-20 05:18:57.036 [info] <0.804.0> Stopping application 'sysmon_handler' 2025-10-20 05:18:57.036 [info] <0.804.0> Stopping application 'os_mon' 2025-10-20 05:18:57.036 [info] <0.804.0> Successfully stopped RabbitMQ and its dependencies 2025-10-20 05:18:57.036 [info] <0.804.0> Halting Erlang VM with the following applications: ranch ssl public_key asn1 observer_cli tools credentials_obfuscation crypto stdout_formatter jsx recon inets xmerl lager goldrush compiler syntax_tools sasl stdlib kernel 2025-10-20 05:19:05.302 [info] <0.8.0> Log file opened with Lager 2025-10-20 05:19:07.544 [info] <0.8.0> Feature flags: list of feature flags found: 2025-10-20 05:19:07.544 [info] <0.8.0> Feature flags: feature flag states written to disk: yes 2025-10-20 05:19:07.565 [info] <0.240.0> Starting RabbitMQ 3.7.28 on Erlang 21.3.8.21 Copyright (c) 2007-2020 Pivotal Software, Inc. Licensed under the MPL. See https://www.rabbitmq.com/ 2025-10-20 05:19:07.565 [info] <0.240.0> node : contrail@cn-jenkins-deploy-platform-ansible-os-4371-3 home dir : /var/lib/rabbitmq config file(s) : /etc/rabbitmq/rabbitmq.config cookie hash : esZsDxSN6VGbi9JkMSxNZA== log(s) : /var/log/contrail/config-database-rabbitmq/rabbitmq.log : /var/log/rabbitmq/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3_upgrade.log database dir : /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3 2025-10-20 05:19:07.582 [info] <0.240.0> Running boot step pre_boot defined by app rabbit 2025-10-20 05:19:07.582 [info] <0.240.0> Running boot step rabbit_core_metrics defined by app rabbit 2025-10-20 05:19:07.582 [info] <0.240.0> Running boot step rabbit_alarm defined by app rabbit 2025-10-20 05:19:07.587 [info] <0.246.0> Memory high watermark set to 12637 MiB (13251084288 bytes) of 15796 MiB (16563855360 bytes) total 2025-10-20 05:19:07.594 [info] <0.248.0> Enabling free disk space monitoring 2025-10-20 05:19:07.594 [info] <0.248.0> Disk free limit set to 50MB 2025-10-20 05:19:07.599 [info] <0.240.0> Running boot step code_server_cache defined by app rabbit 2025-10-20 05:19:07.599 [info] <0.240.0> Running boot step file_handle_cache defined by app rabbit 2025-10-20 05:19:07.600 [info] <0.251.0> Limiting to approx 65436 file handles (58890 sockets) 2025-10-20 05:19:07.600 [info] <0.252.0> FHC read buffering: OFF 2025-10-20 05:19:07.600 [info] <0.252.0> FHC write buffering: ON 2025-10-20 05:19:07.600 [info] <0.240.0> Running boot step worker_pool defined by app rabbit 2025-10-20 05:19:07.600 [info] <0.241.0> Will use 4 processes for default worker pool 2025-10-20 05:19:07.600 [info] <0.241.0> Starting worker pool 'worker_pool' with 4 processes in it 2025-10-20 05:19:07.600 [info] <0.240.0> Running boot step database defined by app rabbit 2025-10-20 05:19:07.601 [info] <0.240.0> Node database directory at /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3 is empty. Assuming we need to join an existing cluster or initialise from scratch... 2025-10-20 05:19:07.601 [info] <0.240.0> Configured peer discovery backend: rabbit_peer_discovery_classic_config 2025-10-20 05:19:07.601 [info] <0.240.0> Will try to lock with peer discovery backend rabbit_peer_discovery_classic_config 2025-10-20 05:19:07.601 [info] <0.240.0> Peer discovery backend does not support locking, falling back to randomized delay 2025-10-20 05:19:07.601 [info] <0.240.0> Peer discovery backend rabbit_peer_discovery_classic_config supports registration. 2025-10-20 05:19:07.601 [info] <0.240.0> Will wait for 16317 milliseconds before proceeding with registration... 2025-10-20 05:19:23.919 [info] <0.240.0> All discovered existing cluster peers: contrail@cn-jenkins-deploy-platform-ansible-os-4371-1, contrail@cn-jenkins-deploy-platform-ansible-os-4371-2, contrail@cn-jenkins-deploy-platform-ansible-os-4371-3 2025-10-20 05:19:23.919 [info] <0.240.0> Peer nodes we can cluster with: contrail@cn-jenkins-deploy-platform-ansible-os-4371-1, contrail@cn-jenkins-deploy-platform-ansible-os-4371-2 2025-10-20 05:19:23.923 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:19:23.927 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:19:23.927 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 9 retries left... 2025-10-20 05:19:24.428 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:19:24.429 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:19:24.429 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 8 retries left... 2025-10-20 05:19:24.930 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:19:24.931 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:19:24.931 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 7 retries left... 2025-10-20 05:19:25.433 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:19:25.434 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:19:25.434 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 6 retries left... 2025-10-20 05:19:25.936 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:19:25.936 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:19:25.937 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 5 retries left... 2025-10-20 05:19:26.440 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:19:26.443 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:19:26.443 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 4 retries left... 2025-10-20 05:19:26.945 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:19:26.946 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:19:26.946 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 3 retries left... 2025-10-20 05:19:27.450 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:19:27.451 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:19:27.451 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 2 retries left... 2025-10-20 05:19:27.956 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:19:27.956 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:19:27.957 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 1 retries left... 2025-10-20 05:19:28.458 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:19:28.459 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:19:28.459 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 0 retries left... 2025-10-20 05:19:28.959 [warning] <0.240.0> Could not successfully contact any node of: contrail@cn-jenkins-deploy-platform-ansible-os-4371-1,contrail@cn-jenkins-deploy-platform-ansible-os-4371-2 (as in Erlang distribution). Starting as a blank standalone node... 2025-10-20 05:19:28.961 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-10-20 05:19:29.135 [info] <0.240.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-10-20 05:19:29.135 [info] <0.240.0> Successfully synced tables from a peer 2025-10-20 05:19:29.233 [info] <0.240.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-10-20 05:19:29.234 [info] <0.240.0> Successfully synced tables from a peer 2025-10-20 05:19:29.297 [info] <0.240.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-10-20 05:19:29.297 [info] <0.240.0> Successfully synced tables from a peer 2025-10-20 05:19:29.297 [info] <0.240.0> Will register with peer discovery backend rabbit_peer_discovery_classic_config 2025-10-20 05:19:29.297 [info] <0.240.0> Running boot step database_sync defined by app rabbit 2025-10-20 05:19:29.297 [info] <0.240.0> Running boot step feature_flags defined by app rabbit 2025-10-20 05:19:29.297 [info] <0.240.0> Running boot step codec_correctness_check defined by app rabbit 2025-10-20 05:19:29.297 [info] <0.240.0> Running boot step external_infrastructure defined by app rabbit 2025-10-20 05:19:29.298 [info] <0.240.0> Running boot step rabbit_registry defined by app rabbit 2025-10-20 05:19:29.298 [info] <0.240.0> Running boot step rabbit_auth_mechanism_cr_demo defined by app rabbit 2025-10-20 05:19:29.298 [info] <0.240.0> Running boot step rabbit_queue_location_random defined by app rabbit 2025-10-20 05:19:29.298 [info] <0.240.0> Running boot step rabbit_event defined by app rabbit 2025-10-20 05:19:29.298 [info] <0.240.0> Running boot step rabbit_auth_mechanism_amqplain defined by app rabbit 2025-10-20 05:19:29.298 [info] <0.240.0> Running boot step rabbit_auth_mechanism_plain defined by app rabbit 2025-10-20 05:19:29.299 [info] <0.240.0> Running boot step rabbit_exchange_type_direct defined by app rabbit 2025-10-20 05:19:29.299 [info] <0.240.0> Running boot step rabbit_exchange_type_fanout defined by app rabbit 2025-10-20 05:19:29.299 [info] <0.240.0> Running boot step rabbit_exchange_type_headers defined by app rabbit 2025-10-20 05:19:29.299 [info] <0.240.0> Running boot step rabbit_exchange_type_topic defined by app rabbit 2025-10-20 05:19:29.299 [info] <0.240.0> Running boot step rabbit_mirror_queue_mode_all defined by app rabbit 2025-10-20 05:19:29.299 [info] <0.240.0> Running boot step rabbit_mirror_queue_mode_exactly defined by app rabbit 2025-10-20 05:19:29.300 [info] <0.240.0> Running boot step rabbit_mirror_queue_mode_nodes defined by app rabbit 2025-10-20 05:19:29.300 [info] <0.240.0> Running boot step rabbit_priority_queue defined by app rabbit 2025-10-20 05:19:29.300 [info] <0.240.0> Priority queues enabled, real BQ is rabbit_variable_queue 2025-10-20 05:19:29.300 [info] <0.240.0> Running boot step rabbit_queue_location_client_local defined by app rabbit 2025-10-20 05:19:29.300 [info] <0.240.0> Running boot step rabbit_queue_location_min_masters defined by app rabbit 2025-10-20 05:19:29.300 [info] <0.240.0> Running boot step kernel_ready defined by app rabbit 2025-10-20 05:19:29.300 [info] <0.240.0> Running boot step rabbit_sysmon_minder defined by app rabbit 2025-10-20 05:19:29.301 [info] <0.240.0> Running boot step rabbit_epmd_monitor defined by app rabbit 2025-10-20 05:19:29.302 [info] <0.462.0> epmd monitor knows us, inter-node communication (distribution) port: 25673 2025-10-20 05:19:29.303 [info] <0.240.0> Running boot step guid_generator defined by app rabbit 2025-10-20 05:19:29.311 [info] <0.240.0> Running boot step rabbit_node_monitor defined by app rabbit 2025-10-20 05:19:29.311 [info] <0.466.0> Starting rabbit_node_monitor 2025-10-20 05:19:29.311 [info] <0.240.0> Running boot step delegate_sup defined by app rabbit 2025-10-20 05:19:29.311 [info] <0.240.0> Running boot step rabbit_memory_monitor defined by app rabbit 2025-10-20 05:19:29.312 [info] <0.240.0> Running boot step core_initialized defined by app rabbit 2025-10-20 05:19:29.312 [info] <0.240.0> Running boot step upgrade_queues defined by app rabbit 2025-10-20 05:19:29.341 [info] <0.240.0> message_store upgrades: 1 to apply 2025-10-20 05:19:29.341 [info] <0.240.0> message_store upgrades: Applying rabbit_variable_queue:move_messages_to_vhost_store 2025-10-20 05:19:29.342 [info] <0.240.0> message_store upgrades: No durable queues found. Skipping message store migration 2025-10-20 05:19:29.342 [info] <0.240.0> message_store upgrades: Removing the old message store data 2025-10-20 05:19:29.344 [info] <0.240.0> message_store upgrades: All upgrades applied successfully 2025-10-20 05:19:29.381 [info] <0.240.0> Running boot step rabbit_connection_tracking defined by app rabbit 2025-10-20 05:19:29.381 [info] <0.240.0> Running boot step rabbit_connection_tracking_handler defined by app rabbit 2025-10-20 05:19:29.381 [info] <0.240.0> Running boot step rabbit_exchange_parameters defined by app rabbit 2025-10-20 05:19:29.381 [info] <0.240.0> Running boot step rabbit_mirror_queue_misc defined by app rabbit 2025-10-20 05:19:29.382 [info] <0.240.0> Running boot step rabbit_policies defined by app rabbit 2025-10-20 05:19:29.383 [info] <0.240.0> Running boot step rabbit_policy defined by app rabbit 2025-10-20 05:19:29.383 [info] <0.240.0> Running boot step rabbit_queue_location_validator defined by app rabbit 2025-10-20 05:19:29.383 [info] <0.240.0> Running boot step rabbit_vhost_limit defined by app rabbit 2025-10-20 05:19:29.383 [info] <0.240.0> Running boot step rabbit_mgmt_reset_handler defined by app rabbitmq_management 2025-10-20 05:19:29.383 [info] <0.240.0> Running boot step rabbit_mgmt_db_handler defined by app rabbitmq_management_agent 2025-10-20 05:19:29.383 [warning] <0.240.0> force_fine_statistics set to true; ignored. Replaced by {rates_mode, none} in the rabbitmq_management application. 2025-10-20 05:19:29.383 [info] <0.240.0> Management plugin: using rates mode 'basic' 2025-10-20 05:19:29.384 [info] <0.240.0> Running boot step recovery defined by app rabbit 2025-10-20 05:19:29.385 [info] <0.240.0> Running boot step load_definitions defined by app rabbitmq_management 2025-10-20 05:19:29.385 [info] <0.240.0> Applying definitions from /etc/rabbitmq/definitions.json 2025-10-20 05:19:29.385 [info] <0.240.0> Asked to import definitions. Acting user: rmq-internal 2025-10-20 05:19:29.385 [info] <0.240.0> Importing users... 2025-10-20 05:19:29.385 [info] <0.240.0> Creating user 'guest' 2025-10-20 05:19:29.394 [info] <0.240.0> Setting user tags for user 'guest' to [administrator] 2025-10-20 05:19:29.396 [info] <0.240.0> Importing vhosts... 2025-10-20 05:19:29.397 [info] <0.240.0> Adding vhost '/' 2025-10-20 05:19:29.429 [info] <0.514.0> Making sure data directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists 2025-10-20 05:19:29.434 [info] <0.514.0> Starting message stores for vhost '/' 2025-10-20 05:19:29.435 [info] <0.518.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index 2025-10-20 05:19:29.439 [info] <0.514.0> Started message store of type transient for vhost '/' 2025-10-20 05:19:29.439 [info] <0.521.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index 2025-10-20 05:19:29.440 [warning] <0.521.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": rebuilding indices from scratch 2025-10-20 05:19:29.442 [info] <0.514.0> Started message store of type persistent for vhost '/' 2025-10-20 05:19:29.443 [info] <0.240.0> Importing user permissions... 2025-10-20 05:19:29.443 [info] <0.240.0> Setting permissions for 'guest' in '/' to '.*', '.*', '.*' 2025-10-20 05:19:29.450 [info] <0.240.0> Importing topic permissions... 2025-10-20 05:19:29.450 [info] <0.240.0> Importing parameters... 2025-10-20 05:19:29.450 [info] <0.240.0> Importing global parameters... 2025-10-20 05:19:29.450 [info] <0.240.0> Importing policies... 2025-10-20 05:19:29.457 [info] <0.240.0> Importing queues... 2025-10-20 05:19:29.457 [info] <0.240.0> Importing exchanges... 2025-10-20 05:19:29.457 [info] <0.240.0> Importing bindings... 2025-10-20 05:19:29.457 [info] <0.240.0> Running boot step empty_db_check defined by app rabbit 2025-10-20 05:19:29.457 [info] <0.240.0> Running boot step rabbit_looking_glass defined by app rabbit 2025-10-20 05:19:29.457 [info] <0.240.0> Running boot step rabbit_core_metrics_gc defined by app rabbit 2025-10-20 05:19:29.458 [info] <0.240.0> Running boot step background_gc defined by app rabbit 2025-10-20 05:19:29.458 [info] <0.240.0> Running boot step connection_tracking defined by app rabbit 2025-10-20 05:19:29.465 [info] <0.240.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-4371-3' 2025-10-20 05:19:29.469 [info] <0.240.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-4371-3' 2025-10-20 05:19:29.469 [info] <0.240.0> Running boot step routing_ready defined by app rabbit 2025-10-20 05:19:29.469 [info] <0.240.0> Running boot step pre_flight defined by app rabbit 2025-10-20 05:19:29.469 [info] <0.240.0> Running boot step notify_cluster defined by app rabbit 2025-10-20 05:19:29.469 [info] <0.240.0> Running boot step networking defined by app rabbit 2025-10-20 05:19:29.471 [info] <0.567.0> started TCP listener on [::]:5673 2025-10-20 05:19:29.471 [info] <0.240.0> Running boot step cluster_name defined by app rabbit 2025-10-20 05:19:29.471 [info] <0.240.0> Running boot step direct_client defined by app rabbit 2025-10-20 05:19:29.471 [info] <0.240.0> Running boot step os_signal_handler defined by app rabbit 2025-10-20 05:19:29.471 [info] <0.569.0> Swapping OS signal event handler (erl_signal_server) for our own 2025-10-20 05:19:29.523 [info] <0.619.0> Management plugin: HTTP (non-TLS) listener started on port 15673 2025-10-20 05:19:29.523 [info] <0.725.0> Statistics database started. 2025-10-20 05:19:29.523 [info] <0.724.0> Starting worker pool 'management_worker_pool' with 3 processes in it 2025-10-20 05:19:29.523 [notice] <0.106.0> Changed loghwm of /var/log/contrail/config-database-rabbitmq/rabbitmq.log to 50 2025-10-20 05:19:29.953 [info] <0.8.0> Server startup complete; 3 plugins started. * rabbitmq_management * rabbitmq_management_agent * rabbitmq_web_dispatch 2025-10-20 05:19:33.735 [info] <0.748.0> accepting AMQP connection <0.748.0> (10.0.0.49:47194 -> 10.0.0.49:5673) 2025-10-20 05:19:33.738 [info] <0.751.0> accepting AMQP connection <0.751.0> (10.0.0.49:47196 -> 10.0.0.49:5673) 2025-10-20 05:19:33.739 [info] <0.748.0> connection <0.748.0> (10.0.0.49:47194 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:19:33.741 [info] <0.760.0> accepting AMQP connection <0.760.0> (10.0.0.48:47964 -> 10.0.0.49:5673) 2025-10-20 05:19:33.742 [info] <0.763.0> accepting AMQP connection <0.763.0> (10.0.0.48:47966 -> 10.0.0.49:5673) 2025-10-20 05:19:33.742 [info] <0.766.0> accepting AMQP connection <0.766.0> (10.0.0.48:47968 -> 10.0.0.49:5673) 2025-10-20 05:19:33.742 [info] <0.769.0> accepting AMQP connection <0.769.0> (10.0.0.49:47204 -> 10.0.0.49:5673) 2025-10-20 05:19:33.742 [info] <0.774.0> accepting AMQP connection <0.774.0> (10.0.0.254:58008 -> 10.0.0.49:5673) 2025-10-20 05:19:33.742 [info] <0.751.0> connection <0.751.0> (10.0.0.49:47196 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:19:33.744 [info] <0.769.0> connection <0.769.0> (10.0.0.49:47204 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:19:33.744 [info] <0.766.0> connection <0.766.0> (10.0.0.48:47968 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:19:33.745 [info] <0.763.0> connection <0.763.0> (10.0.0.48:47966 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:19:33.745 [info] <0.799.0> accepting AMQP connection <0.799.0> (10.0.0.254:58018 -> 10.0.0.49:5673) 2025-10-20 05:19:33.745 [info] <0.774.0> connection <0.774.0> (10.0.0.254:58008 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:19:33.747 [info] <0.760.0> connection <0.760.0> (10.0.0.48:47964 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:19:33.749 [info] <0.799.0> connection <0.799.0> (10.0.0.254:58018 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:19:33.750 [info] <0.850.0> accepting AMQP connection <0.850.0> (10.0.0.254:58022 -> 10.0.0.49:5673) 2025-10-20 05:19:33.756 [info] <0.850.0> connection <0.850.0> (10.0.0.254:58022 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:19:33.822 [info] <0.912.0> RabbitMQ is asked to stop... 2025-10-20 05:19:33.905 [info] <0.912.0> Stopping RabbitMQ applications and their dependencies in the following order: rabbitmq_management amqp_client rabbitmq_web_dispatch cowboy cowlib rabbitmq_management_agent rabbit mnesia rabbit_common sysmon_handler os_mon 2025-10-20 05:19:33.905 [info] <0.912.0> Stopping application 'rabbitmq_management' 2025-10-20 05:19:33.907 [warning] <0.575.0> RabbitMQ HTTP listener registry could not find context rabbitmq_management_tls 2025-10-20 05:19:33.909 [info] <0.43.0> Application rabbitmq_management exited with reason: stopped 2025-10-20 05:19:33.909 [info] <0.912.0> Stopping application 'amqp_client' 2025-10-20 05:19:33.912 [info] <0.43.0> Application amqp_client exited with reason: stopped 2025-10-20 05:19:33.912 [info] <0.912.0> Stopping application 'rabbitmq_web_dispatch' 2025-10-20 05:19:33.915 [info] <0.912.0> Stopping application 'cowboy' 2025-10-20 05:19:33.914 [info] <0.43.0> Application rabbitmq_web_dispatch exited with reason: stopped 2025-10-20 05:19:33.917 [info] <0.43.0> Application cowboy exited with reason: stopped 2025-10-20 05:19:33.917 [info] <0.912.0> Stopping application 'cowlib' 2025-10-20 05:19:33.917 [info] <0.43.0> Application cowlib exited with reason: stopped 2025-10-20 05:19:33.917 [info] <0.912.0> Stopping application 'rabbitmq_management_agent' 2025-10-20 05:19:33.921 [info] <0.43.0> Application rabbitmq_management_agent exited with reason: stopped 2025-10-20 05:19:33.921 [info] <0.912.0> Stopping application 'rabbit' 2025-10-20 05:19:33.921 [info] <0.240.0> Will unregister with peer discovery backend rabbit_peer_discovery_classic_config 2025-10-20 05:19:33.921 [info] <0.567.0> stopped TCP listener on [::]:5673 2025-10-20 05:19:33.922 [error] <0.769.0> Error on AMQP connection <0.769.0> (10.0.0.49:47204 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:19:33.922 [error] <0.763.0> Error on AMQP connection <0.763.0> (10.0.0.48:47966 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:19:33.922 [error] <0.751.0> Error on AMQP connection <0.751.0> (10.0.0.49:47196 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:19:33.922 [error] <0.774.0> Error on AMQP connection <0.774.0> (10.0.0.254:58008 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:19:33.922 [error] <0.766.0> Error on AMQP connection <0.766.0> (10.0.0.48:47968 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:19:33.922 [error] <0.748.0> Error on AMQP connection <0.748.0> (10.0.0.49:47194 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:19:33.922 [error] <0.850.0> Error on AMQP connection <0.850.0> (10.0.0.254:58022 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:19:33.922 [error] <0.760.0> Error on AMQP connection <0.760.0> (10.0.0.48:47964 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:19:33.922 [error] <0.799.0> Error on AMQP connection <0.799.0> (10.0.0.254:58018 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:19:33.927 [error] <0.816.0> Supervisor {<0.816.0>,rabbit_channel_sup_sup} had child channel_sup started with rabbit_channel_sup:start_link() at undefined exit with reason shutdown in context shutdown_error 2025-10-20 05:19:33.928 [info] <0.491.0> Closing all connections in vhost '/' on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4371-3' because the vhost is stopping 2025-10-20 05:19:33.929 [warning] <0.819.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-4371-3' in vhost '/': Stopping all nodes on master shutdown since no synchronised slave is available 2025-10-20 05:19:33.929 [warning] <0.864.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-4371-1.' in vhost '/': Stopping all nodes on master shutdown since no synchronised slave is available 2025-10-20 05:19:33.929 [warning] <0.806.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-4371-3' in vhost '/': Stopping all nodes on master shutdown since no synchronised slave is available 2025-10-20 05:19:34.227 [warning] lager_file_backend dropped 20 messages in the last second that exceeded the limit of 50 messages/sec 2025-10-20 05:21:07.192 [info] <0.971.0> Resetting Rabbit forcefully 2025-10-20 05:21:08.842 [info] <0.986.0> RabbitMQ is asked to stop... 2025-10-20 05:21:08.913 [info] <0.986.0> Stopping RabbitMQ applications and their dependencies in the following order: rabbitmq_management amqp_client rabbitmq_management_agent rabbitmq_web_dispatch rabbit mnesia rabbit_common sysmon_handler os_mon 2025-10-20 05:21:08.913 [info] <0.986.0> Stopping application 'rabbitmq_management' 2025-10-20 05:21:08.913 [info] <0.986.0> Stopping application 'amqp_client' 2025-10-20 05:21:08.913 [info] <0.986.0> Stopping application 'rabbitmq_management_agent' 2025-10-20 05:21:08.913 [info] <0.986.0> Stopping application 'rabbitmq_web_dispatch' 2025-10-20 05:21:08.913 [info] <0.986.0> Stopping application 'rabbit' 2025-10-20 05:21:08.913 [info] <0.986.0> Stopping application 'mnesia' 2025-10-20 05:21:08.913 [info] <0.986.0> Stopping application 'rabbit_common' 2025-10-20 05:21:08.913 [info] <0.986.0> Stopping application 'sysmon_handler' 2025-10-20 05:21:08.913 [info] <0.986.0> Stopping application 'os_mon' 2025-10-20 05:21:08.913 [info] <0.986.0> Successfully stopped RabbitMQ and its dependencies 2025-10-20 05:21:08.913 [info] <0.986.0> Halting Erlang VM with the following applications: ranch ssl public_key asn1 observer_cli tools credentials_obfuscation crypto stdout_formatter jsx recon inets xmerl lager goldrush compiler syntax_tools sasl stdlib kernel 2025-10-20 05:21:17.186 [info] <0.8.0> Log file opened with Lager 2025-10-20 05:21:20.047 [info] <0.8.0> Feature flags: list of feature flags found: 2025-10-20 05:21:20.048 [info] <0.8.0> Feature flags: feature flag states written to disk: yes 2025-10-20 05:21:20.076 [info] <0.240.0> Starting RabbitMQ 3.7.28 on Erlang 21.3.8.21 Copyright (c) 2007-2020 Pivotal Software, Inc. Licensed under the MPL. See https://www.rabbitmq.com/ 2025-10-20 05:21:20.076 [info] <0.240.0> node : contrail@cn-jenkins-deploy-platform-ansible-os-4371-3 home dir : /var/lib/rabbitmq config file(s) : /etc/rabbitmq/rabbitmq.config cookie hash : esZsDxSN6VGbi9JkMSxNZA== log(s) : /var/log/contrail/config-database-rabbitmq/rabbitmq.log : /var/log/rabbitmq/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3_upgrade.log database dir : /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3 2025-10-20 05:21:20.091 [info] <0.240.0> Running boot step pre_boot defined by app rabbit 2025-10-20 05:21:20.091 [info] <0.240.0> Running boot step rabbit_core_metrics defined by app rabbit 2025-10-20 05:21:20.091 [info] <0.240.0> Running boot step rabbit_alarm defined by app rabbit 2025-10-20 05:21:20.099 [info] <0.246.0> Memory high watermark set to 12637 MiB (13251084288 bytes) of 15796 MiB (16563855360 bytes) total 2025-10-20 05:21:20.106 [info] <0.248.0> Enabling free disk space monitoring 2025-10-20 05:21:20.106 [info] <0.248.0> Disk free limit set to 50MB 2025-10-20 05:21:20.110 [info] <0.240.0> Running boot step code_server_cache defined by app rabbit 2025-10-20 05:21:20.111 [info] <0.240.0> Running boot step file_handle_cache defined by app rabbit 2025-10-20 05:21:20.112 [info] <0.251.0> Limiting to approx 65436 file handles (58890 sockets) 2025-10-20 05:21:20.113 [info] <0.252.0> FHC read buffering: OFF 2025-10-20 05:21:20.113 [info] <0.252.0> FHC write buffering: ON 2025-10-20 05:21:20.113 [info] <0.240.0> Running boot step worker_pool defined by app rabbit 2025-10-20 05:21:20.113 [info] <0.241.0> Will use 4 processes for default worker pool 2025-10-20 05:21:20.113 [info] <0.241.0> Starting worker pool 'worker_pool' with 4 processes in it 2025-10-20 05:21:20.114 [info] <0.240.0> Running boot step database defined by app rabbit 2025-10-20 05:21:20.114 [info] <0.240.0> Node database directory at /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3 is empty. Assuming we need to join an existing cluster or initialise from scratch... 2025-10-20 05:21:20.114 [info] <0.240.0> Configured peer discovery backend: rabbit_peer_discovery_classic_config 2025-10-20 05:21:20.114 [info] <0.240.0> Will try to lock with peer discovery backend rabbit_peer_discovery_classic_config 2025-10-20 05:21:20.114 [info] <0.240.0> Peer discovery backend does not support locking, falling back to randomized delay 2025-10-20 05:21:20.114 [info] <0.240.0> Peer discovery backend rabbit_peer_discovery_classic_config supports registration. 2025-10-20 05:21:20.114 [info] <0.240.0> Will wait for 5514 milliseconds before proceeding with registration... 2025-10-20 05:21:25.629 [info] <0.240.0> All discovered existing cluster peers: contrail@cn-jenkins-deploy-platform-ansible-os-4371-1, contrail@cn-jenkins-deploy-platform-ansible-os-4371-2, contrail@cn-jenkins-deploy-platform-ansible-os-4371-3 2025-10-20 05:21:25.629 [info] <0.240.0> Peer nodes we can cluster with: contrail@cn-jenkins-deploy-platform-ansible-os-4371-1, contrail@cn-jenkins-deploy-platform-ansible-os-4371-2 2025-10-20 05:21:25.631 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:21:25.634 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,tables_not_present} 2025-10-20 05:21:25.634 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 9 retries left... 2025-10-20 05:21:26.137 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:21:26.138 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,tables_not_present} 2025-10-20 05:21:26.138 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 8 retries left... 2025-10-20 05:21:26.639 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:21:26.640 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,tables_not_present} 2025-10-20 05:21:26.640 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 7 retries left... 2025-10-20 05:21:27.142 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:21:27.143 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,tables_not_present} 2025-10-20 05:21:27.143 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 6 retries left... 2025-10-20 05:21:27.645 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:21:27.647 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,tables_not_present} 2025-10-20 05:21:27.647 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 5 retries left... 2025-10-20 05:21:28.150 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:21:28.151 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,tables_not_present} 2025-10-20 05:21:28.152 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 4 retries left... 2025-10-20 05:21:28.653 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:21:28.654 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,tables_not_present} 2025-10-20 05:21:28.654 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 3 retries left... 2025-10-20 05:21:29.156 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:21:29.157 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,tables_not_present} 2025-10-20 05:21:29.158 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 2 retries left... 2025-10-20 05:21:29.659 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:21:29.660 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,tables_not_present} 2025-10-20 05:21:29.661 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 1 retries left... 2025-10-20 05:21:30.162 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:21:30.163 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,tables_not_present} 2025-10-20 05:21:30.164 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 0 retries left... 2025-10-20 05:21:30.665 [warning] <0.240.0> Could not successfully contact any node of: contrail@cn-jenkins-deploy-platform-ansible-os-4371-1,contrail@cn-jenkins-deploy-platform-ansible-os-4371-2 (as in Erlang distribution). Starting as a blank standalone node... 2025-10-20 05:21:30.667 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-10-20 05:21:30.778 [info] <0.240.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-10-20 05:21:30.778 [info] <0.240.0> Successfully synced tables from a peer 2025-10-20 05:21:30.813 [info] <0.240.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-10-20 05:21:30.813 [info] <0.240.0> Successfully synced tables from a peer 2025-10-20 05:21:30.845 [info] <0.240.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-10-20 05:21:30.845 [info] <0.240.0> Successfully synced tables from a peer 2025-10-20 05:21:30.845 [info] <0.240.0> Will register with peer discovery backend rabbit_peer_discovery_classic_config 2025-10-20 05:21:30.845 [info] <0.240.0> Running boot step database_sync defined by app rabbit 2025-10-20 05:21:30.845 [info] <0.240.0> Running boot step feature_flags defined by app rabbit 2025-10-20 05:21:30.845 [info] <0.240.0> Running boot step codec_correctness_check defined by app rabbit 2025-10-20 05:21:30.845 [info] <0.240.0> Running boot step external_infrastructure defined by app rabbit 2025-10-20 05:21:30.845 [info] <0.240.0> Running boot step rabbit_registry defined by app rabbit 2025-10-20 05:21:30.846 [info] <0.240.0> Running boot step rabbit_auth_mechanism_cr_demo defined by app rabbit 2025-10-20 05:21:30.846 [info] <0.240.0> Running boot step rabbit_queue_location_random defined by app rabbit 2025-10-20 05:21:30.846 [info] <0.240.0> Running boot step rabbit_event defined by app rabbit 2025-10-20 05:21:30.846 [info] <0.240.0> Running boot step rabbit_auth_mechanism_amqplain defined by app rabbit 2025-10-20 05:21:30.846 [info] <0.240.0> Running boot step rabbit_auth_mechanism_plain defined by app rabbit 2025-10-20 05:21:30.846 [info] <0.240.0> Running boot step rabbit_exchange_type_direct defined by app rabbit 2025-10-20 05:21:30.846 [info] <0.240.0> Running boot step rabbit_exchange_type_fanout defined by app rabbit 2025-10-20 05:21:30.846 [info] <0.240.0> Running boot step rabbit_exchange_type_headers defined by app rabbit 2025-10-20 05:21:30.846 [info] <0.240.0> Running boot step rabbit_exchange_type_topic defined by app rabbit 2025-10-20 05:21:30.847 [info] <0.240.0> Running boot step rabbit_mirror_queue_mode_all defined by app rabbit 2025-10-20 05:21:30.847 [info] <0.240.0> Running boot step rabbit_mirror_queue_mode_exactly defined by app rabbit 2025-10-20 05:21:30.847 [info] <0.240.0> Running boot step rabbit_mirror_queue_mode_nodes defined by app rabbit 2025-10-20 05:21:30.847 [info] <0.240.0> Running boot step rabbit_priority_queue defined by app rabbit 2025-10-20 05:21:30.847 [info] <0.240.0> Priority queues enabled, real BQ is rabbit_variable_queue 2025-10-20 05:21:30.847 [info] <0.240.0> Running boot step rabbit_queue_location_client_local defined by app rabbit 2025-10-20 05:21:30.847 [info] <0.240.0> Running boot step rabbit_queue_location_min_masters defined by app rabbit 2025-10-20 05:21:30.847 [info] <0.240.0> Running boot step kernel_ready defined by app rabbit 2025-10-20 05:21:30.847 [info] <0.240.0> Running boot step rabbit_sysmon_minder defined by app rabbit 2025-10-20 05:21:30.847 [info] <0.240.0> Running boot step rabbit_epmd_monitor defined by app rabbit 2025-10-20 05:21:30.848 [info] <0.450.0> epmd monitor knows us, inter-node communication (distribution) port: 25673 2025-10-20 05:21:30.848 [info] <0.240.0> Running boot step guid_generator defined by app rabbit 2025-10-20 05:21:30.851 [info] <0.240.0> Running boot step rabbit_node_monitor defined by app rabbit 2025-10-20 05:21:30.851 [info] <0.454.0> Starting rabbit_node_monitor 2025-10-20 05:21:30.851 [info] <0.240.0> Running boot step delegate_sup defined by app rabbit 2025-10-20 05:21:30.852 [info] <0.240.0> Running boot step rabbit_memory_monitor defined by app rabbit 2025-10-20 05:21:30.852 [info] <0.240.0> Running boot step core_initialized defined by app rabbit 2025-10-20 05:21:30.852 [info] <0.240.0> Running boot step upgrade_queues defined by app rabbit 2025-10-20 05:21:30.877 [info] <0.240.0> message_store upgrades: 1 to apply 2025-10-20 05:21:30.877 [info] <0.240.0> message_store upgrades: Applying rabbit_variable_queue:move_messages_to_vhost_store 2025-10-20 05:21:30.877 [info] <0.240.0> message_store upgrades: No durable queues found. Skipping message store migration 2025-10-20 05:21:30.877 [info] <0.240.0> message_store upgrades: Removing the old message store data 2025-10-20 05:21:30.877 [info] <0.240.0> message_store upgrades: All upgrades applied successfully 2025-10-20 05:21:30.903 [info] <0.240.0> Running boot step rabbit_connection_tracking defined by app rabbit 2025-10-20 05:21:30.903 [info] <0.240.0> Running boot step rabbit_connection_tracking_handler defined by app rabbit 2025-10-20 05:21:30.903 [info] <0.240.0> Running boot step rabbit_exchange_parameters defined by app rabbit 2025-10-20 05:21:30.903 [info] <0.240.0> Running boot step rabbit_mirror_queue_misc defined by app rabbit 2025-10-20 05:21:30.903 [info] <0.240.0> Running boot step rabbit_policies defined by app rabbit 2025-10-20 05:21:30.904 [info] <0.240.0> Running boot step rabbit_policy defined by app rabbit 2025-10-20 05:21:30.904 [info] <0.240.0> Running boot step rabbit_queue_location_validator defined by app rabbit 2025-10-20 05:21:30.904 [info] <0.240.0> Running boot step rabbit_vhost_limit defined by app rabbit 2025-10-20 05:21:30.904 [info] <0.240.0> Running boot step rabbit_mgmt_reset_handler defined by app rabbitmq_management 2025-10-20 05:21:30.904 [info] <0.240.0> Running boot step rabbit_mgmt_db_handler defined by app rabbitmq_management_agent 2025-10-20 05:21:30.904 [warning] <0.240.0> force_fine_statistics set to true; ignored. Replaced by {rates_mode, none} in the rabbitmq_management application. 2025-10-20 05:21:30.904 [info] <0.240.0> Management plugin: using rates mode 'basic' 2025-10-20 05:21:30.905 [info] <0.240.0> Running boot step recovery defined by app rabbit 2025-10-20 05:21:30.905 [info] <0.240.0> Running boot step load_definitions defined by app rabbitmq_management 2025-10-20 05:21:30.905 [info] <0.240.0> Applying definitions from /etc/rabbitmq/definitions.json 2025-10-20 05:21:30.905 [info] <0.240.0> Asked to import definitions. Acting user: rmq-internal 2025-10-20 05:21:30.905 [info] <0.240.0> Importing users... 2025-10-20 05:21:30.906 [info] <0.240.0> Creating user 'guest' 2025-10-20 05:21:30.911 [info] <0.240.0> Setting user tags for user 'guest' to [administrator] 2025-10-20 05:21:30.914 [info] <0.240.0> Importing vhosts... 2025-10-20 05:21:30.914 [info] <0.240.0> Adding vhost '/' 2025-10-20 05:21:30.940 [info] <0.502.0> Making sure data directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists 2025-10-20 05:21:30.944 [info] <0.502.0> Starting message stores for vhost '/' 2025-10-20 05:21:30.944 [info] <0.506.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index 2025-10-20 05:21:30.946 [info] <0.502.0> Started message store of type transient for vhost '/' 2025-10-20 05:21:30.946 [info] <0.509.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index 2025-10-20 05:21:30.947 [warning] <0.509.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": rebuilding indices from scratch 2025-10-20 05:21:30.948 [info] <0.502.0> Started message store of type persistent for vhost '/' 2025-10-20 05:21:30.949 [info] <0.240.0> Importing user permissions... 2025-10-20 05:21:30.949 [info] <0.240.0> Setting permissions for 'guest' in '/' to '.*', '.*', '.*' 2025-10-20 05:21:30.952 [info] <0.240.0> Importing topic permissions... 2025-10-20 05:21:30.952 [info] <0.240.0> Importing parameters... 2025-10-20 05:21:30.952 [info] <0.240.0> Importing global parameters... 2025-10-20 05:21:30.952 [info] <0.240.0> Importing policies... 2025-10-20 05:21:30.959 [info] <0.240.0> Importing queues... 2025-10-20 05:21:30.959 [info] <0.240.0> Importing exchanges... 2025-10-20 05:21:30.959 [info] <0.240.0> Importing bindings... 2025-10-20 05:21:30.959 [info] <0.240.0> Running boot step empty_db_check defined by app rabbit 2025-10-20 05:21:30.959 [info] <0.240.0> Running boot step rabbit_looking_glass defined by app rabbit 2025-10-20 05:21:30.959 [info] <0.240.0> Running boot step rabbit_core_metrics_gc defined by app rabbit 2025-10-20 05:21:30.959 [info] <0.240.0> Running boot step background_gc defined by app rabbit 2025-10-20 05:21:30.959 [info] <0.240.0> Running boot step connection_tracking defined by app rabbit 2025-10-20 05:21:30.963 [info] <0.240.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-4371-3' 2025-10-20 05:21:30.967 [info] <0.240.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-4371-3' 2025-10-20 05:21:30.967 [info] <0.240.0> Running boot step routing_ready defined by app rabbit 2025-10-20 05:21:30.967 [info] <0.240.0> Running boot step pre_flight defined by app rabbit 2025-10-20 05:21:30.967 [info] <0.240.0> Running boot step notify_cluster defined by app rabbit 2025-10-20 05:21:30.967 [info] <0.240.0> Running boot step networking defined by app rabbit 2025-10-20 05:21:30.970 [info] <0.555.0> started TCP listener on [::]:5673 2025-10-20 05:21:30.970 [info] <0.240.0> Running boot step cluster_name defined by app rabbit 2025-10-20 05:21:30.970 [info] <0.240.0> Running boot step direct_client defined by app rabbit 2025-10-20 05:21:30.970 [info] <0.240.0> Running boot step os_signal_handler defined by app rabbit 2025-10-20 05:21:30.970 [info] <0.557.0> Swapping OS signal event handler (erl_signal_server) for our own 2025-10-20 05:21:31.000 [info] <0.607.0> Management plugin: HTTP (non-TLS) listener started on port 15673 2025-10-20 05:21:31.000 [info] <0.713.0> Statistics database started. 2025-10-20 05:21:31.000 [info] <0.712.0> Starting worker pool 'management_worker_pool' with 3 processes in it 2025-10-20 05:21:31.001 [notice] <0.106.0> Changed loghwm of /var/log/contrail/config-database-rabbitmq/rabbitmq.log to 50 2025-10-20 05:21:31.168 [info] <0.8.0> Server startup complete; 3 plugins started. * rabbitmq_management * rabbitmq_management_agent * rabbitmq_web_dispatch 2025-10-20 05:21:32.105 [info] <0.724.0> accepting AMQP connection <0.724.0> (10.0.0.48:52994 -> 10.0.0.49:5673) 2025-10-20 05:21:32.110 [info] <0.728.0> accepting AMQP connection <0.728.0> (10.0.0.254:50748 -> 10.0.0.49:5673) 2025-10-20 05:21:32.112 [info] <0.731.0> accepting AMQP connection <0.731.0> (10.0.0.254:50764 -> 10.0.0.49:5673) 2025-10-20 05:21:32.115 [info] <0.724.0> connection <0.724.0> (10.0.0.48:52994 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:21:32.116 [info] <0.728.0> connection <0.728.0> (10.0.0.254:50748 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:21:32.120 [info] <0.731.0> connection <0.731.0> (10.0.0.254:50764 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:21:33.664 [info] <0.797.0> RabbitMQ is asked to stop... 2025-10-20 05:21:33.751 [info] <0.797.0> Stopping RabbitMQ applications and their dependencies in the following order: rabbitmq_management amqp_client rabbitmq_web_dispatch cowboy cowlib rabbitmq_management_agent rabbit mnesia rabbit_common sysmon_handler os_mon 2025-10-20 05:21:33.751 [info] <0.797.0> Stopping application 'rabbitmq_management' 2025-10-20 05:21:33.752 [warning] <0.563.0> RabbitMQ HTTP listener registry could not find context rabbitmq_management_tls 2025-10-20 05:21:33.754 [info] <0.43.0> Application rabbitmq_management exited with reason: stopped 2025-10-20 05:21:33.754 [info] <0.797.0> Stopping application 'amqp_client' 2025-10-20 05:21:33.756 [info] <0.43.0> Application amqp_client exited with reason: stopped 2025-10-20 05:21:33.756 [info] <0.797.0> Stopping application 'rabbitmq_web_dispatch' 2025-10-20 05:21:33.758 [info] <0.797.0> Stopping application 'cowboy' 2025-10-20 05:21:33.758 [info] <0.43.0> Application rabbitmq_web_dispatch exited with reason: stopped 2025-10-20 05:21:33.760 [info] <0.43.0> Application cowboy exited with reason: stopped 2025-10-20 05:21:33.760 [info] <0.797.0> Stopping application 'cowlib' 2025-10-20 05:21:33.760 [info] <0.43.0> Application cowlib exited with reason: stopped 2025-10-20 05:21:33.760 [info] <0.797.0> Stopping application 'rabbitmq_management_agent' 2025-10-20 05:21:33.762 [info] <0.43.0> Application rabbitmq_management_agent exited with reason: stopped 2025-10-20 05:21:33.762 [info] <0.797.0> Stopping application 'rabbit' 2025-10-20 05:21:33.762 [info] <0.240.0> Will unregister with peer discovery backend rabbit_peer_discovery_classic_config 2025-10-20 05:21:33.763 [info] <0.555.0> stopped TCP listener on [::]:5673 2025-10-20 05:21:33.763 [error] <0.731.0> Error on AMQP connection <0.731.0> (10.0.0.254:50764 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:21:33.763 [error] <0.724.0> Error on AMQP connection <0.724.0> (10.0.0.48:52994 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:21:33.764 [error] <0.728.0> Error on AMQP connection <0.728.0> (10.0.0.254:50748 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:21:33.766 [info] <0.479.0> Closing all connections in vhost '/' on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4371-3' because the vhost is stopping 2025-10-20 05:21:33.766 [warning] <0.754.0> Mirrored queue 'device_manager.cn-jenkins-deploy-platform-ansible-os-4371-2.' in vhost '/': Stopping all nodes on master shutdown since no synchronised slave is available 2025-10-20 05:21:33.766 [warning] <0.765.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-4371-1.-8082-0' in vhost '/': Stopping all nodes on master shutdown since no synchronised slave is available 2025-10-20 05:21:33.768 [info] <0.509.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' 2025-10-20 05:21:33.773 [info] <0.509.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' is stopped 2025-10-20 05:21:33.773 [info] <0.506.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' 2025-10-20 05:21:33.779 [info] <0.506.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' is stopped 2025-10-20 05:21:33.786 [info] <0.43.0> Application rabbit exited with reason: stopped 2025-10-20 05:21:33.786 [info] <0.797.0> Stopping application 'mnesia' 2025-10-20 05:21:33.790 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-10-20 05:21:33.790 [info] <0.797.0> Stopping application 'rabbit_common' 2025-10-20 05:21:33.791 [info] <0.43.0> Application rabbit_common exited with reason: stopped 2025-10-20 05:21:33.791 [info] <0.797.0> Stopping application 'sysmon_handler' 2025-10-20 05:21:33.792 [info] <0.43.0> Application sysmon_handler exited with reason: stopped 2025-10-20 05:21:33.792 [info] <0.797.0> Stopping application 'os_mon' 2025-10-20 05:21:33.793 [info] <0.43.0> Application os_mon exited with reason: stopped 2025-10-20 05:21:33.793 [info] <0.797.0> Successfully stopped RabbitMQ and its dependencies 2025-10-20 05:22:14.520 [info] <0.838.0> Resetting Rabbit forcefully 2025-10-20 05:22:16.215 [info] <0.852.0> RabbitMQ is asked to stop... 2025-10-20 05:22:16.311 [info] <0.852.0> Stopping RabbitMQ applications and their dependencies in the following order: rabbitmq_management amqp_client rabbitmq_management_agent rabbitmq_web_dispatch rabbit mnesia rabbit_common sysmon_handler os_mon 2025-10-20 05:22:16.311 [info] <0.852.0> Stopping application 'rabbitmq_management' 2025-10-20 05:22:16.311 [info] <0.852.0> Stopping application 'amqp_client' 2025-10-20 05:22:16.311 [info] <0.852.0> Stopping application 'rabbitmq_management_agent' 2025-10-20 05:22:16.311 [info] <0.852.0> Stopping application 'rabbitmq_web_dispatch' 2025-10-20 05:22:16.311 [info] <0.852.0> Stopping application 'rabbit' 2025-10-20 05:22:16.311 [info] <0.852.0> Stopping application 'mnesia' 2025-10-20 05:22:16.311 [info] <0.852.0> Stopping application 'rabbit_common' 2025-10-20 05:22:16.311 [info] <0.852.0> Stopping application 'sysmon_handler' 2025-10-20 05:22:16.311 [info] <0.852.0> Stopping application 'os_mon' 2025-10-20 05:22:16.312 [info] <0.852.0> Successfully stopped RabbitMQ and its dependencies 2025-10-20 05:22:16.312 [info] <0.852.0> Halting Erlang VM with the following applications: ranch ssl public_key asn1 observer_cli tools credentials_obfuscation crypto stdout_formatter jsx recon inets xmerl lager goldrush compiler syntax_tools sasl stdlib kernel 2025-10-20 05:22:24.009 [info] <0.8.0> Log file opened with Lager 2025-10-20 05:22:27.183 [info] <0.8.0> Feature flags: list of feature flags found: 2025-10-20 05:22:27.183 [info] <0.8.0> Feature flags: feature flag states written to disk: yes 2025-10-20 05:22:27.210 [info] <0.240.0> Starting RabbitMQ 3.7.28 on Erlang 21.3.8.21 Copyright (c) 2007-2020 Pivotal Software, Inc. Licensed under the MPL. See https://www.rabbitmq.com/ 2025-10-20 05:22:27.210 [info] <0.240.0> node : contrail@cn-jenkins-deploy-platform-ansible-os-4371-3 home dir : /var/lib/rabbitmq config file(s) : /etc/rabbitmq/rabbitmq.config cookie hash : esZsDxSN6VGbi9JkMSxNZA== log(s) : /var/log/contrail/config-database-rabbitmq/rabbitmq.log : /var/log/rabbitmq/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3_upgrade.log database dir : /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3 2025-10-20 05:22:27.224 [info] <0.240.0> Running boot step pre_boot defined by app rabbit 2025-10-20 05:22:27.224 [info] <0.240.0> Running boot step rabbit_core_metrics defined by app rabbit 2025-10-20 05:22:27.225 [info] <0.240.0> Running boot step rabbit_alarm defined by app rabbit 2025-10-20 05:22:27.230 [info] <0.246.0> Memory high watermark set to 12637 MiB (13251084288 bytes) of 15796 MiB (16563855360 bytes) total 2025-10-20 05:22:27.238 [info] <0.248.0> Enabling free disk space monitoring 2025-10-20 05:22:27.238 [info] <0.248.0> Disk free limit set to 50MB 2025-10-20 05:22:27.242 [info] <0.240.0> Running boot step code_server_cache defined by app rabbit 2025-10-20 05:22:27.244 [info] <0.240.0> Running boot step file_handle_cache defined by app rabbit 2025-10-20 05:22:27.246 [info] <0.251.0> Limiting to approx 65436 file handles (58890 sockets) 2025-10-20 05:22:27.247 [info] <0.252.0> FHC read buffering: OFF 2025-10-20 05:22:27.247 [info] <0.252.0> FHC write buffering: ON 2025-10-20 05:22:27.247 [info] <0.240.0> Running boot step worker_pool defined by app rabbit 2025-10-20 05:22:27.247 [info] <0.241.0> Will use 4 processes for default worker pool 2025-10-20 05:22:27.247 [info] <0.241.0> Starting worker pool 'worker_pool' with 4 processes in it 2025-10-20 05:22:27.248 [info] <0.240.0> Running boot step database defined by app rabbit 2025-10-20 05:22:27.248 [info] <0.240.0> Node database directory at /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3 is empty. Assuming we need to join an existing cluster or initialise from scratch... 2025-10-20 05:22:27.248 [info] <0.240.0> Configured peer discovery backend: rabbit_peer_discovery_classic_config 2025-10-20 05:22:27.248 [info] <0.240.0> Will try to lock with peer discovery backend rabbit_peer_discovery_classic_config 2025-10-20 05:22:27.248 [info] <0.240.0> Peer discovery backend does not support locking, falling back to randomized delay 2025-10-20 05:22:27.248 [info] <0.240.0> Peer discovery backend rabbit_peer_discovery_classic_config supports registration. 2025-10-20 05:22:27.248 [info] <0.240.0> Will wait for 39336 milliseconds before proceeding with registration... 2025-10-20 05:23:06.585 [info] <0.240.0> All discovered existing cluster peers: contrail@cn-jenkins-deploy-platform-ansible-os-4371-1, contrail@cn-jenkins-deploy-platform-ansible-os-4371-2, contrail@cn-jenkins-deploy-platform-ansible-os-4371-3 2025-10-20 05:23:06.585 [info] <0.240.0> Peer nodes we can cluster with: contrail@cn-jenkins-deploy-platform-ansible-os-4371-1, contrail@cn-jenkins-deploy-platform-ansible-os-4371-2 2025-10-20 05:23:06.586 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:23:06.592 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:23:06.592 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 9 retries left... 2025-10-20 05:23:07.094 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:23:07.095 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:23:07.095 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 8 retries left... 2025-10-20 05:23:07.597 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:23:07.598 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:23:07.599 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 7 retries left... 2025-10-20 05:23:08.100 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {badrpc,nodedown} 2025-10-20 05:23:08.100 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:23:08.100 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 6 retries left... 2025-10-20 05:23:08.628 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {error,mnesia_not_running} 2025-10-20 05:23:08.628 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:23:08.629 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 5 retries left... 2025-10-20 05:23:09.130 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {error,mnesia_not_running} 2025-10-20 05:23:09.134 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:23:09.134 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 4 retries left... 2025-10-20 05:23:09.636 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {error,mnesia_not_running} 2025-10-20 05:23:09.637 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:23:09.637 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 3 retries left... 2025-10-20 05:23:10.145 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {error,mnesia_not_running} 2025-10-20 05:23:10.146 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:23:10.146 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 2 retries left... 2025-10-20 05:23:10.647 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {error,mnesia_not_running} 2025-10-20 05:23:10.648 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:23:10.648 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 1 retries left... 2025-10-20 05:23:11.149 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-1: {error,mnesia_not_running} 2025-10-20 05:23:11.150 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-4371-2: {error,mnesia_not_running} 2025-10-20 05:23:11.150 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 0 retries left... 2025-10-20 05:23:11.651 [warning] <0.240.0> Could not successfully contact any node of: contrail@cn-jenkins-deploy-platform-ansible-os-4371-1,contrail@cn-jenkins-deploy-platform-ansible-os-4371-2 (as in Erlang distribution). Starting as a blank standalone node... 2025-10-20 05:23:11.653 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-10-20 05:23:11.760 [info] <0.240.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-10-20 05:23:11.760 [info] <0.240.0> Successfully synced tables from a peer 2025-10-20 05:23:11.793 [info] <0.240.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-10-20 05:23:11.793 [info] <0.240.0> Successfully synced tables from a peer 2025-10-20 05:23:11.821 [info] <0.240.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-10-20 05:23:11.821 [info] <0.240.0> Successfully synced tables from a peer 2025-10-20 05:23:11.822 [info] <0.240.0> Will register with peer discovery backend rabbit_peer_discovery_classic_config 2025-10-20 05:23:11.822 [info] <0.240.0> Running boot step database_sync defined by app rabbit 2025-10-20 05:23:11.822 [info] <0.240.0> Running boot step feature_flags defined by app rabbit 2025-10-20 05:23:11.822 [info] <0.240.0> Running boot step codec_correctness_check defined by app rabbit 2025-10-20 05:23:11.822 [info] <0.240.0> Running boot step external_infrastructure defined by app rabbit 2025-10-20 05:23:11.822 [info] <0.240.0> Running boot step rabbit_registry defined by app rabbit 2025-10-20 05:23:11.822 [info] <0.240.0> Running boot step rabbit_auth_mechanism_cr_demo defined by app rabbit 2025-10-20 05:23:11.822 [info] <0.240.0> Running boot step rabbit_queue_location_random defined by app rabbit 2025-10-20 05:23:11.822 [info] <0.240.0> Running boot step rabbit_event defined by app rabbit 2025-10-20 05:23:11.822 [info] <0.240.0> Running boot step rabbit_auth_mechanism_amqplain defined by app rabbit 2025-10-20 05:23:11.822 [info] <0.240.0> Running boot step rabbit_auth_mechanism_plain defined by app rabbit 2025-10-20 05:23:11.823 [info] <0.240.0> Running boot step rabbit_exchange_type_direct defined by app rabbit 2025-10-20 05:23:11.823 [info] <0.240.0> Running boot step rabbit_exchange_type_fanout defined by app rabbit 2025-10-20 05:23:11.823 [info] <0.240.0> Running boot step rabbit_exchange_type_headers defined by app rabbit 2025-10-20 05:23:11.823 [info] <0.240.0> Running boot step rabbit_exchange_type_topic defined by app rabbit 2025-10-20 05:23:11.823 [info] <0.240.0> Running boot step rabbit_mirror_queue_mode_all defined by app rabbit 2025-10-20 05:23:11.823 [info] <0.240.0> Running boot step rabbit_mirror_queue_mode_exactly defined by app rabbit 2025-10-20 05:23:11.823 [info] <0.240.0> Running boot step rabbit_mirror_queue_mode_nodes defined by app rabbit 2025-10-20 05:23:11.823 [info] <0.240.0> Running boot step rabbit_priority_queue defined by app rabbit 2025-10-20 05:23:11.823 [info] <0.240.0> Priority queues enabled, real BQ is rabbit_variable_queue 2025-10-20 05:23:11.823 [info] <0.240.0> Running boot step rabbit_queue_location_client_local defined by app rabbit 2025-10-20 05:23:11.824 [info] <0.240.0> Running boot step rabbit_queue_location_min_masters defined by app rabbit 2025-10-20 05:23:11.824 [info] <0.240.0> Running boot step kernel_ready defined by app rabbit 2025-10-20 05:23:11.824 [info] <0.240.0> Running boot step rabbit_sysmon_minder defined by app rabbit 2025-10-20 05:23:11.824 [info] <0.240.0> Running boot step rabbit_epmd_monitor defined by app rabbit 2025-10-20 05:23:11.825 [info] <0.469.0> epmd monitor knows us, inter-node communication (distribution) port: 25673 2025-10-20 05:23:11.825 [info] <0.240.0> Running boot step guid_generator defined by app rabbit 2025-10-20 05:23:11.828 [info] <0.240.0> Running boot step rabbit_node_monitor defined by app rabbit 2025-10-20 05:23:11.828 [info] <0.473.0> Starting rabbit_node_monitor 2025-10-20 05:23:11.828 [info] <0.240.0> Running boot step delegate_sup defined by app rabbit 2025-10-20 05:23:11.829 [info] <0.240.0> Running boot step rabbit_memory_monitor defined by app rabbit 2025-10-20 05:23:11.829 [info] <0.240.0> Running boot step core_initialized defined by app rabbit 2025-10-20 05:23:11.829 [info] <0.240.0> Running boot step upgrade_queues defined by app rabbit 2025-10-20 05:23:11.857 [info] <0.240.0> message_store upgrades: 1 to apply 2025-10-20 05:23:11.857 [info] <0.240.0> message_store upgrades: Applying rabbit_variable_queue:move_messages_to_vhost_store 2025-10-20 05:23:11.857 [info] <0.240.0> message_store upgrades: No durable queues found. Skipping message store migration 2025-10-20 05:23:11.857 [info] <0.240.0> message_store upgrades: Removing the old message store data 2025-10-20 05:23:11.857 [info] <0.240.0> message_store upgrades: All upgrades applied successfully 2025-10-20 05:23:11.885 [info] <0.240.0> Running boot step rabbit_connection_tracking defined by app rabbit 2025-10-20 05:23:11.885 [info] <0.240.0> Running boot step rabbit_connection_tracking_handler defined by app rabbit 2025-10-20 05:23:11.885 [info] <0.240.0> Running boot step rabbit_exchange_parameters defined by app rabbit 2025-10-20 05:23:11.885 [info] <0.240.0> Running boot step rabbit_mirror_queue_misc defined by app rabbit 2025-10-20 05:23:11.885 [info] <0.240.0> Running boot step rabbit_policies defined by app rabbit 2025-10-20 05:23:11.886 [info] <0.240.0> Running boot step rabbit_policy defined by app rabbit 2025-10-20 05:23:11.886 [info] <0.240.0> Running boot step rabbit_queue_location_validator defined by app rabbit 2025-10-20 05:23:11.886 [info] <0.240.0> Running boot step rabbit_vhost_limit defined by app rabbit 2025-10-20 05:23:11.886 [info] <0.240.0> Running boot step rabbit_mgmt_reset_handler defined by app rabbitmq_management 2025-10-20 05:23:11.887 [info] <0.240.0> Running boot step rabbit_mgmt_db_handler defined by app rabbitmq_management_agent 2025-10-20 05:23:11.887 [warning] <0.240.0> force_fine_statistics set to true; ignored. Replaced by {rates_mode, none} in the rabbitmq_management application. 2025-10-20 05:23:11.887 [info] <0.240.0> Management plugin: using rates mode 'basic' 2025-10-20 05:23:11.887 [info] <0.240.0> Running boot step recovery defined by app rabbit 2025-10-20 05:23:11.887 [info] <0.240.0> Running boot step load_definitions defined by app rabbitmq_management 2025-10-20 05:23:11.888 [info] <0.240.0> Applying definitions from /etc/rabbitmq/definitions.json 2025-10-20 05:23:11.888 [info] <0.240.0> Asked to import definitions. Acting user: rmq-internal 2025-10-20 05:23:11.888 [info] <0.240.0> Importing users... 2025-10-20 05:23:11.888 [info] <0.240.0> Creating user 'guest' 2025-10-20 05:23:11.893 [info] <0.240.0> Setting user tags for user 'guest' to [administrator] 2025-10-20 05:23:11.897 [info] <0.240.0> Importing vhosts... 2025-10-20 05:23:11.897 [info] <0.240.0> Adding vhost '/' 2025-10-20 05:23:11.920 [info] <0.521.0> Making sure data directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists 2025-10-20 05:23:11.924 [info] <0.521.0> Starting message stores for vhost '/' 2025-10-20 05:23:11.924 [info] <0.525.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index 2025-10-20 05:23:11.925 [info] <0.521.0> Started message store of type transient for vhost '/' 2025-10-20 05:23:11.925 [info] <0.528.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index 2025-10-20 05:23:11.925 [warning] <0.528.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": rebuilding indices from scratch 2025-10-20 05:23:11.926 [info] <0.521.0> Started message store of type persistent for vhost '/' 2025-10-20 05:23:11.927 [info] <0.240.0> Importing user permissions... 2025-10-20 05:23:11.927 [info] <0.240.0> Setting permissions for 'guest' in '/' to '.*', '.*', '.*' 2025-10-20 05:23:11.931 [info] <0.240.0> Importing topic permissions... 2025-10-20 05:23:11.931 [info] <0.240.0> Importing parameters... 2025-10-20 05:23:11.931 [info] <0.240.0> Importing global parameters... 2025-10-20 05:23:11.931 [info] <0.240.0> Importing policies... 2025-10-20 05:23:11.937 [info] <0.240.0> Importing queues... 2025-10-20 05:23:11.937 [info] <0.240.0> Importing exchanges... 2025-10-20 05:23:11.937 [info] <0.240.0> Importing bindings... 2025-10-20 05:23:11.937 [info] <0.240.0> Running boot step empty_db_check defined by app rabbit 2025-10-20 05:23:11.937 [info] <0.240.0> Running boot step rabbit_looking_glass defined by app rabbit 2025-10-20 05:23:11.937 [info] <0.240.0> Running boot step rabbit_core_metrics_gc defined by app rabbit 2025-10-20 05:23:11.937 [info] <0.240.0> Running boot step background_gc defined by app rabbit 2025-10-20 05:23:11.937 [info] <0.240.0> Running boot step connection_tracking defined by app rabbit 2025-10-20 05:23:11.942 [info] <0.240.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-4371-3' 2025-10-20 05:23:11.945 [info] <0.240.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-4371-3' 2025-10-20 05:23:11.946 [info] <0.240.0> Running boot step routing_ready defined by app rabbit 2025-10-20 05:23:11.946 [info] <0.240.0> Running boot step pre_flight defined by app rabbit 2025-10-20 05:23:11.946 [info] <0.240.0> Running boot step notify_cluster defined by app rabbit 2025-10-20 05:23:11.946 [info] <0.240.0> Running boot step networking defined by app rabbit 2025-10-20 05:23:11.948 [info] <0.574.0> started TCP listener on [::]:5673 2025-10-20 05:23:11.948 [info] <0.240.0> Running boot step cluster_name defined by app rabbit 2025-10-20 05:23:11.948 [info] <0.240.0> Running boot step direct_client defined by app rabbit 2025-10-20 05:23:11.948 [info] <0.240.0> Running boot step os_signal_handler defined by app rabbit 2025-10-20 05:23:11.948 [info] <0.576.0> Swapping OS signal event handler (erl_signal_server) for our own 2025-10-20 05:23:11.983 [info] <0.626.0> Management plugin: HTTP (non-TLS) listener started on port 15673 2025-10-20 05:23:11.983 [info] <0.732.0> Statistics database started. 2025-10-20 05:23:11.983 [info] <0.731.0> Starting worker pool 'management_worker_pool' with 3 processes in it 2025-10-20 05:23:11.983 [notice] <0.106.0> Changed loghwm of /var/log/contrail/config-database-rabbitmq/rabbitmq.log to 50 2025-10-20 05:23:12.088 [info] <0.8.0> Server startup complete; 3 plugins started. * rabbitmq_management * rabbitmq_management_agent * rabbitmq_web_dispatch 2025-10-20 05:23:13.894 [info] <0.744.0> RabbitMQ is asked to stop... 2025-10-20 05:23:13.974 [info] <0.747.0> accepting AMQP connection <0.747.0> (10.0.0.254:38650 -> 10.0.0.49:5673) 2025-10-20 05:23:13.974 [info] <0.750.0> accepting AMQP connection <0.750.0> (10.0.0.49:47758 -> 10.0.0.49:5673) 2025-10-20 05:23:13.975 [info] <0.753.0> accepting AMQP connection <0.753.0> (10.0.0.49:47760 -> 10.0.0.49:5673) 2025-10-20 05:23:13.976 [info] <0.759.0> accepting AMQP connection <0.759.0> (10.0.0.254:38654 -> 10.0.0.49:5673) 2025-10-20 05:23:13.976 [info] <0.762.0> accepting AMQP connection <0.762.0> (10.0.0.49:47776 -> 10.0.0.49:5673) 2025-10-20 05:23:13.978 [info] <0.765.0> accepting AMQP connection <0.765.0> (10.0.0.254:38662 -> 10.0.0.49:5673) 2025-10-20 05:23:13.980 [info] <0.759.0> connection <0.759.0> (10.0.0.254:38654 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:23:13.988 [info] <0.753.0> connection <0.753.0> (10.0.0.49:47760 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:23:13.988 [info] <0.762.0> connection <0.762.0> (10.0.0.49:47776 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:23:13.989 [info] <0.776.0> accepting AMQP connection <0.776.0> (10.0.0.48:60658 -> 10.0.0.49:5673) 2025-10-20 05:23:13.991 [info] <0.792.0> accepting AMQP connection <0.792.0> (10.0.0.48:60668 -> 10.0.0.49:5673) 2025-10-20 05:23:13.991 [info] <0.765.0> connection <0.765.0> (10.0.0.254:38662 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:23:13.991 [info] <0.747.0> connection <0.747.0> (10.0.0.254:38650 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:23:13.994 [info] <0.750.0> connection <0.750.0> (10.0.0.49:47758 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:23:13.995 [info] <0.817.0> accepting AMQP connection <0.817.0> (10.0.0.48:60670 -> 10.0.0.49:5673) 2025-10-20 05:23:13.998 [info] <0.776.0> connection <0.776.0> (10.0.0.48:60658 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:23:13.998 [info] <0.792.0> connection <0.792.0> (10.0.0.48:60668 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:23:13.999 [info] <0.817.0> connection <0.817.0> (10.0.0.48:60670 -> 10.0.0.49:5673): user 'guest' authenticated and granted access to vhost '/' 2025-10-20 05:23:14.124 [info] <0.744.0> Stopping RabbitMQ applications and their dependencies in the following order: rabbitmq_management amqp_client rabbitmq_web_dispatch cowboy cowlib rabbitmq_management_agent rabbit mnesia rabbit_common sysmon_handler os_mon 2025-10-20 05:23:14.124 [info] <0.744.0> Stopping application 'rabbitmq_management' 2025-10-20 05:23:14.126 [warning] <0.582.0> RabbitMQ HTTP listener registry could not find context rabbitmq_management_tls 2025-10-20 05:23:14.127 [info] <0.744.0> Stopping application 'amqp_client' 2025-10-20 05:23:14.127 [info] <0.43.0> Application rabbitmq_management exited with reason: stopped 2025-10-20 05:23:14.129 [info] <0.43.0> Application amqp_client exited with reason: stopped 2025-10-20 05:23:14.129 [info] <0.744.0> Stopping application 'rabbitmq_web_dispatch' 2025-10-20 05:23:14.130 [info] <0.744.0> Stopping application 'cowboy' 2025-10-20 05:23:14.130 [info] <0.43.0> Application rabbitmq_web_dispatch exited with reason: stopped 2025-10-20 05:23:14.131 [info] <0.43.0> Application cowboy exited with reason: stopped 2025-10-20 05:23:14.131 [info] <0.744.0> Stopping application 'cowlib' 2025-10-20 05:23:14.131 [info] <0.43.0> Application cowlib exited with reason: stopped 2025-10-20 05:23:14.131 [info] <0.744.0> Stopping application 'rabbitmq_management_agent' 2025-10-20 05:23:14.133 [info] <0.43.0> Application rabbitmq_management_agent exited with reason: stopped 2025-10-20 05:23:14.133 [info] <0.744.0> Stopping application 'rabbit' 2025-10-20 05:23:14.133 [info] <0.240.0> Will unregister with peer discovery backend rabbit_peer_discovery_classic_config 2025-10-20 05:23:14.134 [info] <0.574.0> stopped TCP listener on [::]:5673 2025-10-20 05:23:14.135 [error] <0.762.0> Error on AMQP connection <0.762.0> (10.0.0.49:47776 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:23:14.135 [error] <0.817.0> Error on AMQP connection <0.817.0> (10.0.0.48:60670 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:23:14.135 [error] <0.776.0> Error on AMQP connection <0.776.0> (10.0.0.48:60658 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:23:14.135 [error] <0.747.0> Error on AMQP connection <0.747.0> (10.0.0.254:38650 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:23:14.135 [error] <0.792.0> Error on AMQP connection <0.792.0> (10.0.0.48:60668 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:23:14.135 [error] <0.753.0> Error on AMQP connection <0.753.0> (10.0.0.49:47760 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:23:14.135 [error] <0.750.0> Error on AMQP connection <0.750.0> (10.0.0.49:47758 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:23:14.135 [error] <0.759.0> Error on AMQP connection <0.759.0> (10.0.0.254:38654 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:23:14.135 [error] <0.765.0> Error on AMQP connection <0.765.0> (10.0.0.254:38662 -> 10.0.0.49:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-10-20 05:23:14.136 [error] <0.793.0> Supervisor {<0.793.0>,rabbit_channel_sup_sup} had child channel_sup started with rabbit_channel_sup:start_link() at undefined exit with reason shutdown in context shutdown_error 2025-10-20 05:23:14.137 [error] <0.827.0> Supervisor {<0.827.0>,rabbit_channel_sup_sup} had child channel_sup started with rabbit_channel_sup:start_link() at undefined exit with reason shutdown in context shutdown_error 2025-10-20 05:23:14.143 [info] <0.498.0> Closing all connections in vhost '/' on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4371-3' because the vhost is stopping 2025-10-20 05:23:14.144 [warning] <0.888.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-4371-2' in vhost '/': Stopping all nodes on master shutdown since no synchronised slave is available 2025-10-20 05:23:14.144 [warning] <0.808.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-4371-3.' in vhost '/': Stopping all nodes on master shutdown since no synchronised slave is available 2025-10-20 05:23:14.144 [warning] <0.881.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-4371-2.' in vhost '/': Stopping all nodes on master shutdown since no synchronised slave is available 2025-10-20 05:23:14.144 [warning] <0.835.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-4371-1.' in vhost '/': Stopping all nodes on master shutdown since no synchronised slave is available 2025-10-20 05:23:14.144 [warning] <0.855.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-4371-1' in vhost '/': Stopping all nodes on master shutdown since no synchronised slave is available 2025-10-20 05:23:14.144 [warning] <0.844.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-4371-3' in vhost '/': Stopping all nodes on master shutdown since no synchronised slave is available 2025-10-20 05:23:14.144 [warning] <0.872.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-4371-2' in vhost '/': Stopping all nodes on master shutdown since no synchronised slave is available 2025-10-20 05:23:14.145 [warning] <0.799.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-4371-1' in vhost '/': Stopping all nodes on master shutdown since no synchronised slave is available 2025-10-20 05:23:14.145 [warning] <0.821.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-4371-3' in vhost '/': Stopping all nodes on master shutdown since no synchronised slave is available 2025-10-20 05:23:14.148 [info] <0.528.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' 2025-10-20 05:23:14.171 [info] <0.528.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' is stopped 2025-10-20 05:23:14.172 [info] <0.525.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' 2025-10-20 05:23:14.178 [info] <0.525.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4371-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' is stopped 2025-10-20 05:23:14.194 [info] <0.744.0> Stopping application 'mnesia' 2025-10-20 05:23:14.194 [info] <0.43.0> Application rabbit exited with reason: stopped 2025-10-20 05:23:14.204 [info] <0.744.0> Stopping application 'rabbit_common' 2025-10-20 05:23:14.204 [info] <0.744.0> Stopping application 'sysmon_handler' 2025-10-20 05:23:15.237 [warning] lager_file_backend dropped 6 messages in the last second that exceeded the limit of 50 messages/sec