2026-05-28 02:07:47.661964+00:00 [notice] <0.222.0> Logging: configured log handlers are now ACTIVE
2026-05-28 02:07:47.823396+00:00 [info] <0.222.0> ra: starting system quorum_queues
2026-05-28 02:07:47.823493+00:00 [info] <0.222.0> starting Ra system: quorum_queues in directory: /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-5841-3/quorum/contrail@cn-jenkins-deploy-platform-ansible-os-5841-3
2026-05-28 02:07:47.889384+00:00 [info] <0.259.0> ra system 'quorum_queues' running pre init for 0 registered servers
2026-05-28 02:07:47.901198+00:00 [info] <0.260.0> ra: meta data store initialised for system quorum_queues. 0 record(s) recovered
2026-05-28 02:07:47.918071+00:00 [notice] <0.265.0> WAL: ra_log_wal init, open tbls: ra_log_open_mem_tables, closed tbls: ra_log_closed_mem_tables
2026-05-28 02:07:47.936480+00:00 [info] <0.222.0> ra: starting system coordination
2026-05-28 02:07:47.936562+00:00 [info] <0.222.0> starting Ra system: coordination in directory: /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-5841-3/coordination/contrail@cn-jenkins-deploy-platform-ansible-os-5841-3
2026-05-28 02:07:47.937940+00:00 [info] <0.272.0> ra system 'coordination' running pre init for 0 registered servers
2026-05-28 02:07:47.939938+00:00 [info] <0.273.0> ra: meta data store initialised for system coordination. 0 record(s) recovered
2026-05-28 02:07:47.940281+00:00 [notice] <0.278.0> WAL: ra_coordination_log_wal init, open tbls: ra_coordination_log_open_mem_tables, closed tbls: ra_coordination_log_closed_mem_tables
2026-05-28 02:07:47.946869+00:00 [info] <0.222.0> 
2026-05-28 02:07:47.946869+00:00 [info] <0.222.0>  Starting RabbitMQ 3.10.7 on Erlang 25.0.4 [jit]
2026-05-28 02:07:47.946869+00:00 [info] <0.222.0>  Copyright (c) 2007-2022 VMware, Inc. or its affiliates.
2026-05-28 02:07:47.946869+00:00 [info] <0.222.0>  Licensed under the MPL 2.0. Website: https://rabbitmq.com
2026-05-28 02:07:47.948745+00:00 [info] <0.222.0> 
2026-05-28 02:07:47.948745+00:00 [info] <0.222.0>  node           : contrail@cn-jenkins-deploy-platform-ansible-os-5841-3
2026-05-28 02:07:47.948745+00:00 [info] <0.222.0>  home dir       : /var/lib/rabbitmq
2026-05-28 02:07:47.948745+00:00 [info] <0.222.0>  config file(s) : /etc/rabbitmq/rabbitmq.config
2026-05-28 02:07:47.948745+00:00 [info] <0.222.0>                 : /etc/rabbitmq/conf.d/10-defaults.conf
2026-05-28 02:07:47.948745+00:00 [info] <0.222.0>  cookie hash    : esZsDxSN6VGbi9JkMSxNZA==
2026-05-28 02:07:47.948745+00:00 [info] <0.222.0>  log(s)         : /var/log/contrail/config-database-rabbitmq/rabbitmq.log
2026-05-28 02:07:47.948745+00:00 [info] <0.222.0>                 : /var/log/rabbitmq/contrail@cn-jenkins-deploy-platform-ansible-os-5841-3_upgrade.log
2026-05-28 02:07:47.948745+00:00 [info] <0.222.0>                 : <stdout>
2026-05-28 02:07:47.948745+00:00 [info] <0.222.0>  database dir   : /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-5841-3
2026-05-28 02:07:48.291931+00:00 [info] <0.222.0> Feature flags: list of feature flags found:
2026-05-28 02:07:48.292004+00:00 [info] <0.222.0> Feature flags:   [ ] classic_mirrored_queue_version
2026-05-28 02:07:48.292027+00:00 [info] <0.222.0> Feature flags:   [ ] drop_unroutable_metric
2026-05-28 02:07:48.292054+00:00 [info] <0.222.0> Feature flags:   [ ] empty_basic_get_metric
2026-05-28 02:07:48.292114+00:00 [info] <0.222.0> Feature flags:   [ ] implicit_default_bindings
2026-05-28 02:07:48.292132+00:00 [info] <0.222.0> Feature flags:   [ ] maintenance_mode_status
2026-05-28 02:07:48.292150+00:00 [info] <0.222.0> Feature flags:   [ ] quorum_queue
2026-05-28 02:07:48.292221+00:00 [info] <0.222.0> Feature flags:   [ ] stream_queue
2026-05-28 02:07:48.292248+00:00 [info] <0.222.0> Feature flags:   [ ] user_limits
2026-05-28 02:07:48.292272+00:00 [info] <0.222.0> Feature flags:   [ ] virtual_host_metadata
2026-05-28 02:07:48.292302+00:00 [info] <0.222.0> Feature flags: feature flag states written to disk: yes
2026-05-28 02:07:50.629471+00:00 [info] <0.222.0> Running boot step pre_boot defined by app rabbit
2026-05-28 02:07:50.629547+00:00 [info] <0.222.0> Running boot step rabbit_global_counters defined by app rabbit
2026-05-28 02:07:50.629904+00:00 [info] <0.222.0> Running boot step rabbit_osiris_metrics defined by app rabbit
2026-05-28 02:07:50.630021+00:00 [info] <0.222.0> Running boot step rabbit_core_metrics defined by app rabbit
2026-05-28 02:07:50.630567+00:00 [info] <0.222.0> Running boot step rabbit_alarm defined by app rabbit
2026-05-28 02:07:50.636988+00:00 [info] <0.299.0> Memory high watermark set to 12637 MiB (13251084288 bytes) of 15796 MiB (16563855360 bytes) total
2026-05-28 02:07:50.641085+00:00 [info] <0.301.0> Enabling free disk space monitoring
2026-05-28 02:07:50.641146+00:00 [info] <0.301.0> Disk free limit set to 50MB
2026-05-28 02:07:50.646998+00:00 [info] <0.222.0> Running boot step code_server_cache defined by app rabbit
2026-05-28 02:07:50.647125+00:00 [info] <0.222.0> Running boot step file_handle_cache defined by app rabbit
2026-05-28 02:07:50.647361+00:00 [info] <0.306.0> Limiting to approx 927 file handles (832 sockets)
2026-05-28 02:07:50.647499+00:00 [info] <0.307.0> FHC read buffering: OFF
2026-05-28 02:07:50.647741+00:00 [info] <0.307.0> FHC write buffering: ON
2026-05-28 02:07:50.648224+00:00 [info] <0.222.0> Running boot step worker_pool defined by app rabbit
2026-05-28 02:07:50.648289+00:00 [info] <0.280.0> Will use 4 processes for default worker pool
2026-05-28 02:07:50.648324+00:00 [info] <0.280.0> Starting worker pool 'worker_pool' with 4 processes in it
2026-05-28 02:07:50.648556+00:00 [info] <0.222.0> Running boot step database defined by app rabbit
2026-05-28 02:07:50.648924+00:00 [info] <0.222.0> Node database directory at /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-5841-3 is empty. Assuming we need to join an existing cluster or initialise from scratch...
2026-05-28 02:07:50.648969+00:00 [info] <0.222.0> Configured peer discovery backend: rabbit_peer_discovery_classic_config
2026-05-28 02:07:50.648993+00:00 [info] <0.222.0> Will try to lock with peer discovery backend rabbit_peer_discovery_classic_config
2026-05-28 02:07:55.197166+00:00 [info] <0.222.0> All discovered existing cluster peers: contrail@cn-jenkins-deploy-platform-ansible-os-5841-1, contrail@cn-jenkins-deploy-platform-ansible-os-5841-2, contrail@cn-jenkins-deploy-platform-ansible-os-5841-3
2026-05-28 02:07:55.197346+00:00 [info] <0.222.0> Peer nodes we can cluster with: contrail@cn-jenkins-deploy-platform-ansible-os-5841-1, contrail@cn-jenkins-deploy-platform-ansible-os-5841-2
2026-05-28 02:07:55.198229+00:00 [warning] <0.222.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-5841-1: {error,
2026-05-28 02:07:55.198229+00:00 [warning] <0.222.0>                                                                                          tables_not_present}
2026-05-28 02:07:55.201860+00:00 [info] <0.222.0> Node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-2' selected for auto-clustering
2026-05-28 02:07:55.220160+00:00 [info] <0.222.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
2026-05-28 02:07:55.220372+00:00 [info] <0.222.0> Successfully synced tables from a peer
2026-05-28 02:07:55.462767+00:00 [info] <0.222.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
2026-05-28 02:07:55.462958+00:00 [info] <0.222.0> Successfully synced tables from a peer
2026-05-28 02:07:55.463030+00:00 [warning] <0.222.0> Feature flags: the previous instance of this node must have failed to write the `feature_flags` file at `/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-5841-3-feature_flags`:
2026-05-28 02:07:55.463104+00:00 [warning] <0.222.0> Feature flags:   - list of previously enabled feature flags now marked as such: [maintenance_mode_status,
2026-05-28 02:07:55.463104+00:00 [warning] <0.222.0>                                                                                  quorum_queue,
2026-05-28 02:07:55.463104+00:00 [warning] <0.222.0>                                                                                  user_limits,
2026-05-28 02:07:55.463104+00:00 [warning] <0.222.0>                                                                                  virtual_host_metadata]
2026-05-28 02:07:55.464884+00:00 [info] <0.222.0> Feature flags: list of feature flags found:
2026-05-28 02:07:55.464953+00:00 [info] <0.222.0> Feature flags:   [ ] classic_mirrored_queue_version
2026-05-28 02:07:55.464987+00:00 [info] <0.222.0> Feature flags:   [ ] drop_unroutable_metric
2026-05-28 02:07:55.465013+00:00 [info] <0.222.0> Feature flags:   [ ] empty_basic_get_metric
2026-05-28 02:07:55.465085+00:00 [info] <0.222.0> Feature flags:   [ ] implicit_default_bindings
2026-05-28 02:07:55.465114+00:00 [info] <0.222.0> Feature flags:   [x] maintenance_mode_status
2026-05-28 02:07:55.465141+00:00 [info] <0.222.0> Feature flags:   [x] quorum_queue
2026-05-28 02:07:55.465183+00:00 [info] <0.222.0> Feature flags:   [ ] stream_queue
2026-05-28 02:07:55.465242+00:00 [info] <0.222.0> Feature flags:   [x] user_limits
2026-05-28 02:07:55.465265+00:00 [info] <0.222.0> Feature flags:   [x] virtual_host_metadata
2026-05-28 02:07:55.465289+00:00 [info] <0.222.0> Feature flags: feature flag states written to disk: yes
2026-05-28 02:07:55.475681+00:00 [info] <0.222.0> Feature flag `classic_mirrored_queue_version`: mark as enabled=true
2026-05-28 02:07:55.478162+00:00 [info] <0.222.0> Feature flags: list of feature flags found:
2026-05-28 02:07:55.478235+00:00 [info] <0.222.0> Feature flags:   [x] classic_mirrored_queue_version
2026-05-28 02:07:55.478259+00:00 [info] <0.222.0> Feature flags:   [ ] drop_unroutable_metric
2026-05-28 02:07:55.478278+00:00 [info] <0.222.0> Feature flags:   [ ] empty_basic_get_metric
2026-05-28 02:07:55.478318+00:00 [info] <0.222.0> Feature flags:   [ ] implicit_default_bindings
2026-05-28 02:07:55.478337+00:00 [info] <0.222.0> Feature flags:   [x] maintenance_mode_status
2026-05-28 02:07:55.478355+00:00 [info] <0.222.0> Feature flags:   [x] quorum_queue
2026-05-28 02:07:55.478371+00:00 [info] <0.222.0> Feature flags:   [ ] stream_queue
2026-05-28 02:07:55.478438+00:00 [info] <0.222.0> Feature flags:   [x] user_limits
2026-05-28 02:07:55.478454+00:00 [info] <0.222.0> Feature flags:   [x] virtual_host_metadata
2026-05-28 02:07:55.478470+00:00 [info] <0.222.0> Feature flags: feature flag states written to disk: yes
2026-05-28 02:07:55.488604+00:00 [info] <0.222.0> Feature flag `drop_unroutable_metric`: mark as enabled=true
2026-05-28 02:07:55.490498+00:00 [info] <0.222.0> Feature flags: list of feature flags found:
2026-05-28 02:07:55.490579+00:00 [info] <0.222.0> Feature flags:   [x] classic_mirrored_queue_version
2026-05-28 02:07:55.490605+00:00 [info] <0.222.0> Feature flags:   [x] drop_unroutable_metric
2026-05-28 02:07:55.490623+00:00 [info] <0.222.0> Feature flags:   [ ] empty_basic_get_metric
2026-05-28 02:07:55.490660+00:00 [info] <0.222.0> Feature flags:   [ ] implicit_default_bindings
2026-05-28 02:07:55.490678+00:00 [info] <0.222.0> Feature flags:   [x] maintenance_mode_status
2026-05-28 02:07:55.490699+00:00 [info] <0.222.0> Feature flags:   [x] quorum_queue
2026-05-28 02:07:55.490716+00:00 [info] <0.222.0> Feature flags:   [ ] stream_queue
2026-05-28 02:07:55.490828+00:00 [info] <0.222.0> Feature flags:   [x] user_limits
2026-05-28 02:07:55.490870+00:00 [info] <0.222.0> Feature flags:   [x] virtual_host_metadata
2026-05-28 02:07:55.490890+00:00 [info] <0.222.0> Feature flags: feature flag states written to disk: yes
2026-05-28 02:07:55.499918+00:00 [info] <0.222.0> Feature flag `empty_basic_get_metric`: mark as enabled=true
2026-05-28 02:07:55.501591+00:00 [info] <0.222.0> Feature flags: list of feature flags found:
2026-05-28 02:07:55.501666+00:00 [info] <0.222.0> Feature flags:   [x] classic_mirrored_queue_version
2026-05-28 02:07:55.501692+00:00 [info] <0.222.0> Feature flags:   [x] drop_unroutable_metric
2026-05-28 02:07:55.501710+00:00 [info] <0.222.0> Feature flags:   [x] empty_basic_get_metric
2026-05-28 02:07:55.501755+00:00 [info] <0.222.0> Feature flags:   [ ] implicit_default_bindings
2026-05-28 02:07:55.501774+00:00 [info] <0.222.0> Feature flags:   [x] maintenance_mode_status
2026-05-28 02:07:55.501790+00:00 [info] <0.222.0> Feature flags:   [x] quorum_queue
2026-05-28 02:07:55.501805+00:00 [info] <0.222.0> Feature flags:   [ ] stream_queue
2026-05-28 02:07:55.501991+00:00 [info] <0.222.0> Feature flags:   [x] user_limits
2026-05-28 02:07:55.502007+00:00 [info] <0.222.0> Feature flags:   [x] virtual_host_metadata
2026-05-28 02:07:55.502023+00:00 [info] <0.222.0> Feature flags: feature flag states written to disk: yes
2026-05-28 02:07:55.510164+00:00 [info] <0.222.0> Waiting for Mnesia tables for 30000 ms, 0 retries left
2026-05-28 02:07:55.510263+00:00 [info] <0.222.0> Successfully synced tables from a peer
2026-05-28 02:07:55.510322+00:00 [info] <0.222.0> Feature flag `implicit_default_bindings`: mark as enabled=true
2026-05-28 02:07:55.512347+00:00 [info] <0.222.0> Feature flags: list of feature flags found:
2026-05-28 02:07:55.512424+00:00 [info] <0.222.0> Feature flags:   [x] classic_mirrored_queue_version
2026-05-28 02:07:55.512466+00:00 [info] <0.222.0> Feature flags:   [x] drop_unroutable_metric
2026-05-28 02:07:55.512487+00:00 [info] <0.222.0> Feature flags:   [x] empty_basic_get_metric
2026-05-28 02:07:55.512505+00:00 [info] <0.222.0> Feature flags:   [x] implicit_default_bindings
2026-05-28 02:07:55.512523+00:00 [info] <0.222.0> Feature flags:   [x] maintenance_mode_status
2026-05-28 02:07:55.512565+00:00 [info] <0.222.0> Feature flags:   [x] quorum_queue
2026-05-28 02:07:55.512607+00:00 [info] <0.222.0> Feature flags:   [ ] stream_queue
2026-05-28 02:07:55.512648+00:00 [info] <0.222.0> Feature flags:   [x] user_limits
2026-05-28 02:07:55.512673+00:00 [info] <0.222.0> Feature flags:   [x] virtual_host_metadata
2026-05-28 02:07:55.512735+00:00 [info] <0.222.0> Feature flags: feature flag states written to disk: yes
2026-05-28 02:07:55.520067+00:00 [info] <0.222.0> Feature flag `stream_queue`: mark as enabled=true
2026-05-28 02:07:55.521759+00:00 [info] <0.222.0> Feature flags: list of feature flags found:
2026-05-28 02:07:55.521831+00:00 [info] <0.222.0> Feature flags:   [x] classic_mirrored_queue_version
2026-05-28 02:07:55.521854+00:00 [info] <0.222.0> Feature flags:   [x] drop_unroutable_metric
2026-05-28 02:07:55.521871+00:00 [info] <0.222.0> Feature flags:   [x] empty_basic_get_metric
2026-05-28 02:07:55.521889+00:00 [info] <0.222.0> Feature flags:   [x] implicit_default_bindings
2026-05-28 02:07:55.521908+00:00 [info] <0.222.0> Feature flags:   [x] maintenance_mode_status
2026-05-28 02:07:55.521925+00:00 [info] <0.222.0> Feature flags:   [x] quorum_queue
2026-05-28 02:07:55.521941+00:00 [info] <0.222.0> Feature flags:   [x] stream_queue
2026-05-28 02:07:55.521974+00:00 [info] <0.222.0> Feature flags:   [x] user_limits
2026-05-28 02:07:55.521997+00:00 [info] <0.222.0> Feature flags:   [x] virtual_host_metadata
2026-05-28 02:07:55.522013+00:00 [info] <0.222.0> Feature flags: feature flag states written to disk: yes
2026-05-28 02:07:55.533214+00:00 [info] <0.222.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
2026-05-28 02:07:55.533366+00:00 [info] <0.222.0> Successfully synced tables from a peer
2026-05-28 02:07:55.566807+00:00 [info] <0.222.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
2026-05-28 02:07:55.567082+00:00 [info] <0.222.0> Successfully synced tables from a peer
2026-05-28 02:07:55.600901+00:00 [info] <0.222.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-5841-3'
2026-05-28 02:07:55.610825+00:00 [info] <0.222.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-5841-3'
2026-05-28 02:07:55.621599+00:00 [info] <0.222.0> Setting up a table for per-user connection counting on this node: 'tracked_connection_table_per_user_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-5841-3'
2026-05-28 02:07:55.622677+00:00 [info] <0.222.0> Peer discovery backend rabbit_peer_discovery_classic_config does not support registration, skipping registration.
2026-05-28 02:07:55.622750+00:00 [info] <0.222.0> Will try to unlock with peer discovery backend rabbit_peer_discovery_classic_config
2026-05-28 02:07:55.623623+00:00 [info] <0.222.0> Running boot step database_sync defined by app rabbit
2026-05-28 02:07:55.623754+00:00 [info] <0.222.0> Running boot step feature_flags defined by app rabbit
2026-05-28 02:07:55.623895+00:00 [info] <0.222.0> Running boot step codec_correctness_check defined by app rabbit
2026-05-28 02:07:55.623964+00:00 [info] <0.222.0> Running boot step external_infrastructure defined by app rabbit
2026-05-28 02:07:55.623998+00:00 [info] <0.222.0> Running boot step rabbit_event defined by app rabbit
2026-05-28 02:07:55.624197+00:00 [info] <0.222.0> Running boot step rabbit_registry defined by app rabbit
2026-05-28 02:07:55.624302+00:00 [info] <0.222.0> Running boot step rabbit_auth_mechanism_amqplain defined by app rabbit
2026-05-28 02:07:55.624439+00:00 [info] <0.222.0> Running boot step rabbit_auth_mechanism_cr_demo defined by app rabbit
2026-05-28 02:07:55.624541+00:00 [info] <0.222.0> Running boot step rabbit_auth_mechanism_plain defined by app rabbit
2026-05-28 02:07:55.624605+00:00 [info] <0.222.0> Running boot step rabbit_exchange_type_direct defined by app rabbit
2026-05-28 02:07:55.624697+00:00 [info] <0.222.0> Running boot step rabbit_exchange_type_fanout defined by app rabbit
2026-05-28 02:07:55.624783+00:00 [info] <0.222.0> Running boot step rabbit_exchange_type_headers defined by app rabbit
2026-05-28 02:07:55.624897+00:00 [info] <0.222.0> Running boot step rabbit_exchange_type_topic defined by app rabbit
2026-05-28 02:07:55.624974+00:00 [info] <0.222.0> Running boot step rabbit_mirror_queue_mode_all defined by app rabbit
2026-05-28 02:07:55.625045+00:00 [info] <0.222.0> Running boot step rabbit_mirror_queue_mode_exactly defined by app rabbit
2026-05-28 02:07:55.625154+00:00 [info] <0.222.0> Running boot step rabbit_mirror_queue_mode_nodes defined by app rabbit
2026-05-28 02:07:55.625325+00:00 [info] <0.222.0> Running boot step rabbit_priority_queue defined by app rabbit
2026-05-28 02:07:55.625373+00:00 [info] <0.222.0> Priority queues enabled, real BQ is rabbit_variable_queue
2026-05-28 02:07:55.625454+00:00 [info] <0.222.0> Running boot step rabbit_queue_location_client_local defined by app rabbit
2026-05-28 02:07:55.625522+00:00 [info] <0.222.0> Running boot step rabbit_queue_location_min_masters defined by app rabbit
2026-05-28 02:07:55.625567+00:00 [info] <0.222.0> Running boot step rabbit_queue_location_random defined by app rabbit
2026-05-28 02:07:55.625615+00:00 [info] <0.222.0> Running boot step kernel_ready defined by app rabbit
2026-05-28 02:07:55.625658+00:00 [info] <0.222.0> Running boot step rabbit_sysmon_minder defined by app rabbit
2026-05-28 02:07:55.625792+00:00 [info] <0.222.0> Running boot step rabbit_epmd_monitor defined by app rabbit
2026-05-28 02:07:55.626664+00:00 [info] <0.499.0> epmd monitor knows us, inter-node communication (distribution) port: 25673
2026-05-28 02:07:55.626778+00:00 [info] <0.222.0> Running boot step guid_generator defined by app rabbit
2026-05-28 02:07:55.630294+00:00 [info] <0.222.0> Running boot step rabbit_node_monitor defined by app rabbit
2026-05-28 02:07:55.630971+00:00 [info] <0.503.0> Starting rabbit_node_monitor
2026-05-28 02:07:55.631103+00:00 [info] <0.222.0> Running boot step delegate_sup defined by app rabbit
2026-05-28 02:07:55.631688+00:00 [info] <0.222.0> Running boot step rabbit_memory_monitor defined by app rabbit
2026-05-28 02:07:55.631893+00:00 [info] <0.222.0> Running boot step rabbit_fifo_dlx_sup defined by app rabbit
2026-05-28 02:07:55.632011+00:00 [info] <0.222.0> Running boot step core_initialized defined by app rabbit
2026-05-28 02:07:55.632072+00:00 [info] <0.222.0> Running boot step upgrade_queues defined by app rabbit
2026-05-28 02:07:55.640140+00:00 [info] <0.222.0> message_store upgrades: 1 to apply
2026-05-28 02:07:55.640255+00:00 [info] <0.222.0> message_store upgrades: Applying rabbit_variable_queue:move_messages_to_vhost_store
2026-05-28 02:07:55.640340+00:00 [info] <0.222.0> message_store upgrades: No durable queues found. Skipping message store migration
2026-05-28 02:07:55.640374+00:00 [info] <0.222.0> message_store upgrades: Removing the old message store data
2026-05-28 02:07:55.640759+00:00 [info] <0.222.0> message_store upgrades: All upgrades applied successfully
2026-05-28 02:07:55.651511+00:00 [info] <0.222.0> Running boot step channel_tracking defined by app rabbit
2026-05-28 02:07:55.660974+00:00 [info] <0.222.0> Setting up a table for channel tracking on this node: 'tracked_channel_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-5841-3'
2026-05-28 02:07:55.668406+00:00 [info] <0.222.0> Setting up a table for channel tracking on this node: 'tracked_channel_table_per_user_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-5841-3'
2026-05-28 02:07:55.668699+00:00 [info] <0.222.0> Running boot step rabbit_channel_tracking_handler defined by app rabbit
2026-05-28 02:07:55.668802+00:00 [info] <0.222.0> Running boot step connection_tracking defined by app rabbit
2026-05-28 02:07:55.669302+00:00 [info] <0.222.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-5841-3'
2026-05-28 02:07:55.669994+00:00 [info] <0.222.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-5841-3'
2026-05-28 02:07:55.670433+00:00 [info] <0.222.0> Setting up a table for per-user connection counting on this node: 'tracked_connection_table_per_user_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-5841-3'
2026-05-28 02:07:55.670666+00:00 [info] <0.222.0> Running boot step rabbit_connection_tracking_handler defined by app rabbit
2026-05-28 02:07:55.670722+00:00 [info] <0.222.0> Running boot step rabbit_definitions_hashing defined by app rabbit
2026-05-28 02:07:55.670875+00:00 [info] <0.222.0> Running boot step rabbit_exchange_parameters defined by app rabbit
2026-05-28 02:07:55.670954+00:00 [info] <0.222.0> Running boot step rabbit_mirror_queue_misc defined by app rabbit
2026-05-28 02:07:55.671046+00:00 [info] <0.222.0> Running boot step rabbit_policies defined by app rabbit
2026-05-28 02:07:55.671259+00:00 [info] <0.222.0> Running boot step rabbit_policy defined by app rabbit
2026-05-28 02:07:55.671306+00:00 [info] <0.222.0> Running boot step rabbit_queue_location_validator defined by app rabbit
2026-05-28 02:07:55.671340+00:00 [info] <0.222.0> Running boot step rabbit_quorum_memory_manager defined by app rabbit
2026-05-28 02:07:55.671374+00:00 [info] <0.222.0> Running boot step rabbit_stream_coordinator defined by app rabbit
2026-05-28 02:07:55.671455+00:00 [info] <0.222.0> Running boot step rabbit_vhost_limit defined by app rabbit
2026-05-28 02:07:55.671491+00:00 [info] <0.222.0> Running boot step rabbit_mgmt_reset_handler defined by app rabbitmq_management
2026-05-28 02:07:55.671521+00:00 [info] <0.222.0> Running boot step rabbit_mgmt_db_handler defined by app rabbitmq_management_agent
2026-05-28 02:07:55.671551+00:00 [warning] <0.222.0> force_fine_statistics set to true; ignored.
2026-05-28 02:07:55.671551+00:00 [warning] <0.222.0> Replaced by {rates_mode, none} in the rabbitmq_management application.
2026-05-28 02:07:55.671581+00:00 [info] <0.222.0> Management plugin: using rates mode 'basic'
2026-05-28 02:07:55.673898+00:00 [info] <0.222.0> Running boot step recovery defined by app rabbit
2026-05-28 02:07:55.674951+00:00 [info] <0.552.0> Making sure data directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-5841-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists
2026-05-28 02:07:55.675886+00:00 [info] <0.552.0> Setting segment_entry_count for vhost '/' with 0 queues to '2048'
2026-05-28 02:07:55.681152+00:00 [info] <0.552.0> Starting message stores for vhost '/'
2026-05-28 02:07:55.681495+00:00 [info] <0.557.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index
2026-05-28 02:07:55.682624+00:00 [info] <0.552.0> Started message store of type transient for vhost '/'
2026-05-28 02:07:55.682915+00:00 [info] <0.561.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index
2026-05-28 02:07:55.684276+00:00 [warning] <0.561.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": rebuilding indices from scratch
2026-05-28 02:07:55.684875+00:00 [info] <0.552.0> Started message store of type persistent for vhost '/'
2026-05-28 02:07:55.685402+00:00 [info] <0.552.0> Recovering 0 queues of type rabbit_classic_queue took 9ms
2026-05-28 02:07:55.685447+00:00 [info] <0.552.0> Recovering 0 queues of type rabbit_quorum_queue took 0ms
2026-05-28 02:07:55.685472+00:00 [info] <0.552.0> Recovering 0 queues of type rabbit_stream_queue took 0ms
2026-05-28 02:07:55.686730+00:00 [info] <0.222.0> Running boot step empty_db_check defined by app rabbit
2026-05-28 02:07:55.686799+00:00 [info] <0.222.0> Will not seed default virtual host and user: have definitions to load...
2026-05-28 02:07:55.686899+00:00 [info] <0.222.0> Running boot step rabbit_looking_glass defined by app rabbit
2026-05-28 02:07:55.686938+00:00 [info] <0.222.0> Running boot step rabbit_core_metrics_gc defined by app rabbit
2026-05-28 02:07:55.687086+00:00 [info] <0.222.0> Running boot step background_gc defined by app rabbit
2026-05-28 02:07:55.687219+00:00 [info] <0.222.0> Running boot step routing_ready defined by app rabbit
2026-05-28 02:07:55.687271+00:00 [info] <0.222.0> Running boot step pre_flight defined by app rabbit
2026-05-28 02:07:55.687291+00:00 [info] <0.222.0> Running boot step notify_cluster defined by app rabbit
2026-05-28 02:07:55.687331+00:00 [info] <0.222.0> Running boot step networking defined by app rabbit
2026-05-28 02:07:55.687353+00:00 [info] <0.222.0> Running boot step definition_import_worker_pool defined by app rabbit
2026-05-28 02:07:55.687397+00:00 [info] <0.280.0> Starting worker pool 'definition_import_pool' with 4 processes in it
2026-05-28 02:07:55.687582+00:00 [info] <0.222.0> Running boot step cluster_name defined by app rabbit
2026-05-28 02:07:55.687628+00:00 [info] <0.222.0> Running boot step direct_client defined by app rabbit
2026-05-28 02:07:55.687702+00:00 [info] <0.222.0> Running boot step rabbit_maintenance_mode_state defined by app rabbit
2026-05-28 02:07:55.687731+00:00 [info] <0.222.0> Creating table rabbit_node_maintenance_states for feature flag `maintenance_mode_status`
2026-05-28 02:07:55.688082+00:00 [info] <0.503.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-2' up
2026-05-28 02:07:55.688390+00:00 [info] <0.222.0> Running boot step rabbit_management_load_definitions defined by app rabbitmq_management
2026-05-28 02:07:55.688780+00:00 [info] <0.222.0> Applying definitions from file at '/etc/rabbitmq/definitions.json'
2026-05-28 02:07:55.688846+00:00 [info] <0.222.0> Asked to import definitions. Acting user: rmq-internal
2026-05-28 02:07:55.688976+00:00 [info] <0.222.0> Importing concurrently 1 users...
2026-05-28 02:07:55.694440+00:00 [info] <0.222.0> Importing concurrently 1 vhosts...
2026-05-28 02:07:55.699039+00:00 [info] <0.222.0> Importing concurrently 1 permissions...
2026-05-28 02:07:55.703477+00:00 [info] <0.585.0> Successfully set permissions for 'guest' in virtual host '/' to '.*', '.*', '.*'
2026-05-28 02:07:55.703624+00:00 [info] <0.222.0> Importing sequentially 1 policies...
2026-05-28 02:07:55.707275+00:00 [info] <0.596.0> Resetting node maintenance status
2026-05-28 02:07:55.717913+00:00 [info] <0.655.0> Management plugin: HTTP (non-TLS) listener started on port 15673
2026-05-28 02:07:55.718041+00:00 [info] <0.683.0> Statistics database started.
2026-05-28 02:07:55.718124+00:00 [info] <0.682.0> Starting worker pool 'management_worker_pool' with 3 processes in it
2026-05-28 02:07:55.725086+00:00 [info] <0.697.0> Prometheus metrics: HTTP (non-TLS) listener started on port 15692
2026-05-28 02:07:55.725224+00:00 [info] <0.596.0> Ready to start client connection listeners
2026-05-28 02:07:55.726320+00:00 [info] <0.741.0> started TCP listener on 10.0.0.131:5673
2026-05-28 02:07:55.808525+00:00 [info] <0.596.0> Server startup complete; 4 plugins started.
2026-05-28 02:07:55.808525+00:00 [info] <0.596.0>  * rabbitmq_prometheus
2026-05-28 02:07:55.808525+00:00 [info] <0.596.0>  * rabbitmq_management
2026-05-28 02:07:55.808525+00:00 [info] <0.596.0>  * rabbitmq_web_dispatch
2026-05-28 02:07:55.808525+00:00 [info] <0.596.0>  * rabbitmq_management_agent
2026-05-28 02:07:56.996205+00:00 [info] <0.503.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-1' up
2026-05-28 02:08:35.835776+00:00 [info] <0.1101.0> RabbitMQ is asked to stop...
2026-05-28 02:08:35.927400+00:00 [info] <0.1101.0> Stopping RabbitMQ applications and their dependencies in the following order:
2026-05-28 02:08:35.927400+00:00 [info] <0.1101.0>     rabbitmq_management
2026-05-28 02:08:35.927400+00:00 [info] <0.1101.0>     rabbitmq_prometheus
2026-05-28 02:08:35.927400+00:00 [info] <0.1101.0>     rabbitmq_web_dispatch
2026-05-28 02:08:35.927400+00:00 [info] <0.1101.0>     cowboy
2026-05-28 02:08:35.927400+00:00 [info] <0.1101.0>     cowlib
2026-05-28 02:08:35.927400+00:00 [info] <0.1101.0>     rabbitmq_management_agent
2026-05-28 02:08:35.927400+00:00 [info] <0.1101.0>     rabbit
2026-05-28 02:08:35.927400+00:00 [info] <0.1101.0>     amqp_client
2026-05-28 02:08:35.927400+00:00 [info] <0.1101.0>     rabbitmq_prelaunch
2026-05-28 02:08:35.927400+00:00 [info] <0.1101.0>     rabbit_common
2026-05-28 02:08:35.927400+00:00 [info] <0.1101.0>     prometheus
2026-05-28 02:08:35.927400+00:00 [info] <0.1101.0>     ra
2026-05-28 02:08:35.927400+00:00 [info] <0.1101.0>     accept
2026-05-28 02:08:35.927400+00:00 [info] <0.1101.0>     os_mon
2026-05-28 02:08:35.927400+00:00 [info] <0.1101.0>     sysmon_handler
2026-05-28 02:08:35.927400+00:00 [info] <0.1101.0>     osiris
2026-05-28 02:08:35.927400+00:00 [info] <0.1101.0>     mnesia
2026-05-28 02:08:35.927400+00:00 [info] <0.1101.0> 
2026-05-28 02:08:35.927527+00:00 [info] <0.1101.0> Stopping application 'rabbitmq_management'
2026-05-28 02:08:35.928998+00:00 [warning] <0.647.0> HTTP listener registry could not find context rabbitmq_management_tls
2026-05-28 02:08:35.930590+00:00 [notice] <0.44.0> Application rabbitmq_management exited with reason: stopped
2026-05-28 02:08:35.930684+00:00 [info] <0.1101.0> Stopping application 'rabbitmq_prometheus'
2026-05-28 02:08:35.931662+00:00 [warning] <0.647.0> HTTP listener registry could not find context rabbitmq_prometheus_tls
2026-05-28 02:08:35.933463+00:00 [notice] <0.44.0> Application rabbitmq_prometheus exited with reason: stopped
2026-05-28 02:08:35.933607+00:00 [info] <0.1101.0> Stopping application 'rabbitmq_web_dispatch'
2026-05-28 02:08:35.942112+00:00 [notice] <0.44.0> Application rabbitmq_web_dispatch exited with reason: stopped
2026-05-28 02:08:35.942197+00:00 [info] <0.1101.0> Stopping application 'cowboy'
2026-05-28 02:08:35.944111+00:00 [notice] <0.44.0> Application cowboy exited with reason: stopped
2026-05-28 02:08:35.944170+00:00 [info] <0.1101.0> Stopping application 'cowlib'
2026-05-28 02:08:35.944209+00:00 [notice] <0.44.0> Application cowlib exited with reason: stopped
2026-05-28 02:08:35.944316+00:00 [info] <0.1101.0> Stopping application 'rabbitmq_management_agent'
2026-05-28 02:08:35.946434+00:00 [notice] <0.44.0> Application rabbitmq_management_agent exited with reason: stopped
2026-05-28 02:08:35.946493+00:00 [info] <0.1101.0> Stopping application 'rabbit'
2026-05-28 02:08:35.953024+00:00 [info] <0.222.0> Peer discovery backend rabbit_peer_discovery_classic_config does not support registration, skipping unregistration.
2026-05-28 02:08:35.953136+00:00 [info] <0.741.0> stopped TCP listener on 10.0.0.131:5673
2026-05-28 02:08:35.954341+00:00 [info] <0.1102.0> Closing all connections in vhost '/' on node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-3' because the vhost is stopping
2026-05-28 02:08:35.954421+00:00 [info] <0.561.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-5841-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent'
2026-05-28 02:08:35.982383+00:00 [info] <0.561.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-5841-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' is stopped
2026-05-28 02:08:35.982660+00:00 [info] <0.557.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-5841-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient'
2026-05-28 02:08:35.987198+00:00 [info] <0.557.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-5841-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' is stopped
2026-05-28 02:08:35.987906+00:00 [info] <0.494.0> Management plugin: to stop collect_statistics.
2026-05-28 02:08:40.991056+00:00 [notice] <0.44.0> Application rabbit exited with reason: stopped
2026-05-28 02:08:40.991179+00:00 [info] <0.1101.0> Stopping application 'amqp_client'
2026-05-28 02:08:40.993026+00:00 [notice] <0.44.0> Application amqp_client exited with reason: stopped
2026-05-28 02:08:40.993142+00:00 [info] <0.1101.0> Stopping application 'rabbitmq_prelaunch'
2026-05-28 02:08:40.995272+00:00 [notice] <0.44.0> Application rabbitmq_prelaunch exited with reason: stopped
2026-05-28 02:08:40.995362+00:00 [info] <0.1101.0> Stopping application 'rabbit_common'
2026-05-28 02:08:40.995419+00:00 [notice] <0.44.0> Application rabbit_common exited with reason: stopped
2026-05-28 02:08:40.995542+00:00 [info] <0.1101.0> Stopping application 'prometheus'
2026-05-28 02:08:40.997521+00:00 [notice] <0.44.0> Application prometheus exited with reason: stopped
2026-05-28 02:08:40.997587+00:00 [info] <0.1101.0> Stopping application 'ra'
2026-05-28 02:08:41.000579+00:00 [notice] <0.44.0> Application ra exited with reason: stopped
2026-05-28 02:08:41.000669+00:00 [info] <0.1101.0> Stopping application 'accept'
2026-05-28 02:08:41.000722+00:00 [notice] <0.44.0> Application accept exited with reason: stopped
2026-05-28 02:08:41.000870+00:00 [info] <0.1101.0> Stopping application 'os_mon'
2026-05-28 02:08:41.002567+00:00 [notice] <0.44.0> Application os_mon exited with reason: stopped
2026-05-28 02:08:41.002669+00:00 [info] <0.1101.0> Stopping application 'sysmon_handler'
2026-05-28 02:08:41.004301+00:00 [notice] <0.44.0> Application sysmon_handler exited with reason: stopped
2026-05-28 02:08:41.004396+00:00 [info] <0.1101.0> Stopping application 'osiris'
2026-05-28 02:08:41.006169+00:00 [notice] <0.44.0> Application osiris exited with reason: stopped
2026-05-28 02:08:41.006257+00:00 [info] <0.1101.0> Stopping application 'mnesia'
2026-05-28 02:08:41.013597+00:00 [notice] <0.44.0> Application mnesia exited with reason: stopped
2026-05-28 02:08:41.013703+00:00 [info] <0.1101.0> Successfully stopped RabbitMQ and its dependencies
2026-05-28 02:08:47.916628+00:00 [info] <0.1174.0> Resetting Rabbit forcefully
2026-05-28 02:08:48.436845+00:00 [error] <0.1184.0> Node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-1' thinks it's clustered with node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-3', but 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-3' disagrees
2026-05-28 02:08:49.004330+00:00 [info] <0.1193.0> Halting Erlang VM with the following applications:
2026-05-28 02:08:49.004330+00:00 [info] <0.1193.0>     amqp10_common
2026-05-28 02:08:49.004330+00:00 [info] <0.1193.0>     observer_cli
2026-05-28 02:08:49.004330+00:00 [info] <0.1193.0>     stdout_formatter
2026-05-28 02:08:49.004330+00:00 [info] <0.1193.0>     seshat
2026-05-28 02:08:49.004330+00:00 [info] <0.1193.0>     gen_batch_server
2026-05-28 02:08:49.004330+00:00 [info] <0.1193.0>     aten
2026-05-28 02:08:49.004330+00:00 [info] <0.1193.0>     ranch
2026-05-28 02:08:49.004330+00:00 [info] <0.1193.0>     inets
2026-05-28 02:08:49.004330+00:00 [info] <0.1193.0>     systemd
2026-05-28 02:08:49.004330+00:00 [info] <0.1193.0>     enough
2026-05-28 02:08:49.004330+00:00 [info] <0.1193.0>     cuttlefish
2026-05-28 02:08:49.004330+00:00 [info] <0.1193.0>     credentials_obfuscation
2026-05-28 02:08:49.004330+00:00 [info] <0.1193.0>     recon
2026-05-28 02:08:49.004330+00:00 [info] <0.1193.0>     jsx
2026-05-28 02:08:49.004330+00:00 [info] <0.1193.0>     xmerl
2026-05-28 02:08:49.004330+00:00 [info] <0.1193.0>     tools
2026-05-28 02:08:49.004330+00:00 [info] <0.1193.0>     syntax_tools
2026-05-28 02:08:49.004330+00:00 [info] <0.1193.0>     ssl
2026-05-28 02:08:49.004330+00:00 [info] <0.1193.0>     public_key
2026-05-28 02:08:49.004330+00:00 [info] <0.1193.0>     asn1
2026-05-28 02:08:49.004330+00:00 [info] <0.1193.0>     crypto
2026-05-28 02:08:49.004330+00:00 [info] <0.1193.0>     compiler
2026-05-28 02:08:49.004330+00:00 [info] <0.1193.0>     sasl
2026-05-28 02:08:49.004330+00:00 [info] <0.1193.0>     stdlib
2026-05-28 02:08:49.004330+00:00 [info] <0.1193.0>     kernel
2026-05-28 02:08:49.004330+00:00 [info] <0.1193.0> 
2026-05-28 02:08:53.261039+00:00 [notice] <0.222.0> Logging: configured log handlers are now ACTIVE
2026-05-28 02:08:53.383429+00:00 [info] <0.222.0> ra: starting system quorum_queues
2026-05-28 02:08:53.383506+00:00 [info] <0.222.0> starting Ra system: quorum_queues in directory: /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-5841-3/quorum/contrail@cn-jenkins-deploy-platform-ansible-os-5841-3
2026-05-28 02:08:53.416187+00:00 [info] <0.259.0> ra system 'quorum_queues' running pre init for 0 registered servers
2026-05-28 02:08:53.421447+00:00 [info] <0.260.0> ra: meta data store initialised for system quorum_queues. 0 record(s) recovered
2026-05-28 02:08:53.429138+00:00 [notice] <0.265.0> WAL: ra_log_wal init, open tbls: ra_log_open_mem_tables, closed tbls: ra_log_closed_mem_tables
2026-05-28 02:08:53.437724+00:00 [info] <0.222.0> ra: starting system coordination
2026-05-28 02:08:53.437790+00:00 [info] <0.222.0> starting Ra system: coordination in directory: /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-5841-3/coordination/contrail@cn-jenkins-deploy-platform-ansible-os-5841-3
2026-05-28 02:08:53.438625+00:00 [info] <0.272.0> ra system 'coordination' running pre init for 0 registered servers
2026-05-28 02:08:53.439222+00:00 [info] <0.273.0> ra: meta data store initialised for system coordination. 0 record(s) recovered
2026-05-28 02:08:53.439495+00:00 [notice] <0.278.0> WAL: ra_coordination_log_wal init, open tbls: ra_coordination_log_open_mem_tables, closed tbls: ra_coordination_log_closed_mem_tables
2026-05-28 02:08:53.444227+00:00 [info] <0.222.0> 
2026-05-28 02:08:53.444227+00:00 [info] <0.222.0>  Starting RabbitMQ 3.10.7 on Erlang 25.0.4 [jit]
2026-05-28 02:08:53.444227+00:00 [info] <0.222.0>  Copyright (c) 2007-2022 VMware, Inc. or its affiliates.
2026-05-28 02:08:53.444227+00:00 [info] <0.222.0>  Licensed under the MPL 2.0. Website: https://rabbitmq.com
2026-05-28 02:08:53.445366+00:00 [info] <0.222.0> 
2026-05-28 02:08:53.445366+00:00 [info] <0.222.0>  node           : contrail@cn-jenkins-deploy-platform-ansible-os-5841-3
2026-05-28 02:08:53.445366+00:00 [info] <0.222.0>  home dir       : /var/lib/rabbitmq
2026-05-28 02:08:53.445366+00:00 [info] <0.222.0>  config file(s) : /etc/rabbitmq/rabbitmq.config
2026-05-28 02:08:53.445366+00:00 [info] <0.222.0>                 : /etc/rabbitmq/conf.d/10-defaults.conf
2026-05-28 02:08:53.445366+00:00 [info] <0.222.0>  cookie hash    : esZsDxSN6VGbi9JkMSxNZA==
2026-05-28 02:08:53.445366+00:00 [info] <0.222.0>  log(s)         : /var/log/contrail/config-database-rabbitmq/rabbitmq.log
2026-05-28 02:08:53.445366+00:00 [info] <0.222.0>                 : /var/log/rabbitmq/contrail@cn-jenkins-deploy-platform-ansible-os-5841-3_upgrade.log
2026-05-28 02:08:53.445366+00:00 [info] <0.222.0>                 : <stdout>
2026-05-28 02:08:53.445366+00:00 [info] <0.222.0>  database dir   : /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-5841-3
2026-05-28 02:08:53.633519+00:00 [info] <0.222.0> Feature flags: list of feature flags found:
2026-05-28 02:08:53.633599+00:00 [info] <0.222.0> Feature flags:   [x] classic_mirrored_queue_version
2026-05-28 02:08:53.633623+00:00 [info] <0.222.0> Feature flags:   [x] drop_unroutable_metric
2026-05-28 02:08:53.633651+00:00 [info] <0.222.0> Feature flags:   [x] empty_basic_get_metric
2026-05-28 02:08:53.633708+00:00 [info] <0.222.0> Feature flags:   [x] implicit_default_bindings
2026-05-28 02:08:53.633732+00:00 [info] <0.222.0> Feature flags:   [x] maintenance_mode_status
2026-05-28 02:08:53.633750+00:00 [info] <0.222.0> Feature flags:   [x] quorum_queue
2026-05-28 02:08:53.633801+00:00 [info] <0.222.0> Feature flags:   [x] stream_queue
2026-05-28 02:08:53.633818+00:00 [info] <0.222.0> Feature flags:   [x] user_limits
2026-05-28 02:08:53.633833+00:00 [info] <0.222.0> Feature flags:   [x] virtual_host_metadata
2026-05-28 02:08:53.633853+00:00 [info] <0.222.0> Feature flags: feature flag states written to disk: yes
2026-05-28 02:08:55.409916+00:00 [info] <0.222.0> Running boot step pre_boot defined by app rabbit
2026-05-28 02:08:55.409995+00:00 [info] <0.222.0> Running boot step rabbit_global_counters defined by app rabbit
2026-05-28 02:08:55.410249+00:00 [info] <0.222.0> Running boot step rabbit_osiris_metrics defined by app rabbit
2026-05-28 02:08:55.410329+00:00 [info] <0.222.0> Running boot step rabbit_core_metrics defined by app rabbit
2026-05-28 02:08:55.410826+00:00 [info] <0.222.0> Running boot step rabbit_alarm defined by app rabbit
2026-05-28 02:08:55.414435+00:00 [info] <0.290.0> Memory high watermark set to 12637 MiB (13251084288 bytes) of 15796 MiB (16563855360 bytes) total
2026-05-28 02:08:55.417862+00:00 [info] <0.292.0> Enabling free disk space monitoring
2026-05-28 02:08:55.417924+00:00 [info] <0.292.0> Disk free limit set to 50MB
2026-05-28 02:08:55.420715+00:00 [info] <0.222.0> Running boot step code_server_cache defined by app rabbit
2026-05-28 02:08:55.420830+00:00 [info] <0.222.0> Running boot step file_handle_cache defined by app rabbit
2026-05-28 02:08:55.421075+00:00 [info] <0.297.0> Limiting to approx 927 file handles (832 sockets)
2026-05-28 02:08:55.421172+00:00 [info] <0.298.0> FHC read buffering: OFF
2026-05-28 02:08:55.421213+00:00 [info] <0.298.0> FHC write buffering: ON
2026-05-28 02:08:55.421718+00:00 [info] <0.222.0> Running boot step worker_pool defined by app rabbit
2026-05-28 02:08:55.421780+00:00 [info] <0.280.0> Will use 4 processes for default worker pool
2026-05-28 02:08:55.421815+00:00 [info] <0.280.0> Starting worker pool 'worker_pool' with 4 processes in it
2026-05-28 02:08:55.422106+00:00 [info] <0.222.0> Running boot step database defined by app rabbit
2026-05-28 02:08:55.422387+00:00 [info] <0.222.0> Node database directory at /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-5841-3 is empty. Assuming we need to join an existing cluster or initialise from scratch...
2026-05-28 02:08:55.422441+00:00 [info] <0.222.0> Configured peer discovery backend: rabbit_peer_discovery_classic_config
2026-05-28 02:08:55.422467+00:00 [info] <0.222.0> Will try to lock with peer discovery backend rabbit_peer_discovery_classic_config
2026-05-28 02:08:55.426959+00:00 [info] <0.222.0> All discovered existing cluster peers: contrail@cn-jenkins-deploy-platform-ansible-os-5841-1, contrail@cn-jenkins-deploy-platform-ansible-os-5841-2, contrail@cn-jenkins-deploy-platform-ansible-os-5841-3
2026-05-28 02:08:55.427017+00:00 [info] <0.222.0> Peer nodes we can cluster with: contrail@cn-jenkins-deploy-platform-ansible-os-5841-1, contrail@cn-jenkins-deploy-platform-ansible-os-5841-2
2026-05-28 02:08:55.432437+00:00 [info] <0.222.0> Node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-1' selected for auto-clustering
2026-05-28 02:08:55.483771+00:00 [info] <0.222.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
2026-05-28 02:08:55.725353+00:00 [info] <0.222.0> Successfully synced tables from a peer
2026-05-28 02:08:55.726029+00:00 [info] <0.222.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
2026-05-28 02:08:55.726129+00:00 [info] <0.222.0> Successfully synced tables from a peer
2026-05-28 02:08:55.726172+00:00 [warning] <0.222.0> Feature flags: the previous instance of this node must have failed to write the `feature_flags` file at `/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-5841-3-feature_flags`:
2026-05-28 02:08:55.726217+00:00 [warning] <0.222.0> Feature flags:   - list of previously disabled feature flags now marked as such: [maintenance_mode_status]
2026-05-28 02:08:55.728998+00:00 [info] <0.222.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
2026-05-28 02:08:55.729099+00:00 [info] <0.222.0> Successfully synced tables from a peer
2026-05-28 02:08:55.745349+00:00 [info] <0.222.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
2026-05-28 02:08:55.745592+00:00 [info] <0.222.0> Successfully synced tables from a peer
2026-05-28 02:08:55.811879+00:00 [info] <0.222.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-5841-3'
2026-05-28 02:08:55.812828+00:00 [info] <0.222.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-5841-3'
2026-05-28 02:08:55.813812+00:00 [info] <0.222.0> Setting up a table for per-user connection counting on this node: 'tracked_connection_table_per_user_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-5841-3'
2026-05-28 02:08:55.814773+00:00 [info] <0.222.0> Peer discovery backend rabbit_peer_discovery_classic_config does not support registration, skipping registration.
2026-05-28 02:08:55.814818+00:00 [info] <0.222.0> Will try to unlock with peer discovery backend rabbit_peer_discovery_classic_config
2026-05-28 02:08:55.817370+00:00 [info] <0.222.0> Running boot step database_sync defined by app rabbit
2026-05-28 02:08:55.817496+00:00 [info] <0.222.0> Running boot step feature_flags defined by app rabbit
2026-05-28 02:08:55.817622+00:00 [info] <0.222.0> Running boot step codec_correctness_check defined by app rabbit
2026-05-28 02:08:55.817683+00:00 [info] <0.222.0> Running boot step external_infrastructure defined by app rabbit
2026-05-28 02:08:55.817714+00:00 [info] <0.222.0> Running boot step rabbit_event defined by app rabbit
2026-05-28 02:08:55.817838+00:00 [info] <0.222.0> Running boot step rabbit_registry defined by app rabbit
2026-05-28 02:08:55.817928+00:00 [info] <0.222.0> Running boot step rabbit_auth_mechanism_amqplain defined by app rabbit
2026-05-28 02:08:55.817984+00:00 [info] <0.222.0> Running boot step rabbit_auth_mechanism_cr_demo defined by app rabbit
2026-05-28 02:08:55.818025+00:00 [info] <0.222.0> Running boot step rabbit_auth_mechanism_plain defined by app rabbit
2026-05-28 02:08:55.818056+00:00 [info] <0.222.0> Running boot step rabbit_exchange_type_direct defined by app rabbit
2026-05-28 02:08:55.818089+00:00 [info] <0.222.0> Running boot step rabbit_exchange_type_fanout defined by app rabbit
2026-05-28 02:08:55.818125+00:00 [info] <0.222.0> Running boot step rabbit_exchange_type_headers defined by app rabbit
2026-05-28 02:08:55.818175+00:00 [info] <0.222.0> Running boot step rabbit_exchange_type_topic defined by app rabbit
2026-05-28 02:08:55.818214+00:00 [info] <0.222.0> Running boot step rabbit_mirror_queue_mode_all defined by app rabbit
2026-05-28 02:08:55.818251+00:00 [info] <0.222.0> Running boot step rabbit_mirror_queue_mode_exactly defined by app rabbit
2026-05-28 02:08:55.818292+00:00 [info] <0.222.0> Running boot step rabbit_mirror_queue_mode_nodes defined by app rabbit
2026-05-28 02:08:55.818328+00:00 [info] <0.222.0> Running boot step rabbit_priority_queue defined by app rabbit
2026-05-28 02:08:55.818358+00:00 [info] <0.222.0> Priority queues enabled, real BQ is rabbit_variable_queue
2026-05-28 02:08:55.818406+00:00 [info] <0.222.0> Running boot step rabbit_queue_location_client_local defined by app rabbit
2026-05-28 02:08:55.818460+00:00 [info] <0.222.0> Running boot step rabbit_queue_location_min_masters defined by app rabbit
2026-05-28 02:08:55.818536+00:00 [info] <0.222.0> Running boot step rabbit_queue_location_random defined by app rabbit
2026-05-28 02:08:55.818623+00:00 [info] <0.222.0> Running boot step kernel_ready defined by app rabbit
2026-05-28 02:08:55.819670+00:00 [info] <0.222.0> Running boot step rabbit_sysmon_minder defined by app rabbit
2026-05-28 02:08:55.819880+00:00 [info] <0.222.0> Running boot step rabbit_epmd_monitor defined by app rabbit
2026-05-28 02:08:55.822713+00:00 [info] <0.514.0> epmd monitor knows us, inter-node communication (distribution) port: 25673
2026-05-28 02:08:55.822814+00:00 [info] <0.222.0> Running boot step guid_generator defined by app rabbit
2026-05-28 02:08:55.835225+00:00 [info] <0.222.0> Running boot step rabbit_node_monitor defined by app rabbit
2026-05-28 02:08:55.836820+00:00 [info] <0.520.0> Starting rabbit_node_monitor
2026-05-28 02:08:55.837110+00:00 [info] <0.222.0> Running boot step delegate_sup defined by app rabbit
2026-05-28 02:08:55.837778+00:00 [info] <0.222.0> Running boot step rabbit_memory_monitor defined by app rabbit
2026-05-28 02:08:55.837985+00:00 [info] <0.222.0> Running boot step rabbit_fifo_dlx_sup defined by app rabbit
2026-05-28 02:08:55.838062+00:00 [info] <0.222.0> Running boot step core_initialized defined by app rabbit
2026-05-28 02:08:55.838091+00:00 [info] <0.222.0> Running boot step upgrade_queues defined by app rabbit
2026-05-28 02:08:55.846043+00:00 [info] <0.222.0> message_store upgrades: 1 to apply
2026-05-28 02:08:55.846176+00:00 [info] <0.222.0> message_store upgrades: Applying rabbit_variable_queue:move_messages_to_vhost_store
2026-05-28 02:08:55.846319+00:00 [info] <0.222.0> message_store upgrades: No durable queues found. Skipping message store migration
2026-05-28 02:08:55.846354+00:00 [info] <0.222.0> message_store upgrades: Removing the old message store data
2026-05-28 02:08:55.847028+00:00 [info] <0.222.0> message_store upgrades: All upgrades applied successfully
2026-05-28 02:08:55.858022+00:00 [info] <0.222.0> Running boot step channel_tracking defined by app rabbit
2026-05-28 02:08:55.901932+00:00 [info] <0.222.0> Setting up a table for channel tracking on this node: 'tracked_channel_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-5841-3'
2026-05-28 02:08:55.902930+00:00 [info] <0.222.0> Setting up a table for channel tracking on this node: 'tracked_channel_table_per_user_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-5841-3'
2026-05-28 02:08:55.903049+00:00 [info] <0.222.0> Running boot step rabbit_channel_tracking_handler defined by app rabbit
2026-05-28 02:08:55.903192+00:00 [info] <0.222.0> Running boot step connection_tracking defined by app rabbit
2026-05-28 02:08:55.904005+00:00 [info] <0.222.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-5841-3'
2026-05-28 02:08:55.965739+00:00 [info] <0.222.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-5841-3'
2026-05-28 02:08:55.967025+00:00 [info] <0.222.0> Setting up a table for per-user connection counting on this node: 'tracked_connection_table_per_user_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-5841-3'
2026-05-28 02:08:55.967224+00:00 [info] <0.222.0> Running boot step rabbit_connection_tracking_handler defined by app rabbit
2026-05-28 02:08:55.967278+00:00 [info] <0.222.0> Running boot step rabbit_definitions_hashing defined by app rabbit
2026-05-28 02:08:55.967327+00:00 [info] <0.222.0> Running boot step rabbit_exchange_parameters defined by app rabbit
2026-05-28 02:08:55.967484+00:00 [info] <0.222.0> Running boot step rabbit_mirror_queue_misc defined by app rabbit
2026-05-28 02:08:55.967682+00:00 [info] <0.222.0> Running boot step rabbit_policies defined by app rabbit
2026-05-28 02:08:55.967953+00:00 [info] <0.222.0> Running boot step rabbit_policy defined by app rabbit
2026-05-28 02:08:55.968030+00:00 [info] <0.222.0> Running boot step rabbit_queue_location_validator defined by app rabbit
2026-05-28 02:08:55.968117+00:00 [info] <0.222.0> Running boot step rabbit_quorum_memory_manager defined by app rabbit
2026-05-28 02:08:55.968214+00:00 [info] <0.222.0> Running boot step rabbit_stream_coordinator defined by app rabbit
2026-05-28 02:08:55.968338+00:00 [info] <0.222.0> Running boot step rabbit_vhost_limit defined by app rabbit
2026-05-28 02:08:55.968430+00:00 [info] <0.222.0> Running boot step rabbit_mgmt_reset_handler defined by app rabbitmq_management
2026-05-28 02:08:55.968477+00:00 [info] <0.222.0> Running boot step rabbit_mgmt_db_handler defined by app rabbitmq_management_agent
2026-05-28 02:08:55.968538+00:00 [warning] <0.222.0> force_fine_statistics set to true; ignored.
2026-05-28 02:08:55.968538+00:00 [warning] <0.222.0> Replaced by {rates_mode, none} in the rabbitmq_management application.
2026-05-28 02:08:55.968599+00:00 [info] <0.222.0> Management plugin: using rates mode 'basic'
2026-05-28 02:08:55.970678+00:00 [info] <0.222.0> Running boot step recovery defined by app rabbit
2026-05-28 02:08:55.973075+00:00 [info] <0.594.0> Making sure data directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-5841-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists
2026-05-28 02:08:55.974136+00:00 [info] <0.594.0> Setting segment_entry_count for vhost '/' with 0 queues to '2048'
2026-05-28 02:08:55.978922+00:00 [info] <0.594.0> Starting message stores for vhost '/'
2026-05-28 02:08:55.979260+00:00 [info] <0.602.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index
2026-05-28 02:08:55.980351+00:00 [info] <0.594.0> Started message store of type transient for vhost '/'
2026-05-28 02:08:55.980527+00:00 [info] <0.606.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index
2026-05-28 02:08:55.981047+00:00 [warning] <0.606.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": rebuilding indices from scratch
2026-05-28 02:08:55.981657+00:00 [info] <0.594.0> Started message store of type persistent for vhost '/'
2026-05-28 02:08:55.981770+00:00 [info] <0.594.0> Recovering 0 queues of type rabbit_classic_queue took 7ms
2026-05-28 02:08:55.981810+00:00 [info] <0.594.0> Recovering 0 queues of type rabbit_quorum_queue took 0ms
2026-05-28 02:08:55.981839+00:00 [info] <0.594.0> Recovering 0 queues of type rabbit_stream_queue took 0ms
2026-05-28 02:08:55.983745+00:00 [info] <0.594.0> Mirrored queue 'job_status_consumer.dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-3': <0.629.0>
2026-05-28 02:08:55.984127+00:00 [info] <0.594.0> Mirrored queue 'job_request_consumer' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-3': <0.633.0>
2026-05-28 02:08:55.984442+00:00 [info] <0.594.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-5841-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-3': <0.637.0>
2026-05-28 02:08:55.984612+00:00 [info] <0.222.0> Running boot step empty_db_check defined by app rabbit
2026-05-28 02:08:55.985056+00:00 [info] <0.222.0> Will not seed default virtual host and user: have definitions to load...
2026-05-28 02:08:55.985084+00:00 [info] <0.222.0> Running boot step rabbit_looking_glass defined by app rabbit
2026-05-28 02:08:55.985106+00:00 [info] <0.222.0> Running boot step rabbit_core_metrics_gc defined by app rabbit
2026-05-28 02:08:55.985252+00:00 [info] <0.222.0> Running boot step background_gc defined by app rabbit
2026-05-28 02:08:55.985432+00:00 [info] <0.222.0> Running boot step routing_ready defined by app rabbit
2026-05-28 02:08:55.985484+00:00 [info] <0.222.0> Running boot step pre_flight defined by app rabbit
2026-05-28 02:08:55.985505+00:00 [info] <0.222.0> Running boot step notify_cluster defined by app rabbit
2026-05-28 02:08:55.985528+00:00 [info] <0.222.0> Running boot step networking defined by app rabbit
2026-05-28 02:08:55.985551+00:00 [info] <0.222.0> Running boot step definition_import_worker_pool defined by app rabbit
2026-05-28 02:08:55.985628+00:00 [info] <0.280.0> Starting worker pool 'definition_import_pool' with 4 processes in it
2026-05-28 02:08:55.985974+00:00 [info] <0.222.0> Running boot step cluster_name defined by app rabbit
2026-05-28 02:08:55.986018+00:00 [info] <0.222.0> Running boot step direct_client defined by app rabbit
2026-05-28 02:08:55.986122+00:00 [info] <0.222.0> Running boot step rabbit_maintenance_mode_state defined by app rabbit
2026-05-28 02:08:55.986163+00:00 [info] <0.222.0> Creating table rabbit_node_maintenance_states for feature flag `maintenance_mode_status`
2026-05-28 02:08:55.986235+00:00 [info] <0.520.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-2' up
2026-05-28 02:08:55.995534+00:00 [info] <0.520.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-1' up
2026-05-28 02:08:56.006763+00:00 [info] <0.520.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-2' down
2026-05-28 02:08:56.015934+00:00 [info] <0.520.0> Keeping contrail@cn-jenkins-deploy-platform-ansible-os-5841-2 listeners: the node is already back
2026-05-28 02:08:56.040663+00:00 [info] <0.222.0> Running boot step rabbit_management_load_definitions defined by app rabbitmq_management
2026-05-28 02:08:56.041058+00:00 [info] <0.222.0> Applying definitions from file at '/etc/rabbitmq/definitions.json'
2026-05-28 02:08:56.041114+00:00 [info] <0.222.0> Asked to import definitions. Acting user: rmq-internal
2026-05-28 02:08:56.041208+00:00 [info] <0.222.0> Importing concurrently 1 users...
2026-05-28 02:08:56.055101+00:00 [info] <0.222.0> Importing concurrently 1 vhosts...
2026-05-28 02:08:56.062100+00:00 [info] <0.222.0> Importing concurrently 1 permissions...
2026-05-28 02:08:56.066629+00:00 [info] <0.647.0> Successfully set permissions for 'guest' in virtual host '/' to '.*', '.*', '.*'
2026-05-28 02:08:56.066820+00:00 [info] <0.222.0> Importing sequentially 1 policies...
2026-05-28 02:08:56.071318+00:00 [info] <0.682.0> Resetting node maintenance status
2026-05-28 02:08:56.083597+00:00 [info] <0.741.0> Management plugin: HTTP (non-TLS) listener started on port 15673
2026-05-28 02:08:56.083731+00:00 [info] <0.769.0> Statistics database started.
2026-05-28 02:08:56.083794+00:00 [info] <0.768.0> Starting worker pool 'management_worker_pool' with 3 processes in it
2026-05-28 02:08:56.091861+00:00 [info] <0.783.0> Prometheus metrics: HTTP (non-TLS) listener started on port 15692
2026-05-28 02:08:56.091993+00:00 [info] <0.682.0> Ready to start client connection listeners
2026-05-28 02:08:56.093296+00:00 [info] <0.827.0> started TCP listener on 10.0.0.131:5673
2026-05-28 02:08:56.161586+00:00 [info] <0.682.0> Server startup complete; 4 plugins started.
2026-05-28 02:08:56.161586+00:00 [info] <0.682.0>  * rabbitmq_prometheus
2026-05-28 02:08:56.161586+00:00 [info] <0.682.0>  * rabbitmq_management
2026-05-28 02:08:56.161586+00:00 [info] <0.682.0>  * rabbitmq_web_dispatch
2026-05-28 02:08:56.161586+00:00 [info] <0.682.0>  * rabbitmq_management_agent
2026-05-28 02:08:56.612351+00:00 [info] <0.520.0> node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-2' down: connection_closed
2026-05-28 02:09:00.085744+00:00 [info] <0.520.0> node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-2' up
2026-05-28 02:09:03.770191+00:00 [info] <0.520.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-2' up
2026-05-28 02:09:42.997623+00:00 [info] <0.1239.0> RabbitMQ is asked to stop...
2026-05-28 02:09:43.031106+00:00 [info] <0.1239.0> Stopping RabbitMQ applications and their dependencies in the following order:
2026-05-28 02:09:43.031106+00:00 [info] <0.1239.0>     rabbitmq_management
2026-05-28 02:09:43.031106+00:00 [info] <0.1239.0>     rabbitmq_prometheus
2026-05-28 02:09:43.031106+00:00 [info] <0.1239.0>     rabbitmq_web_dispatch
2026-05-28 02:09:43.031106+00:00 [info] <0.1239.0>     cowboy
2026-05-28 02:09:43.031106+00:00 [info] <0.1239.0>     cowlib
2026-05-28 02:09:43.031106+00:00 [info] <0.1239.0>     rabbitmq_management_agent
2026-05-28 02:09:43.031106+00:00 [info] <0.1239.0>     rabbit
2026-05-28 02:09:43.031106+00:00 [info] <0.1239.0>     amqp_client
2026-05-28 02:09:43.031106+00:00 [info] <0.1239.0>     rabbitmq_prelaunch
2026-05-28 02:09:43.031106+00:00 [info] <0.1239.0>     rabbit_common
2026-05-28 02:09:43.031106+00:00 [info] <0.1239.0>     prometheus
2026-05-28 02:09:43.031106+00:00 [info] <0.1239.0>     ra
2026-05-28 02:09:43.031106+00:00 [info] <0.1239.0>     accept
2026-05-28 02:09:43.031106+00:00 [info] <0.1239.0>     os_mon
2026-05-28 02:09:43.031106+00:00 [info] <0.1239.0>     sysmon_handler
2026-05-28 02:09:43.031106+00:00 [info] <0.1239.0>     osiris
2026-05-28 02:09:43.031106+00:00 [info] <0.1239.0>     mnesia
2026-05-28 02:09:43.031106+00:00 [info] <0.1239.0> 
2026-05-28 02:09:43.031252+00:00 [info] <0.1239.0> Stopping application 'rabbitmq_management'
2026-05-28 02:09:43.033018+00:00 [warning] <0.733.0> HTTP listener registry could not find context rabbitmq_management_tls
2026-05-28 02:09:43.034691+00:00 [notice] <0.44.0> Application rabbitmq_management exited with reason: stopped
2026-05-28 02:09:43.034787+00:00 [info] <0.1239.0> Stopping application 'rabbitmq_prometheus'
2026-05-28 02:09:43.035988+00:00 [warning] <0.733.0> HTTP listener registry could not find context rabbitmq_prometheus_tls
2026-05-28 02:09:43.037830+00:00 [notice] <0.44.0> Application rabbitmq_prometheus exited with reason: stopped
2026-05-28 02:09:43.037895+00:00 [info] <0.1239.0> Stopping application 'rabbitmq_web_dispatch'
2026-05-28 02:09:43.039551+00:00 [notice] <0.44.0> Application rabbitmq_web_dispatch exited with reason: stopped
2026-05-28 02:09:43.039607+00:00 [info] <0.1239.0> Stopping application 'cowboy'
2026-05-28 02:09:43.041136+00:00 [notice] <0.44.0> Application cowboy exited with reason: stopped
2026-05-28 02:09:43.041208+00:00 [info] <0.1239.0> Stopping application 'cowlib'
2026-05-28 02:09:43.041249+00:00 [notice] <0.44.0> Application cowlib exited with reason: stopped
2026-05-28 02:09:43.041337+00:00 [info] <0.1239.0> Stopping application 'rabbitmq_management_agent'
2026-05-28 02:09:43.043400+00:00 [notice] <0.44.0> Application rabbitmq_management_agent exited with reason: stopped
2026-05-28 02:09:43.043462+00:00 [info] <0.1239.0> Stopping application 'rabbit'
2026-05-28 02:09:43.043594+00:00 [info] <0.222.0> Peer discovery backend rabbit_peer_discovery_classic_config does not support registration, skipping unregistration.
2026-05-28 02:09:43.043700+00:00 [info] <0.827.0> stopped TCP listener on 10.0.0.131:5673
2026-05-28 02:09:43.044502+00:00 [info] <0.1240.0> Closing all connections in vhost '/' on node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-3' because the vhost is stopping
2026-05-28 02:09:43.045662+00:00 [info] <0.606.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-5841-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent'
2026-05-28 02:09:43.051570+00:00 [info] <0.606.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-5841-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' is stopped
2026-05-28 02:09:43.051706+00:00 [info] <0.602.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-5841-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient'
2026-05-28 02:09:43.055876+00:00 [info] <0.602.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-5841-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' is stopped
2026-05-28 02:09:43.060379+00:00 [info] <0.509.0> Management plugin: to stop collect_statistics.
2026-05-28 02:09:48.064450+00:00 [notice] <0.44.0> Application rabbit exited with reason: stopped
2026-05-28 02:09:48.064570+00:00 [info] <0.1239.0> Stopping application 'amqp_client'
2026-05-28 02:09:48.066699+00:00 [notice] <0.44.0> Application amqp_client exited with reason: stopped
2026-05-28 02:09:48.066843+00:00 [info] <0.1239.0> Stopping application 'rabbitmq_prelaunch'
2026-05-28 02:09:48.068572+00:00 [notice] <0.44.0> Application rabbitmq_prelaunch exited with reason: stopped
2026-05-28 02:09:48.068667+00:00 [info] <0.1239.0> Stopping application 'rabbit_common'
2026-05-28 02:09:48.068788+00:00 [notice] <0.44.0> Application rabbit_common exited with reason: stopped
2026-05-28 02:09:48.068895+00:00 [info] <0.1239.0> Stopping application 'prometheus'
2026-05-28 02:09:48.070624+00:00 [notice] <0.44.0> Application prometheus exited with reason: stopped
2026-05-28 02:09:48.070710+00:00 [info] <0.1239.0> Stopping application 'ra'
2026-05-28 02:09:48.073695+00:00 [notice] <0.44.0> Application ra exited with reason: stopped
2026-05-28 02:09:48.073767+00:00 [info] <0.1239.0> Stopping application 'accept'
2026-05-28 02:09:48.073833+00:00 [notice] <0.44.0> Application accept exited with reason: stopped
2026-05-28 02:09:48.073944+00:00 [info] <0.1239.0> Stopping application 'os_mon'
2026-05-28 02:09:48.075455+00:00 [notice] <0.44.0> Application os_mon exited with reason: stopped
2026-05-28 02:09:48.075519+00:00 [info] <0.1239.0> Stopping application 'sysmon_handler'
2026-05-28 02:09:48.076895+00:00 [notice] <0.44.0> Application sysmon_handler exited with reason: stopped
2026-05-28 02:09:48.076990+00:00 [info] <0.1239.0> Stopping application 'osiris'
2026-05-28 02:09:48.078346+00:00 [notice] <0.44.0> Application osiris exited with reason: stopped
2026-05-28 02:09:48.078399+00:00 [info] <0.1239.0> Stopping application 'mnesia'
2026-05-28 02:09:48.081672+00:00 [notice] <0.44.0> Application mnesia exited with reason: stopped
2026-05-28 02:09:48.081741+00:00 [info] <0.1239.0> Successfully stopped RabbitMQ and its dependencies
2026-05-28 02:09:57.587491+00:00 [info] <0.1262.0> Resetting Rabbit forcefully
2026-05-28 02:09:58.079430+00:00 [notice] <0.44.0> Application mnesia exited with reason: stopped
2026-05-28 02:09:58.091178+00:00 [notice] <0.44.0> Application mnesia exited with reason: stopped
2026-05-28 02:09:58.217079+00:00 [info] <0.1268.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
2026-05-28 02:09:58.217212+00:00 [info] <0.1268.0> Successfully synced tables from a peer
2026-05-28 02:09:58.227584+00:00 [info] <0.1268.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
2026-05-28 02:09:58.227694+00:00 [info] <0.1268.0> Successfully synced tables from a peer
2026-05-28 02:09:58.227756+00:00 [info] <0.1268.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
2026-05-28 02:09:58.227807+00:00 [info] <0.1268.0> Successfully synced tables from a peer
2026-05-28 02:09:58.241287+00:00 [info] <0.1268.0> Waiting for Mnesia tables for 30000 ms, 0 retries left
2026-05-28 02:09:58.241442+00:00 [info] <0.1268.0> Successfully synced tables from a peer
2026-05-28 02:09:58.246895+00:00 [notice] <0.44.0> Application mnesia exited with reason: stopped
2026-05-28 02:09:58.258156+00:00 [info] <0.1268.0> Clustering with ['contrail@cn-jenkins-deploy-platform-ansible-os-5841-2',
2026-05-28 02:09:58.258156+00:00 [info] <0.1268.0>                  'contrail@cn-jenkins-deploy-platform-ansible-os-5841-1'] as disc node
2026-05-28 02:09:58.281767+00:00 [info] <0.1268.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
2026-05-28 02:09:58.281900+00:00 [info] <0.1268.0> Successfully synced tables from a peer
2026-05-28 02:09:58.623112+00:00 [info] <0.1268.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
2026-05-28 02:09:58.623320+00:00 [info] <0.1268.0> Successfully synced tables from a peer
2026-05-28 02:09:58.624746+00:00 [info] <0.1268.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
2026-05-28 02:09:58.624855+00:00 [info] <0.1268.0> Successfully synced tables from a peer
2026-05-28 02:09:58.639668+00:00 [info] <0.1268.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
2026-05-28 02:09:58.639814+00:00 [info] <0.1268.0> Successfully synced tables from a peer
2026-05-28 02:09:58.646487+00:00 [notice] <0.44.0> Application mnesia exited with reason: stopped
2026-05-28 02:09:58.646623+00:00 [error] <0.1268.0> Failed to create a tracked connection table for node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-3': {node_not_running,
2026-05-28 02:09:58.646623+00:00 [error] <0.1268.0>                                                                                                                'contrail@cn-jenkins-deploy-platform-ansible-os-5841-3'}
2026-05-28 02:09:58.646719+00:00 [info] <0.1268.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-5841-3'
2026-05-28 02:09:58.646773+00:00 [error] <0.1268.0> Failed to create a per-vhost tracked connection table for node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-3': {node_not_running,
2026-05-28 02:09:58.646773+00:00 [error] <0.1268.0>                                                                                                                          'contrail@cn-jenkins-deploy-platform-ansible-os-5841-3'}
2026-05-28 02:09:58.646864+00:00 [info] <0.1268.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-5841-3'
2026-05-28 02:09:58.646929+00:00 [error] <0.1268.0> Failed to create a per-user tracked connection table for node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-3': {node_not_running,
2026-05-28 02:09:58.646929+00:00 [error] <0.1268.0>                                                                                                                         'contrail@cn-jenkins-deploy-platform-ansible-os-5841-3'}
2026-05-28 02:09:58.646987+00:00 [info] <0.1268.0> Setting up a table for per-user connection counting on this node: 'tracked_connection_table_per_user_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-5841-3'
2026-05-28 02:09:59.157822+00:00 [info] <0.1629.0> Halting Erlang VM with the following applications:
2026-05-28 02:09:59.157822+00:00 [info] <0.1629.0>     amqp10_common
2026-05-28 02:09:59.157822+00:00 [info] <0.1629.0>     observer_cli
2026-05-28 02:09:59.157822+00:00 [info] <0.1629.0>     stdout_formatter
2026-05-28 02:09:59.157822+00:00 [info] <0.1629.0>     seshat
2026-05-28 02:09:59.157822+00:00 [info] <0.1629.0>     gen_batch_server
2026-05-28 02:09:59.157822+00:00 [info] <0.1629.0>     aten
2026-05-28 02:09:59.157822+00:00 [info] <0.1629.0>     ranch
2026-05-28 02:09:59.157822+00:00 [info] <0.1629.0>     inets
2026-05-28 02:09:59.157822+00:00 [info] <0.1629.0>     systemd
2026-05-28 02:09:59.157822+00:00 [info] <0.1629.0>     enough
2026-05-28 02:09:59.157822+00:00 [info] <0.1629.0>     cuttlefish
2026-05-28 02:09:59.157822+00:00 [info] <0.1629.0>     credentials_obfuscation
2026-05-28 02:09:59.157822+00:00 [info] <0.1629.0>     recon
2026-05-28 02:09:59.157822+00:00 [info] <0.1629.0>     jsx
2026-05-28 02:09:59.157822+00:00 [info] <0.1629.0>     xmerl
2026-05-28 02:09:59.157822+00:00 [info] <0.1629.0>     tools
2026-05-28 02:09:59.157822+00:00 [info] <0.1629.0>     syntax_tools
2026-05-28 02:09:59.157822+00:00 [info] <0.1629.0>     ssl
2026-05-28 02:09:59.157822+00:00 [info] <0.1629.0>     public_key
2026-05-28 02:09:59.157822+00:00 [info] <0.1629.0>     asn1
2026-05-28 02:09:59.157822+00:00 [info] <0.1629.0>     crypto
2026-05-28 02:09:59.157822+00:00 [info] <0.1629.0>     compiler
2026-05-28 02:09:59.157822+00:00 [info] <0.1629.0>     sasl
2026-05-28 02:09:59.157822+00:00 [info] <0.1629.0>     stdlib
2026-05-28 02:09:59.157822+00:00 [info] <0.1629.0>     kernel
2026-05-28 02:09:59.157822+00:00 [info] <0.1629.0> 
2026-05-28 02:10:03.420150+00:00 [notice] <0.222.0> Logging: configured log handlers are now ACTIVE
2026-05-28 02:10:05.177797+00:00 [info] <0.222.0> upgrades: Backing up mnesia dir to "/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-5841-3-upgrade-backup"
2026-05-28 02:10:05.184190+00:00 [info] <0.222.0> upgrades: Mnesia dir backed up to "/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-5841-3-upgrade-backup"
2026-05-28 02:10:05.192894+00:00 [notice] <0.44.0> Application mnesia exited with reason: stopped
2026-05-28 02:10:05.194305+00:00 [info] <0.222.0> Feature flags: list of feature flags found:
2026-05-28 02:10:05.194354+00:00 [info] <0.222.0> Feature flags:   [x] classic_mirrored_queue_version
2026-05-28 02:10:05.194389+00:00 [info] <0.222.0> Feature flags:   [x] drop_unroutable_metric
2026-05-28 02:10:05.194412+00:00 [info] <0.222.0> Feature flags:   [x] empty_basic_get_metric
2026-05-28 02:10:05.194428+00:00 [info] <0.222.0> Feature flags:   [x] implicit_default_bindings
2026-05-28 02:10:05.194471+00:00 [info] <0.222.0> Feature flags:   [ ] maintenance_mode_status
2026-05-28 02:10:05.194495+00:00 [info] <0.222.0> Feature flags:   [x] quorum_queue
2026-05-28 02:10:05.194510+00:00 [info] <0.222.0> Feature flags:   [x] stream_queue
2026-05-28 02:10:05.194524+00:00 [info] <0.222.0> Feature flags:   [x] user_limits
2026-05-28 02:10:05.194555+00:00 [info] <0.222.0> Feature flags:   [x] virtual_host_metadata
2026-05-28 02:10:05.194571+00:00 [info] <0.222.0> Feature flags: feature flag states written to disk: yes
2026-05-28 02:10:05.254131+00:00 [info] <0.222.0> ra: starting system quorum_queues
2026-05-28 02:10:05.254231+00:00 [info] <0.222.0> starting Ra system: quorum_queues in directory: /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-5841-3/quorum/contrail@cn-jenkins-deploy-platform-ansible-os-5841-3
2026-05-28 02:10:05.255975+00:00 [info] <0.313.0> ra system 'quorum_queues' running pre init for 0 registered servers
2026-05-28 02:10:05.257635+00:00 [info] <0.314.0> ra: meta data store initialised for system quorum_queues. 0 record(s) recovered
2026-05-28 02:10:05.257878+00:00 [notice] <0.319.0> WAL: ra_log_wal init, open tbls: ra_log_open_mem_tables, closed tbls: ra_log_closed_mem_tables
2026-05-28 02:10:05.263448+00:00 [info] <0.222.0> ra: starting system coordination
2026-05-28 02:10:05.263533+00:00 [info] <0.222.0> starting Ra system: coordination in directory: /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-5841-3/coordination/contrail@cn-jenkins-deploy-platform-ansible-os-5841-3
2026-05-28 02:10:05.272156+00:00 [info] <0.336.0> ra system 'coordination' running pre init for 0 registered servers
2026-05-28 02:10:05.272738+00:00 [info] <0.338.0> ra: meta data store initialised for system coordination. 0 record(s) recovered
2026-05-28 02:10:05.272922+00:00 [notice] <0.343.0> WAL: ra_coordination_log_wal init, open tbls: ra_coordination_log_open_mem_tables, closed tbls: ra_coordination_log_closed_mem_tables
2026-05-28 02:10:05.278474+00:00 [info] <0.222.0> 
2026-05-28 02:10:05.278474+00:00 [info] <0.222.0>  Starting RabbitMQ 3.10.7 on Erlang 25.0.4 [jit]
2026-05-28 02:10:05.278474+00:00 [info] <0.222.0>  Copyright (c) 2007-2022 VMware, Inc. or its affiliates.
2026-05-28 02:10:05.278474+00:00 [info] <0.222.0>  Licensed under the MPL 2.0. Website: https://rabbitmq.com
2026-05-28 02:10:05.280070+00:00 [info] <0.222.0> 
2026-05-28 02:10:05.280070+00:00 [info] <0.222.0>  node           : contrail@cn-jenkins-deploy-platform-ansible-os-5841-3
2026-05-28 02:10:05.280070+00:00 [info] <0.222.0>  home dir       : /var/lib/rabbitmq
2026-05-28 02:10:05.280070+00:00 [info] <0.222.0>  config file(s) : /etc/rabbitmq/rabbitmq.config
2026-05-28 02:10:05.280070+00:00 [info] <0.222.0>                 : /etc/rabbitmq/conf.d/10-defaults.conf
2026-05-28 02:10:05.280070+00:00 [info] <0.222.0>  cookie hash    : esZsDxSN6VGbi9JkMSxNZA==
2026-05-28 02:10:05.280070+00:00 [info] <0.222.0>  log(s)         : /var/log/contrail/config-database-rabbitmq/rabbitmq.log
2026-05-28 02:10:05.280070+00:00 [info] <0.222.0>                 : /var/log/rabbitmq/contrail@cn-jenkins-deploy-platform-ansible-os-5841-3_upgrade.log
2026-05-28 02:10:05.280070+00:00 [info] <0.222.0>                 : <stdout>
2026-05-28 02:10:05.280070+00:00 [info] <0.222.0>  database dir   : /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-5841-3
2026-05-28 02:10:05.503785+00:00 [info] <0.222.0> Feature flag `drop_unroutable_metric`: mark as enabled=true
2026-05-28 02:10:05.505421+00:00 [info] <0.222.0> Feature flag `empty_basic_get_metric`: mark as enabled=true
2026-05-28 02:10:05.693352+00:00 [info] <0.222.0> Running boot step pre_boot defined by app rabbit
2026-05-28 02:10:05.693454+00:00 [info] <0.222.0> Running boot step rabbit_global_counters defined by app rabbit
2026-05-28 02:10:05.693912+00:00 [info] <0.222.0> Running boot step rabbit_osiris_metrics defined by app rabbit
2026-05-28 02:10:05.694100+00:00 [info] <0.222.0> Running boot step rabbit_core_metrics defined by app rabbit
2026-05-28 02:10:05.694869+00:00 [info] <0.222.0> Running boot step rabbit_alarm defined by app rabbit
2026-05-28 02:10:05.699091+00:00 [info] <0.418.0> Memory high watermark set to 12637 MiB (13251084288 bytes) of 15796 MiB (16563855360 bytes) total
2026-05-28 02:10:05.702762+00:00 [info] <0.420.0> Enabling free disk space monitoring
2026-05-28 02:10:05.702825+00:00 [info] <0.420.0> Disk free limit set to 50MB
2026-05-28 02:10:05.705649+00:00 [info] <0.222.0> Running boot step code_server_cache defined by app rabbit
2026-05-28 02:10:05.705796+00:00 [info] <0.222.0> Running boot step file_handle_cache defined by app rabbit
2026-05-28 02:10:05.705999+00:00 [info] <0.425.0> Limiting to approx 927 file handles (832 sockets)
2026-05-28 02:10:05.706120+00:00 [info] <0.426.0> FHC read buffering: OFF
2026-05-28 02:10:05.706159+00:00 [info] <0.426.0> FHC write buffering: ON
2026-05-28 02:10:05.706594+00:00 [info] <0.222.0> Running boot step worker_pool defined by app rabbit
2026-05-28 02:10:05.706760+00:00 [info] <0.349.0> Will use 4 processes for default worker pool
2026-05-28 02:10:05.706900+00:00 [info] <0.349.0> Starting worker pool 'worker_pool' with 4 processes in it
2026-05-28 02:10:05.707157+00:00 [info] <0.222.0> Running boot step database defined by app rabbit
2026-05-28 02:10:05.710615+00:00 [info] <0.222.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
2026-05-28 02:10:05.713851+00:00 [info] <0.222.0> Successfully synced tables from a peer
2026-05-28 02:10:05.714387+00:00 [info] <0.222.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
2026-05-28 02:10:05.714496+00:00 [info] <0.222.0> Successfully synced tables from a peer
2026-05-28 02:10:05.714529+00:00 [warning] <0.222.0> Feature flags: the previous instance of this node must have failed to write the `feature_flags` file at `/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-5841-3-feature_flags`:
2026-05-28 02:10:05.714583+00:00 [warning] <0.222.0> Feature flags:   - list of previously disabled feature flags now marked as such: [empty_basic_get_metric]
2026-05-28 02:10:05.717146+00:00 [info] <0.222.0> Creating table rabbit_node_maintenance_states for feature flag `maintenance_mode_status`
2026-05-28 02:10:05.730281+00:00 [info] <0.222.0> Feature flag `maintenance_mode_status`: mark as enabled=true
2026-05-28 02:10:05.732074+00:00 [info] <0.222.0> Feature flags: list of feature flags found:
2026-05-28 02:10:05.732147+00:00 [info] <0.222.0> Feature flags:   [x] classic_mirrored_queue_version
2026-05-28 02:10:05.732181+00:00 [info] <0.222.0> Feature flags:   [x] drop_unroutable_metric
2026-05-28 02:10:05.732254+00:00 [info] <0.222.0> Feature flags:   [x] empty_basic_get_metric
2026-05-28 02:10:05.732298+00:00 [info] <0.222.0> Feature flags:   [x] implicit_default_bindings
2026-05-28 02:10:05.732325+00:00 [info] <0.222.0> Feature flags:   [x] maintenance_mode_status
2026-05-28 02:10:05.732354+00:00 [info] <0.222.0> Feature flags:   [x] quorum_queue
2026-05-28 02:10:05.732415+00:00 [info] <0.222.0> Feature flags:   [x] stream_queue
2026-05-28 02:10:05.732442+00:00 [info] <0.222.0> Feature flags:   [x] user_limits
2026-05-28 02:10:05.732466+00:00 [info] <0.222.0> Feature flags:   [x] virtual_host_metadata
2026-05-28 02:10:05.732496+00:00 [info] <0.222.0> Feature flags: feature flag states written to disk: yes
2026-05-28 02:10:05.741970+00:00 [info] <0.222.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
2026-05-28 02:10:05.742087+00:00 [info] <0.222.0> Successfully synced tables from a peer
2026-05-28 02:10:05.761973+00:00 [info] <0.222.0> upgrades: Mnesia backup removed
2026-05-28 02:10:05.762054+00:00 [info] <0.222.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
2026-05-28 02:10:05.762176+00:00 [info] <0.222.0> Successfully synced tables from a peer
2026-05-28 02:10:05.762245+00:00 [info] <0.222.0> Peer discovery backend rabbit_peer_discovery_classic_config does not support registration, skipping registration.
2026-05-28 02:10:05.762447+00:00 [info] <0.222.0> Running boot step database_sync defined by app rabbit
2026-05-28 02:10:05.762546+00:00 [info] <0.222.0> Running boot step feature_flags defined by app rabbit
2026-05-28 02:10:05.762651+00:00 [info] <0.222.0> Running boot step codec_correctness_check defined by app rabbit
2026-05-28 02:10:05.762703+00:00 [info] <0.222.0> Running boot step external_infrastructure defined by app rabbit
2026-05-28 02:10:05.762722+00:00 [info] <0.222.0> Running boot step rabbit_event defined by app rabbit
2026-05-28 02:10:05.762807+00:00 [info] <0.222.0> Running boot step rabbit_registry defined by app rabbit
2026-05-28 02:10:05.762866+00:00 [info] <0.222.0> Running boot step rabbit_auth_mechanism_amqplain defined by app rabbit
2026-05-28 02:10:05.762921+00:00 [info] <0.222.0> Running boot step rabbit_auth_mechanism_cr_demo defined by app rabbit
2026-05-28 02:10:05.763002+00:00 [info] <0.222.0> Running boot step rabbit_auth_mechanism_plain defined by app rabbit
2026-05-28 02:10:05.763094+00:00 [info] <0.222.0> Running boot step rabbit_exchange_type_direct defined by app rabbit
2026-05-28 02:10:05.763312+00:00 [info] <0.222.0> Running boot step rabbit_exchange_type_fanout defined by app rabbit
2026-05-28 02:10:05.763371+00:00 [info] <0.222.0> Running boot step rabbit_exchange_type_headers defined by app rabbit
2026-05-28 02:10:05.763407+00:00 [info] <0.222.0> Running boot step rabbit_exchange_type_topic defined by app rabbit
2026-05-28 02:10:05.763437+00:00 [info] <0.222.0> Running boot step rabbit_mirror_queue_mode_all defined by app rabbit
2026-05-28 02:10:05.763478+00:00 [info] <0.222.0> Running boot step rabbit_mirror_queue_mode_exactly defined by app rabbit
2026-05-28 02:10:05.763512+00:00 [info] <0.222.0> Running boot step rabbit_mirror_queue_mode_nodes defined by app rabbit
2026-05-28 02:10:05.763543+00:00 [info] <0.222.0> Running boot step rabbit_priority_queue defined by app rabbit
2026-05-28 02:10:05.763569+00:00 [info] <0.222.0> Priority queues enabled, real BQ is rabbit_variable_queue
2026-05-28 02:10:05.763607+00:00 [info] <0.222.0> Running boot step rabbit_queue_location_client_local defined by app rabbit
2026-05-28 02:10:05.763662+00:00 [info] <0.222.0> Running boot step rabbit_queue_location_min_masters defined by app rabbit
2026-05-28 02:10:05.763730+00:00 [info] <0.222.0> Running boot step rabbit_queue_location_random defined by app rabbit
2026-05-28 02:10:05.763773+00:00 [info] <0.222.0> Running boot step kernel_ready defined by app rabbit
2026-05-28 02:10:05.763813+00:00 [info] <0.222.0> Running boot step rabbit_sysmon_minder defined by app rabbit
2026-05-28 02:10:05.763922+00:00 [info] <0.222.0> Running boot step rabbit_epmd_monitor defined by app rabbit
2026-05-28 02:10:05.764955+00:00 [info] <0.457.0> epmd monitor knows us, inter-node communication (distribution) port: 25673
2026-05-28 02:10:05.765071+00:00 [info] <0.222.0> Running boot step guid_generator defined by app rabbit
2026-05-28 02:10:05.768415+00:00 [info] <0.222.0> Running boot step rabbit_node_monitor defined by app rabbit
2026-05-28 02:10:05.770182+00:00 [info] <0.461.0> Starting rabbit_node_monitor
2026-05-28 02:10:05.770282+00:00 [info] <0.222.0> Running boot step delegate_sup defined by app rabbit
2026-05-28 02:10:05.770695+00:00 [info] <0.222.0> Running boot step rabbit_memory_monitor defined by app rabbit
2026-05-28 02:10:05.770920+00:00 [info] <0.222.0> Running boot step rabbit_fifo_dlx_sup defined by app rabbit
2026-05-28 02:10:05.771000+00:00 [info] <0.222.0> Running boot step core_initialized defined by app rabbit
2026-05-28 02:10:05.771023+00:00 [info] <0.222.0> Running boot step upgrade_queues defined by app rabbit
2026-05-28 02:10:05.777335+00:00 [info] <0.222.0> message_store upgrades: 1 to apply
2026-05-28 02:10:05.777434+00:00 [info] <0.222.0> message_store upgrades: Applying rabbit_variable_queue:move_messages_to_vhost_store
2026-05-28 02:10:05.777557+00:00 [info] <0.222.0> message_store upgrades: No durable queues found. Skipping message store migration
2026-05-28 02:10:05.777591+00:00 [info] <0.222.0> message_store upgrades: Removing the old message store data
2026-05-28 02:10:05.778097+00:00 [info] <0.222.0> message_store upgrades: All upgrades applied successfully
2026-05-28 02:10:05.788331+00:00 [info] <0.222.0> Running boot step channel_tracking defined by app rabbit
2026-05-28 02:10:05.798758+00:00 [info] <0.222.0> Setting up a table for channel tracking on this node: 'tracked_channel_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-5841-3'
2026-05-28 02:10:05.807720+00:00 [info] <0.222.0> Setting up a table for channel tracking on this node: 'tracked_channel_table_per_user_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-5841-3'
2026-05-28 02:10:05.807953+00:00 [info] <0.222.0> Running boot step rabbit_channel_tracking_handler defined by app rabbit
2026-05-28 02:10:05.808049+00:00 [info] <0.222.0> Running boot step connection_tracking defined by app rabbit
2026-05-28 02:10:05.816467+00:00 [info] <0.222.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-5841-3'
2026-05-28 02:10:05.825936+00:00 [info] <0.222.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-5841-3'
2026-05-28 02:10:05.838540+00:00 [info] <0.222.0> Setting up a table for per-user connection counting on this node: 'tracked_connection_table_per_user_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-5841-3'
2026-05-28 02:10:05.839963+00:00 [info] <0.222.0> Running boot step rabbit_connection_tracking_handler defined by app rabbit
2026-05-28 02:10:05.840072+00:00 [info] <0.222.0> Running boot step rabbit_definitions_hashing defined by app rabbit
2026-05-28 02:10:05.840188+00:00 [info] <0.222.0> Running boot step rabbit_exchange_parameters defined by app rabbit
2026-05-28 02:10:05.840257+00:00 [info] <0.222.0> Running boot step rabbit_mirror_queue_misc defined by app rabbit
2026-05-28 02:10:05.841172+00:00 [info] <0.222.0> Running boot step rabbit_policies defined by app rabbit
2026-05-28 02:10:05.841410+00:00 [info] <0.222.0> Running boot step rabbit_policy defined by app rabbit
2026-05-28 02:10:05.841459+00:00 [info] <0.222.0> Running boot step rabbit_queue_location_validator defined by app rabbit
2026-05-28 02:10:05.841501+00:00 [info] <0.222.0> Running boot step rabbit_quorum_memory_manager defined by app rabbit
2026-05-28 02:10:05.841562+00:00 [info] <0.222.0> Running boot step rabbit_stream_coordinator defined by app rabbit
2026-05-28 02:10:05.841686+00:00 [info] <0.222.0> Running boot step rabbit_vhost_limit defined by app rabbit
2026-05-28 02:10:05.841749+00:00 [info] <0.222.0> Running boot step rabbit_mgmt_reset_handler defined by app rabbitmq_management
2026-05-28 02:10:05.841792+00:00 [info] <0.222.0> Running boot step rabbit_mgmt_db_handler defined by app rabbitmq_management_agent
2026-05-28 02:10:05.841836+00:00 [warning] <0.222.0> force_fine_statistics set to true; ignored.
2026-05-28 02:10:05.841836+00:00 [warning] <0.222.0> Replaced by {rates_mode, none} in the rabbitmq_management application.
2026-05-28 02:10:05.841887+00:00 [info] <0.222.0> Management plugin: using rates mode 'basic'
2026-05-28 02:10:05.844639+00:00 [info] <0.222.0> Running boot step recovery defined by app rabbit
2026-05-28 02:10:05.853823+00:00 [info] <0.519.0> Making sure data directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-5841-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists
2026-05-28 02:10:05.854609+00:00 [info] <0.519.0> Setting segment_entry_count for vhost '/' with 0 queues to '2048'
2026-05-28 02:10:05.870805+00:00 [info] <0.519.0> Starting message stores for vhost '/'
2026-05-28 02:10:05.871146+00:00 [info] <0.524.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index
2026-05-28 02:10:05.872463+00:00 [info] <0.519.0> Started message store of type transient for vhost '/'
2026-05-28 02:10:05.872708+00:00 [info] <0.528.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index
2026-05-28 02:10:05.873383+00:00 [warning] <0.528.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": rebuilding indices from scratch
2026-05-28 02:10:05.873931+00:00 [info] <0.519.0> Started message store of type persistent for vhost '/'
2026-05-28 02:10:05.874082+00:00 [info] <0.519.0> Recovering 0 queues of type rabbit_classic_queue took 19ms
2026-05-28 02:10:05.874123+00:00 [info] <0.519.0> Recovering 0 queues of type rabbit_quorum_queue took 0ms
2026-05-28 02:10:05.874157+00:00 [info] <0.519.0> Recovering 0 queues of type rabbit_stream_queue took 0ms
2026-05-28 02:10:05.875346+00:00 [info] <0.519.0> Mirrored queue 'schema_transformer.cn-jenkins-deploy-platform-ansible-os-5841-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-3': <0.548.0>
2026-05-28 02:10:05.875575+00:00 [info] <0.519.0> Mirrored queue 'job_status_consumer.dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-3': <0.552.0>
2026-05-28 02:10:05.875770+00:00 [info] <0.519.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-5841-2' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-3': <0.556.0>
2026-05-28 02:10:05.875997+00:00 [info] <0.519.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-5841-1.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-3': <0.560.0>
2026-05-28 02:10:05.876201+00:00 [info] <0.519.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-5841-2..config_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-3': <0.564.0>
2026-05-28 02:10:05.876380+00:00 [info] <0.519.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-5841-2.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-3': <0.568.0>
2026-05-28 02:10:05.876568+00:00 [info] <0.519.0> Mirrored queue 'device_manager_ztp.ztp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-3': <0.572.0>
2026-05-28 02:10:05.876791+00:00 [info] <0.519.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-5841-3' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-3': <0.576.0>
2026-05-28 02:10:05.876989+00:00 [info] <0.519.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-5841-3' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-3': <0.580.0>
2026-05-28 02:10:05.877221+00:00 [info] <0.519.0> Mirrored queue 'svc_monitor.cn-jenkins-deploy-platform-ansible-os-5841-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-3': <0.584.0>
2026-05-28 02:10:05.877495+00:00 [info] <0.519.0> Mirrored queue 'job_request_consumer' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-3': <0.588.0>
2026-05-28 02:10:05.877638+00:00 [info] <0.519.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-5841-2' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-3': <0.592.0>
2026-05-28 02:10:05.877924+00:00 [info] <0.519.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-5841-1' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-3': <0.596.0>
2026-05-28 02:10:05.878147+00:00 [info] <0.519.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-5841-3..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-3': <0.600.0>
2026-05-28 02:10:05.878524+00:00 [info] <0.519.0> Mirrored queue 'device_manager.cn-jenkins-deploy-platform-ansible-os-5841-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-3': <0.605.0>
2026-05-28 02:10:05.878907+00:00 [info] <0.519.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-5841-3.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-3': <0.610.0>
2026-05-28 02:10:05.879158+00:00 [info] <0.519.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-5841-2..tftp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-3': <0.615.0>
2026-05-28 02:10:05.879422+00:00 [info] <0.519.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-5841-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-3': <0.620.0>
2026-05-28 02:10:05.879731+00:00 [info] <0.519.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-5841-1' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-3': <0.624.0>
2026-05-28 02:10:05.879928+00:00 [info] <0.519.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-5841-1..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-3': <0.628.0>
2026-05-28 02:10:05.880092+00:00 [info] <0.519.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-5841-2..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-3': <0.632.0>
2026-05-28 02:10:05.880212+00:00 [info] <0.222.0> Running boot step empty_db_check defined by app rabbit
2026-05-28 02:10:05.880252+00:00 [info] <0.222.0> Will not seed default virtual host and user: have definitions to load...
2026-05-28 02:10:05.880276+00:00 [info] <0.222.0> Running boot step rabbit_looking_glass defined by app rabbit
2026-05-28 02:10:05.880314+00:00 [info] <0.222.0> Running boot step rabbit_core_metrics_gc defined by app rabbit
2026-05-28 02:10:05.880571+00:00 [info] <0.222.0> Running boot step background_gc defined by app rabbit
2026-05-28 02:10:05.880840+00:00 [info] <0.222.0> Running boot step routing_ready defined by app rabbit
2026-05-28 02:10:05.880882+00:00 [info] <0.222.0> Running boot step pre_flight defined by app rabbit
2026-05-28 02:10:05.880902+00:00 [info] <0.222.0> Running boot step notify_cluster defined by app rabbit
2026-05-28 02:10:05.880958+00:00 [info] <0.222.0> Running boot step networking defined by app rabbit
2026-05-28 02:10:05.880997+00:00 [info] <0.222.0> Running boot step definition_import_worker_pool defined by app rabbit
2026-05-28 02:10:05.881081+00:00 [info] <0.349.0> Starting worker pool 'definition_import_pool' with 4 processes in it
2026-05-28 02:10:05.881589+00:00 [info] <0.461.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-1' up
2026-05-28 02:10:05.881585+00:00 [info] <0.222.0> Running boot step cluster_name defined by app rabbit
2026-05-28 02:10:05.881676+00:00 [info] <0.222.0> Running boot step direct_client defined by app rabbit
2026-05-28 02:10:05.881745+00:00 [info] <0.222.0> Running boot step rabbit_maintenance_mode_state defined by app rabbit
2026-05-28 02:10:05.881801+00:00 [info] <0.222.0> Creating table rabbit_node_maintenance_states for feature flag `maintenance_mode_status`
2026-05-28 02:10:05.883529+00:00 [info] <0.222.0> Running boot step rabbit_management_load_definitions defined by app rabbitmq_management
2026-05-28 02:10:05.884028+00:00 [info] <0.222.0> Applying definitions from file at '/etc/rabbitmq/definitions.json'
2026-05-28 02:10:05.884074+00:00 [info] <0.222.0> Asked to import definitions. Acting user: rmq-internal
2026-05-28 02:10:05.884204+00:00 [info] <0.222.0> Importing concurrently 1 users...
2026-05-28 02:10:05.892510+00:00 [info] <0.222.0> Importing concurrently 1 vhosts...
2026-05-28 02:10:05.903697+00:00 [info] <0.222.0> Importing concurrently 1 permissions...
2026-05-28 02:10:05.909193+00:00 [info] <0.640.0> Successfully set permissions for 'guest' in virtual host '/' to '.*', '.*', '.*'
2026-05-28 02:10:05.909354+00:00 [info] <0.222.0> Importing sequentially 1 policies...
2026-05-28 02:10:05.924724+00:00 [info] <0.461.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-5841-2' up
2026-05-28 02:10:05.939160+00:00 [info] <0.741.0> Resetting node maintenance status
2026-05-28 02:10:05.997518+00:00 [info] <0.818.0> Management plugin: HTTP (non-TLS) listener started on port 15673
2026-05-28 02:10:05.997693+00:00 [info] <0.846.0> Statistics database started.
2026-05-28 02:10:05.997773+00:00 [info] <0.845.0> Starting worker pool 'management_worker_pool' with 3 processes in it
2026-05-28 02:10:06.005309+00:00 [info] <0.860.0> Prometheus metrics: HTTP (non-TLS) listener started on port 15692
2026-05-28 02:10:06.005434+00:00 [info] <0.741.0> Ready to start client connection listeners
2026-05-28 02:10:06.006616+00:00 [info] <0.904.0> started TCP listener on 10.0.0.131:5673
2026-05-28 02:10:06.076933+00:00 [info] <0.741.0> Server startup complete; 4 plugins started.
2026-05-28 02:10:06.076933+00:00 [info] <0.741.0>  * rabbitmq_prometheus
2026-05-28 02:10:06.076933+00:00 [info] <0.741.0>  * rabbitmq_management
2026-05-28 02:10:06.076933+00:00 [info] <0.741.0>  * rabbitmq_web_dispatch
2026-05-28 02:10:06.076933+00:00 [info] <0.741.0>  * rabbitmq_management_agent