2025-09-11 01:54:45.474163+00:00 [notice] <0.222.0> Logging: configured log handlers are now ACTIVE 2025-09-11 01:54:45.641572+00:00 [info] <0.222.0> ra: starting system quorum_queues 2025-09-11 01:54:45.641661+00:00 [info] <0.222.0> starting Ra system: quorum_queues in directory: /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4079-3/quorum/contrail@cn-jenkins-deploy-platform-ansible-os-4079-3 2025-09-11 01:54:45.688269+00:00 [info] <0.259.0> ra system 'quorum_queues' running pre init for 0 registered servers 2025-09-11 01:54:45.694685+00:00 [info] <0.260.0> ra: meta data store initialised for system quorum_queues. 0 record(s) recovered 2025-09-11 01:54:45.703011+00:00 [notice] <0.265.0> WAL: ra_log_wal init, open tbls: ra_log_open_mem_tables, closed tbls: ra_log_closed_mem_tables 2025-09-11 01:54:45.714188+00:00 [info] <0.222.0> ra: starting system coordination 2025-09-11 01:54:45.714253+00:00 [info] <0.222.0> starting Ra system: coordination in directory: /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4079-3/coordination/contrail@cn-jenkins-deploy-platform-ansible-os-4079-3 2025-09-11 01:54:45.715255+00:00 [info] <0.272.0> ra system 'coordination' running pre init for 0 registered servers 2025-09-11 01:54:45.715746+00:00 [info] <0.273.0> ra: meta data store initialised for system coordination. 0 record(s) recovered 2025-09-11 01:54:45.715926+00:00 [notice] <0.278.0> WAL: ra_coordination_log_wal init, open tbls: ra_coordination_log_open_mem_tables, closed tbls: ra_coordination_log_closed_mem_tables 2025-09-11 01:54:45.721388+00:00 [info] <0.222.0> 2025-09-11 01:54:45.721388+00:00 [info] <0.222.0> Starting RabbitMQ 3.10.7 on Erlang 25.0.4 [jit] 2025-09-11 01:54:45.721388+00:00 [info] <0.222.0> Copyright (c) 2007-2022 VMware, Inc. or its affiliates. 2025-09-11 01:54:45.721388+00:00 [info] <0.222.0> Licensed under the MPL 2.0. Website: https://rabbitmq.com 2025-09-11 01:54:45.722479+00:00 [info] <0.222.0> 2025-09-11 01:54:45.722479+00:00 [info] <0.222.0> node : contrail@cn-jenkins-deploy-platform-ansible-os-4079-3 2025-09-11 01:54:45.722479+00:00 [info] <0.222.0> home dir : /var/lib/rabbitmq 2025-09-11 01:54:45.722479+00:00 [info] <0.222.0> config file(s) : /etc/rabbitmq/rabbitmq.config 2025-09-11 01:54:45.722479+00:00 [info] <0.222.0> : /etc/rabbitmq/conf.d/10-defaults.conf 2025-09-11 01:54:45.722479+00:00 [info] <0.222.0> cookie hash : esZsDxSN6VGbi9JkMSxNZA== 2025-09-11 01:54:45.722479+00:00 [info] <0.222.0> log(s) : /var/log/contrail/config-database-rabbitmq/rabbitmq.log 2025-09-11 01:54:45.722479+00:00 [info] <0.222.0> : /var/log/rabbitmq/contrail@cn-jenkins-deploy-platform-ansible-os-4079-3_upgrade.log 2025-09-11 01:54:45.722479+00:00 [info] <0.222.0> : 2025-09-11 01:54:45.722479+00:00 [info] <0.222.0> database dir : /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4079-3 2025-09-11 01:54:45.930675+00:00 [info] <0.222.0> Feature flags: list of feature flags found: 2025-09-11 01:54:45.930749+00:00 [info] <0.222.0> Feature flags: [ ] classic_mirrored_queue_version 2025-09-11 01:54:45.930773+00:00 [info] <0.222.0> Feature flags: [ ] drop_unroutable_metric 2025-09-11 01:54:45.930801+00:00 [info] <0.222.0> Feature flags: [ ] empty_basic_get_metric 2025-09-11 01:54:45.930858+00:00 [info] <0.222.0> Feature flags: [ ] implicit_default_bindings 2025-09-11 01:54:45.930876+00:00 [info] <0.222.0> Feature flags: [ ] maintenance_mode_status 2025-09-11 01:54:45.930905+00:00 [info] <0.222.0> Feature flags: [ ] quorum_queue 2025-09-11 01:54:45.930964+00:00 [info] <0.222.0> Feature flags: [ ] stream_queue 2025-09-11 01:54:45.930982+00:00 [info] <0.222.0> Feature flags: [ ] user_limits 2025-09-11 01:54:45.930997+00:00 [info] <0.222.0> Feature flags: [ ] virtual_host_metadata 2025-09-11 01:54:45.931013+00:00 [info] <0.222.0> Feature flags: feature flag states written to disk: yes 2025-09-11 01:54:48.241954+00:00 [info] <0.222.0> Running boot step pre_boot defined by app rabbit 2025-09-11 01:54:48.242086+00:00 [info] <0.222.0> Running boot step rabbit_global_counters defined by app rabbit 2025-09-11 01:54:48.242366+00:00 [info] <0.222.0> Running boot step rabbit_osiris_metrics defined by app rabbit 2025-09-11 01:54:48.242475+00:00 [info] <0.222.0> Running boot step rabbit_core_metrics defined by app rabbit 2025-09-11 01:54:48.243052+00:00 [info] <0.222.0> Running boot step rabbit_alarm defined by app rabbit 2025-09-11 01:54:48.258798+00:00 [info] <0.299.0> Memory high watermark set to 12637 MiB (13251084288 bytes) of 15796 MiB (16563855360 bytes) total 2025-09-11 01:54:48.264349+00:00 [info] <0.301.0> Enabling free disk space monitoring 2025-09-11 01:54:48.264416+00:00 [info] <0.301.0> Disk free limit set to 50MB 2025-09-11 01:54:48.267647+00:00 [info] <0.222.0> Running boot step code_server_cache defined by app rabbit 2025-09-11 01:54:48.267761+00:00 [info] <0.222.0> Running boot step file_handle_cache defined by app rabbit 2025-09-11 01:54:48.267956+00:00 [info] <0.306.0> Limiting to approx 1048479 file handles (943629 sockets) 2025-09-11 01:54:48.268060+00:00 [info] <0.307.0> FHC read buffering: OFF 2025-09-11 01:54:48.268093+00:00 [info] <0.307.0> FHC write buffering: ON 2025-09-11 01:54:48.268426+00:00 [info] <0.222.0> Running boot step worker_pool defined by app rabbit 2025-09-11 01:54:48.268488+00:00 [info] <0.280.0> Will use 4 processes for default worker pool 2025-09-11 01:54:48.268522+00:00 [info] <0.280.0> Starting worker pool 'worker_pool' with 4 processes in it 2025-09-11 01:54:48.268788+00:00 [info] <0.222.0> Running boot step database defined by app rabbit 2025-09-11 01:54:48.269068+00:00 [info] <0.222.0> Node database directory at /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4079-3 is empty. Assuming we need to join an existing cluster or initialise from scratch... 2025-09-11 01:54:48.269129+00:00 [info] <0.222.0> Configured peer discovery backend: rabbit_peer_discovery_classic_config 2025-09-11 01:54:48.269156+00:00 [info] <0.222.0> Will try to lock with peer discovery backend rabbit_peer_discovery_classic_config 2025-09-11 01:54:59.009183+00:00 [info] <0.222.0> All discovered existing cluster peers: contrail@cn-jenkins-deploy-platform-ansible-os-4079-1, contrail@cn-jenkins-deploy-platform-ansible-os-4079-2, contrail@cn-jenkins-deploy-platform-ansible-os-4079-3 2025-09-11 01:54:59.009331+00:00 [info] <0.222.0> Peer nodes we can cluster with: contrail@cn-jenkins-deploy-platform-ansible-os-4079-1, contrail@cn-jenkins-deploy-platform-ansible-os-4079-2 2025-09-11 01:54:59.012296+00:00 [info] <0.222.0> Node 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-1' selected for auto-clustering 2025-09-11 01:54:59.138631+00:00 [info] <0.222.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-09-11 01:54:59.138807+00:00 [info] <0.222.0> Successfully synced tables from a peer 2025-09-11 01:54:59.626216+00:00 [info] <0.222.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-09-11 01:54:59.626347+00:00 [info] <0.222.0> Successfully synced tables from a peer 2025-09-11 01:54:59.626401+00:00 [warning] <0.222.0> Feature flags: the previous instance of this node must have failed to write the `feature_flags` file at `/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4079-3-feature_flags`: 2025-09-11 01:54:59.626437+00:00 [warning] <0.222.0> Feature flags: - list of previously enabled feature flags now marked as such: [maintenance_mode_status, 2025-09-11 01:54:59.626437+00:00 [warning] <0.222.0> quorum_queue, 2025-09-11 01:54:59.626437+00:00 [warning] <0.222.0> user_limits, 2025-09-11 01:54:59.626437+00:00 [warning] <0.222.0> virtual_host_metadata] 2025-09-11 01:54:59.627818+00:00 [info] <0.222.0> Feature flags: list of feature flags found: 2025-09-11 01:54:59.627931+00:00 [info] <0.222.0> Feature flags: [ ] classic_mirrored_queue_version 2025-09-11 01:54:59.627959+00:00 [info] <0.222.0> Feature flags: [ ] drop_unroutable_metric 2025-09-11 01:54:59.627987+00:00 [info] <0.222.0> Feature flags: [ ] empty_basic_get_metric 2025-09-11 01:54:59.628033+00:00 [info] <0.222.0> Feature flags: [ ] implicit_default_bindings 2025-09-11 01:54:59.628052+00:00 [info] <0.222.0> Feature flags: [x] maintenance_mode_status 2025-09-11 01:54:59.628069+00:00 [info] <0.222.0> Feature flags: [x] quorum_queue 2025-09-11 01:54:59.628115+00:00 [info] <0.222.0> Feature flags: [ ] stream_queue 2025-09-11 01:54:59.628133+00:00 [info] <0.222.0> Feature flags: [x] user_limits 2025-09-11 01:54:59.628148+00:00 [info] <0.222.0> Feature flags: [x] virtual_host_metadata 2025-09-11 01:54:59.628181+00:00 [info] <0.222.0> Feature flags: feature flag states written to disk: yes 2025-09-11 01:54:59.638160+00:00 [info] <0.222.0> Feature flag `classic_mirrored_queue_version`: mark as enabled=true 2025-09-11 01:54:59.640445+00:00 [info] <0.222.0> Feature flags: list of feature flags found: 2025-09-11 01:54:59.640514+00:00 [info] <0.222.0> Feature flags: [x] classic_mirrored_queue_version 2025-09-11 01:54:59.640539+00:00 [info] <0.222.0> Feature flags: [ ] drop_unroutable_metric 2025-09-11 01:54:59.640614+00:00 [info] <0.222.0> Feature flags: [ ] empty_basic_get_metric 2025-09-11 01:54:59.640703+00:00 [info] <0.222.0> Feature flags: [ ] implicit_default_bindings 2025-09-11 01:54:59.640757+00:00 [info] <0.222.0> Feature flags: [x] maintenance_mode_status 2025-09-11 01:54:59.640798+00:00 [info] <0.222.0> Feature flags: [x] quorum_queue 2025-09-11 01:54:59.640841+00:00 [info] <0.222.0> Feature flags: [ ] stream_queue 2025-09-11 01:54:59.640916+00:00 [info] <0.222.0> Feature flags: [x] user_limits 2025-09-11 01:54:59.640939+00:00 [info] <0.222.0> Feature flags: [x] virtual_host_metadata 2025-09-11 01:54:59.640991+00:00 [info] <0.222.0> Feature flags: feature flag states written to disk: yes 2025-09-11 01:54:59.649162+00:00 [info] <0.222.0> Feature flag `drop_unroutable_metric`: mark as enabled=true 2025-09-11 01:54:59.650928+00:00 [info] <0.222.0> Feature flags: list of feature flags found: 2025-09-11 01:54:59.650989+00:00 [info] <0.222.0> Feature flags: [x] classic_mirrored_queue_version 2025-09-11 01:54:59.651020+00:00 [info] <0.222.0> Feature flags: [x] drop_unroutable_metric 2025-09-11 01:54:59.651051+00:00 [info] <0.222.0> Feature flags: [ ] empty_basic_get_metric 2025-09-11 01:54:59.651074+00:00 [info] <0.222.0> Feature flags: [ ] implicit_default_bindings 2025-09-11 01:54:59.651091+00:00 [info] <0.222.0> Feature flags: [x] maintenance_mode_status 2025-09-11 01:54:59.651107+00:00 [info] <0.222.0> Feature flags: [x] quorum_queue 2025-09-11 01:54:59.651128+00:00 [info] <0.222.0> Feature flags: [ ] stream_queue 2025-09-11 01:54:59.651143+00:00 [info] <0.222.0> Feature flags: [x] user_limits 2025-09-11 01:54:59.651188+00:00 [info] <0.222.0> Feature flags: [x] virtual_host_metadata 2025-09-11 01:54:59.651227+00:00 [info] <0.222.0> Feature flags: feature flag states written to disk: yes 2025-09-11 01:54:59.658513+00:00 [info] <0.222.0> Feature flag `empty_basic_get_metric`: mark as enabled=true 2025-09-11 01:54:59.660965+00:00 [info] <0.222.0> Feature flags: list of feature flags found: 2025-09-11 01:54:59.661029+00:00 [info] <0.222.0> Feature flags: [x] classic_mirrored_queue_version 2025-09-11 01:54:59.661054+00:00 [info] <0.222.0> Feature flags: [x] drop_unroutable_metric 2025-09-11 01:54:59.661081+00:00 [info] <0.222.0> Feature flags: [x] empty_basic_get_metric 2025-09-11 01:54:59.661100+00:00 [info] <0.222.0> Feature flags: [ ] implicit_default_bindings 2025-09-11 01:54:59.661848+00:00 [info] <0.222.0> Feature flags: [x] maintenance_mode_status 2025-09-11 01:54:59.661874+00:00 [info] <0.222.0> Feature flags: [x] quorum_queue 2025-09-11 01:54:59.662299+00:00 [info] <0.222.0> Feature flags: [ ] stream_queue 2025-09-11 01:54:59.662386+00:00 [info] <0.222.0> Feature flags: [x] user_limits 2025-09-11 01:54:59.662409+00:00 [info] <0.222.0> Feature flags: [x] virtual_host_metadata 2025-09-11 01:54:59.662453+00:00 [info] <0.222.0> Feature flags: feature flag states written to disk: yes 2025-09-11 01:54:59.671320+00:00 [info] <0.222.0> Waiting for Mnesia tables for 30000 ms, 0 retries left 2025-09-11 01:54:59.675622+00:00 [info] <0.222.0> Successfully synced tables from a peer 2025-09-11 01:54:59.675697+00:00 [info] <0.222.0> Feature flag `implicit_default_bindings`: mark as enabled=true 2025-09-11 01:54:59.677417+00:00 [info] <0.222.0> Feature flags: list of feature flags found: 2025-09-11 01:54:59.677471+00:00 [info] <0.222.0> Feature flags: [x] classic_mirrored_queue_version 2025-09-11 01:54:59.677494+00:00 [info] <0.222.0> Feature flags: [x] drop_unroutable_metric 2025-09-11 01:54:59.677512+00:00 [info] <0.222.0> Feature flags: [x] empty_basic_get_metric 2025-09-11 01:54:59.677530+00:00 [info] <0.222.0> Feature flags: [x] implicit_default_bindings 2025-09-11 01:54:59.677547+00:00 [info] <0.222.0> Feature flags: [x] maintenance_mode_status 2025-09-11 01:54:59.677565+00:00 [info] <0.222.0> Feature flags: [x] quorum_queue 2025-09-11 01:54:59.677579+00:00 [info] <0.222.0> Feature flags: [ ] stream_queue 2025-09-11 01:54:59.677648+00:00 [info] <0.222.0> Feature flags: [x] user_limits 2025-09-11 01:54:59.677664+00:00 [info] <0.222.0> Feature flags: [x] virtual_host_metadata 2025-09-11 01:54:59.677680+00:00 [info] <0.222.0> Feature flags: feature flag states written to disk: yes 2025-09-11 01:54:59.684964+00:00 [info] <0.222.0> Feature flag `stream_queue`: mark as enabled=true 2025-09-11 01:54:59.686510+00:00 [info] <0.222.0> Feature flags: list of feature flags found: 2025-09-11 01:54:59.686581+00:00 [info] <0.222.0> Feature flags: [x] classic_mirrored_queue_version 2025-09-11 01:54:59.686604+00:00 [info] <0.222.0> Feature flags: [x] drop_unroutable_metric 2025-09-11 01:54:59.686623+00:00 [info] <0.222.0> Feature flags: [x] empty_basic_get_metric 2025-09-11 01:54:59.686677+00:00 [info] <0.222.0> Feature flags: [x] implicit_default_bindings 2025-09-11 01:54:59.686698+00:00 [info] <0.222.0> Feature flags: [x] maintenance_mode_status 2025-09-11 01:54:59.686715+00:00 [info] <0.222.0> Feature flags: [x] quorum_queue 2025-09-11 01:54:59.686731+00:00 [info] <0.222.0> Feature flags: [x] stream_queue 2025-09-11 01:54:59.686787+00:00 [info] <0.222.0> Feature flags: [x] user_limits 2025-09-11 01:54:59.686803+00:00 [info] <0.222.0> Feature flags: [x] virtual_host_metadata 2025-09-11 01:54:59.686819+00:00 [info] <0.222.0> Feature flags: feature flag states written to disk: yes 2025-09-11 01:54:59.696518+00:00 [info] <0.222.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-09-11 01:54:59.696725+00:00 [info] <0.222.0> Successfully synced tables from a peer 2025-09-11 01:54:59.723952+00:00 [info] <0.222.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-09-11 01:54:59.724112+00:00 [info] <0.222.0> Successfully synced tables from a peer 2025-09-11 01:54:59.739457+00:00 [info] <0.222.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-4079-3' 2025-09-11 01:54:59.752259+00:00 [info] <0.222.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-4079-3' 2025-09-11 01:54:59.761796+00:00 [info] <0.222.0> Setting up a table for per-user connection counting on this node: 'tracked_connection_table_per_user_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-4079-3' 2025-09-11 01:54:59.763396+00:00 [info] <0.222.0> Peer discovery backend rabbit_peer_discovery_classic_config does not support registration, skipping registration. 2025-09-11 01:54:59.763464+00:00 [info] <0.222.0> Will try to unlock with peer discovery backend rabbit_peer_discovery_classic_config 2025-09-11 01:54:59.764651+00:00 [info] <0.222.0> Running boot step database_sync defined by app rabbit 2025-09-11 01:54:59.764797+00:00 [info] <0.222.0> Running boot step feature_flags defined by app rabbit 2025-09-11 01:54:59.764974+00:00 [info] <0.222.0> Running boot step codec_correctness_check defined by app rabbit 2025-09-11 01:54:59.765028+00:00 [info] <0.222.0> Running boot step external_infrastructure defined by app rabbit 2025-09-11 01:54:59.765058+00:00 [info] <0.222.0> Running boot step rabbit_event defined by app rabbit 2025-09-11 01:54:59.765192+00:00 [info] <0.222.0> Running boot step rabbit_registry defined by app rabbit 2025-09-11 01:54:59.765279+00:00 [info] <0.222.0> Running boot step rabbit_auth_mechanism_amqplain defined by app rabbit 2025-09-11 01:54:59.765337+00:00 [info] <0.222.0> Running boot step rabbit_auth_mechanism_cr_demo defined by app rabbit 2025-09-11 01:54:59.765380+00:00 [info] <0.222.0> Running boot step rabbit_auth_mechanism_plain defined by app rabbit 2025-09-11 01:54:59.765416+00:00 [info] <0.222.0> Running boot step rabbit_exchange_type_direct defined by app rabbit 2025-09-11 01:54:59.765448+00:00 [info] <0.222.0> Running boot step rabbit_exchange_type_fanout defined by app rabbit 2025-09-11 01:54:59.765479+00:00 [info] <0.222.0> Running boot step rabbit_exchange_type_headers defined by app rabbit 2025-09-11 01:54:59.765514+00:00 [info] <0.222.0> Running boot step rabbit_exchange_type_topic defined by app rabbit 2025-09-11 01:54:59.765542+00:00 [info] <0.222.0> Running boot step rabbit_mirror_queue_mode_all defined by app rabbit 2025-09-11 01:54:59.765571+00:00 [info] <0.222.0> Running boot step rabbit_mirror_queue_mode_exactly defined by app rabbit 2025-09-11 01:54:59.765599+00:00 [info] <0.222.0> Running boot step rabbit_mirror_queue_mode_nodes defined by app rabbit 2025-09-11 01:54:59.765627+00:00 [info] <0.222.0> Running boot step rabbit_priority_queue defined by app rabbit 2025-09-11 01:54:59.765647+00:00 [info] <0.222.0> Priority queues enabled, real BQ is rabbit_variable_queue 2025-09-11 01:54:59.765705+00:00 [info] <0.222.0> Running boot step rabbit_queue_location_client_local defined by app rabbit 2025-09-11 01:54:59.768284+00:00 [info] <0.222.0> Running boot step rabbit_queue_location_min_masters defined by app rabbit 2025-09-11 01:54:59.768338+00:00 [info] <0.222.0> Running boot step rabbit_queue_location_random defined by app rabbit 2025-09-11 01:54:59.768373+00:00 [info] <0.222.0> Running boot step kernel_ready defined by app rabbit 2025-09-11 01:54:59.768391+00:00 [info] <0.222.0> Running boot step rabbit_sysmon_minder defined by app rabbit 2025-09-11 01:54:59.768491+00:00 [info] <0.222.0> Running boot step rabbit_epmd_monitor defined by app rabbit 2025-09-11 01:54:59.769492+00:00 [info] <0.502.0> epmd monitor knows us, inter-node communication (distribution) port: 25673 2025-09-11 01:54:59.769592+00:00 [info] <0.222.0> Running boot step guid_generator defined by app rabbit 2025-09-11 01:54:59.773105+00:00 [info] <0.222.0> Running boot step rabbit_node_monitor defined by app rabbit 2025-09-11 01:54:59.773920+00:00 [info] <0.506.0> Starting rabbit_node_monitor 2025-09-11 01:54:59.774017+00:00 [info] <0.222.0> Running boot step delegate_sup defined by app rabbit 2025-09-11 01:54:59.774452+00:00 [info] <0.222.0> Running boot step rabbit_memory_monitor defined by app rabbit 2025-09-11 01:54:59.774620+00:00 [info] <0.222.0> Running boot step rabbit_fifo_dlx_sup defined by app rabbit 2025-09-11 01:54:59.774672+00:00 [info] <0.222.0> Running boot step core_initialized defined by app rabbit 2025-09-11 01:54:59.774694+00:00 [info] <0.222.0> Running boot step upgrade_queues defined by app rabbit 2025-09-11 01:54:59.783747+00:00 [info] <0.222.0> message_store upgrades: 1 to apply 2025-09-11 01:54:59.783884+00:00 [info] <0.222.0> message_store upgrades: Applying rabbit_variable_queue:move_messages_to_vhost_store 2025-09-11 01:54:59.784021+00:00 [info] <0.222.0> message_store upgrades: No durable queues found. Skipping message store migration 2025-09-11 01:54:59.784127+00:00 [info] <0.222.0> message_store upgrades: Removing the old message store data 2025-09-11 01:54:59.784529+00:00 [info] <0.222.0> message_store upgrades: All upgrades applied successfully 2025-09-11 01:54:59.798033+00:00 [info] <0.222.0> Running boot step channel_tracking defined by app rabbit 2025-09-11 01:54:59.806029+00:00 [info] <0.222.0> Setting up a table for channel tracking on this node: 'tracked_channel_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-4079-3' 2025-09-11 01:54:59.812696+00:00 [info] <0.222.0> Setting up a table for channel tracking on this node: 'tracked_channel_table_per_user_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-4079-3' 2025-09-11 01:54:59.812986+00:00 [info] <0.222.0> Running boot step rabbit_channel_tracking_handler defined by app rabbit 2025-09-11 01:54:59.813090+00:00 [info] <0.222.0> Running boot step connection_tracking defined by app rabbit 2025-09-11 01:54:59.813909+00:00 [info] <0.222.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-4079-3' 2025-09-11 01:54:59.814508+00:00 [info] <0.222.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-4079-3' 2025-09-11 01:54:59.815142+00:00 [info] <0.222.0> Setting up a table for per-user connection counting on this node: 'tracked_connection_table_per_user_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-4079-3' 2025-09-11 01:54:59.815418+00:00 [info] <0.222.0> Running boot step rabbit_connection_tracking_handler defined by app rabbit 2025-09-11 01:54:59.815496+00:00 [info] <0.222.0> Running boot step rabbit_definitions_hashing defined by app rabbit 2025-09-11 01:54:59.815554+00:00 [info] <0.222.0> Running boot step rabbit_exchange_parameters defined by app rabbit 2025-09-11 01:54:59.815630+00:00 [info] <0.222.0> Running boot step rabbit_mirror_queue_misc defined by app rabbit 2025-09-11 01:54:59.815795+00:00 [info] <0.222.0> Running boot step rabbit_policies defined by app rabbit 2025-09-11 01:54:59.816118+00:00 [info] <0.222.0> Running boot step rabbit_policy defined by app rabbit 2025-09-11 01:54:59.816181+00:00 [info] <0.222.0> Running boot step rabbit_queue_location_validator defined by app rabbit 2025-09-11 01:54:59.816230+00:00 [info] <0.222.0> Running boot step rabbit_quorum_memory_manager defined by app rabbit 2025-09-11 01:54:59.816298+00:00 [info] <0.222.0> Running boot step rabbit_stream_coordinator defined by app rabbit 2025-09-11 01:54:59.816414+00:00 [info] <0.222.0> Running boot step rabbit_vhost_limit defined by app rabbit 2025-09-11 01:54:59.816501+00:00 [info] <0.222.0> Running boot step rabbit_mgmt_reset_handler defined by app rabbitmq_management 2025-09-11 01:54:59.816543+00:00 [info] <0.222.0> Running boot step rabbit_mgmt_db_handler defined by app rabbitmq_management_agent 2025-09-11 01:54:59.816593+00:00 [warning] <0.222.0> force_fine_statistics set to true; ignored. 2025-09-11 01:54:59.816593+00:00 [warning] <0.222.0> Replaced by {rates_mode, none} in the rabbitmq_management application. 2025-09-11 01:54:59.816624+00:00 [info] <0.222.0> Management plugin: using rates mode 'basic' 2025-09-11 01:54:59.817846+00:00 [info] <0.222.0> Running boot step recovery defined by app rabbit 2025-09-11 01:54:59.819425+00:00 [info] <0.555.0> Making sure data directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4079-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists 2025-09-11 01:54:59.820313+00:00 [info] <0.555.0> Setting segment_entry_count for vhost '/' with 0 queues to '2048' 2025-09-11 01:54:59.827058+00:00 [info] <0.555.0> Starting message stores for vhost '/' 2025-09-11 01:54:59.827267+00:00 [info] <0.560.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index 2025-09-11 01:54:59.828387+00:00 [info] <0.555.0> Started message store of type transient for vhost '/' 2025-09-11 01:54:59.828636+00:00 [info] <0.564.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index 2025-09-11 01:54:59.829317+00:00 [warning] <0.564.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": rebuilding indices from scratch 2025-09-11 01:54:59.830066+00:00 [info] <0.555.0> Started message store of type persistent for vhost '/' 2025-09-11 01:54:59.830229+00:00 [info] <0.555.0> Recovering 0 queues of type rabbit_classic_queue took 9ms 2025-09-11 01:54:59.830262+00:00 [info] <0.555.0> Recovering 0 queues of type rabbit_quorum_queue took 0ms 2025-09-11 01:54:59.830285+00:00 [info] <0.555.0> Recovering 0 queues of type rabbit_stream_queue took 0ms 2025-09-11 01:54:59.831404+00:00 [info] <0.222.0> Running boot step empty_db_check defined by app rabbit 2025-09-11 01:54:59.831452+00:00 [info] <0.222.0> Will not seed default virtual host and user: have definitions to load... 2025-09-11 01:54:59.831473+00:00 [info] <0.222.0> Running boot step rabbit_looking_glass defined by app rabbit 2025-09-11 01:54:59.831494+00:00 [info] <0.222.0> Running boot step rabbit_core_metrics_gc defined by app rabbit 2025-09-11 01:54:59.831587+00:00 [info] <0.222.0> Running boot step background_gc defined by app rabbit 2025-09-11 01:54:59.832137+00:00 [info] <0.222.0> Running boot step routing_ready defined by app rabbit 2025-09-11 01:54:59.832282+00:00 [info] <0.222.0> Running boot step pre_flight defined by app rabbit 2025-09-11 01:54:59.832332+00:00 [info] <0.222.0> Running boot step notify_cluster defined by app rabbit 2025-09-11 01:54:59.832409+00:00 [info] <0.222.0> Running boot step networking defined by app rabbit 2025-09-11 01:54:59.832445+00:00 [info] <0.222.0> Running boot step definition_import_worker_pool defined by app rabbit 2025-09-11 01:54:59.832605+00:00 [info] <0.280.0> Starting worker pool 'definition_import_pool' with 4 processes in it 2025-09-11 01:54:59.832811+00:00 [info] <0.222.0> Running boot step cluster_name defined by app rabbit 2025-09-11 01:54:59.832844+00:00 [info] <0.222.0> Running boot step direct_client defined by app rabbit 2025-09-11 01:54:59.832916+00:00 [info] <0.222.0> Running boot step rabbit_maintenance_mode_state defined by app rabbit 2025-09-11 01:54:59.832959+00:00 [info] <0.222.0> Creating table rabbit_node_maintenance_states for feature flag `maintenance_mode_status` 2025-09-11 01:54:59.833682+00:00 [info] <0.506.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-2' up 2025-09-11 01:54:59.834339+00:00 [info] <0.222.0> Running boot step rabbit_management_load_definitions defined by app rabbitmq_management 2025-09-11 01:54:59.834708+00:00 [info] <0.222.0> Applying definitions from file at '/etc/rabbitmq/definitions.json' 2025-09-11 01:54:59.834779+00:00 [info] <0.222.0> Asked to import definitions. Acting user: rmq-internal 2025-09-11 01:54:59.835030+00:00 [info] <0.222.0> Importing concurrently 1 users... 2025-09-11 01:54:59.845498+00:00 [info] <0.222.0> Importing concurrently 1 vhosts... 2025-09-11 01:54:59.845928+00:00 [info] <0.506.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-1' up 2025-09-11 01:54:59.851124+00:00 [info] <0.222.0> Importing concurrently 1 permissions... 2025-09-11 01:54:59.854821+00:00 [info] <0.588.0> Successfully set permissions for 'guest' in virtual host '/' to '.*', '.*', '.*' 2025-09-11 01:54:59.854961+00:00 [info] <0.222.0> Importing sequentially 1 policies... 2025-09-11 01:54:59.859126+00:00 [info] <0.600.0> Resetting node maintenance status 2025-09-11 01:54:59.871641+00:00 [info] <0.659.0> Management plugin: HTTP (non-TLS) listener started on port 15673 2025-09-11 01:54:59.871806+00:00 [info] <0.687.0> Statistics database started. 2025-09-11 01:54:59.871907+00:00 [info] <0.686.0> Starting worker pool 'management_worker_pool' with 3 processes in it 2025-09-11 01:54:59.878988+00:00 [info] <0.701.0> Prometheus metrics: HTTP (non-TLS) listener started on port 15692 2025-09-11 01:54:59.879149+00:00 [info] <0.600.0> Ready to start client connection listeners 2025-09-11 01:54:59.880473+00:00 [info] <0.745.0> started TCP listener on 10.0.0.49:5673 2025-09-11 01:54:59.958946+00:00 [info] <0.600.0> Server startup complete; 4 plugins started. 2025-09-11 01:54:59.958946+00:00 [info] <0.600.0> * rabbitmq_prometheus 2025-09-11 01:54:59.958946+00:00 [info] <0.600.0> * rabbitmq_management 2025-09-11 01:54:59.958946+00:00 [info] <0.600.0> * rabbitmq_web_dispatch 2025-09-11 01:54:59.958946+00:00 [info] <0.600.0> * rabbitmq_management_agent 2025-09-11 01:55:38.010240+00:00 [info] <0.506.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-2' down 2025-09-11 01:55:38.025030+00:00 [info] <0.506.0> Keeping contrail@cn-jenkins-deploy-platform-ansible-os-4079-2 listeners: the node is already back 2025-09-11 01:55:41.524024+00:00 [info] <0.1032.0> RabbitMQ is asked to stop... 2025-09-11 01:55:41.579555+00:00 [info] <0.1032.0> Stopping RabbitMQ applications and their dependencies in the following order: 2025-09-11 01:55:41.579555+00:00 [info] <0.1032.0> rabbitmq_management 2025-09-11 01:55:41.579555+00:00 [info] <0.1032.0> rabbitmq_prometheus 2025-09-11 01:55:41.579555+00:00 [info] <0.1032.0> rabbitmq_web_dispatch 2025-09-11 01:55:41.579555+00:00 [info] <0.1032.0> cowboy 2025-09-11 01:55:41.579555+00:00 [info] <0.1032.0> cowlib 2025-09-11 01:55:41.579555+00:00 [info] <0.1032.0> rabbitmq_management_agent 2025-09-11 01:55:41.579555+00:00 [info] <0.1032.0> rabbit 2025-09-11 01:55:41.579555+00:00 [info] <0.1032.0> amqp_client 2025-09-11 01:55:41.579555+00:00 [info] <0.1032.0> rabbitmq_prelaunch 2025-09-11 01:55:41.579555+00:00 [info] <0.1032.0> rabbit_common 2025-09-11 01:55:41.579555+00:00 [info] <0.1032.0> prometheus 2025-09-11 01:55:41.579555+00:00 [info] <0.1032.0> ra 2025-09-11 01:55:41.579555+00:00 [info] <0.1032.0> accept 2025-09-11 01:55:41.579555+00:00 [info] <0.1032.0> os_mon 2025-09-11 01:55:41.579555+00:00 [info] <0.1032.0> sysmon_handler 2025-09-11 01:55:41.579555+00:00 [info] <0.1032.0> osiris 2025-09-11 01:55:41.579555+00:00 [info] <0.1032.0> mnesia 2025-09-11 01:55:41.579555+00:00 [info] <0.1032.0> 2025-09-11 01:55:41.579691+00:00 [info] <0.1032.0> Stopping application 'rabbitmq_management' 2025-09-11 01:55:41.581010+00:00 [warning] <0.651.0> HTTP listener registry could not find context rabbitmq_management_tls 2025-09-11 01:55:41.582569+00:00 [notice] <0.44.0> Application rabbitmq_management exited with reason: stopped 2025-09-11 01:55:41.582644+00:00 [info] <0.1032.0> Stopping application 'rabbitmq_prometheus' 2025-09-11 01:55:41.588532+00:00 [warning] <0.651.0> HTTP listener registry could not find context rabbitmq_prometheus_tls 2025-09-11 01:55:41.590221+00:00 [notice] <0.44.0> Application rabbitmq_prometheus exited with reason: stopped 2025-09-11 01:55:41.590296+00:00 [info] <0.1032.0> Stopping application 'rabbitmq_web_dispatch' 2025-09-11 01:55:41.591936+00:00 [notice] <0.44.0> Application rabbitmq_web_dispatch exited with reason: stopped 2025-09-11 01:55:41.592016+00:00 [info] <0.1032.0> Stopping application 'cowboy' 2025-09-11 01:55:41.593753+00:00 [notice] <0.44.0> Application cowboy exited with reason: stopped 2025-09-11 01:55:41.593806+00:00 [info] <0.1032.0> Stopping application 'cowlib' 2025-09-11 01:55:41.593980+00:00 [notice] <0.44.0> Application cowlib exited with reason: stopped 2025-09-11 01:55:41.594012+00:00 [info] <0.1032.0> Stopping application 'rabbitmq_management_agent' 2025-09-11 01:55:41.595974+00:00 [notice] <0.44.0> Application rabbitmq_management_agent exited with reason: stopped 2025-09-11 01:55:41.596040+00:00 [info] <0.1032.0> Stopping application 'rabbit' 2025-09-11 01:55:41.596174+00:00 [info] <0.222.0> Peer discovery backend rabbit_peer_discovery_classic_config does not support registration, skipping unregistration. 2025-09-11 01:55:41.596272+00:00 [info] <0.745.0> stopped TCP listener on 10.0.0.49:5673 2025-09-11 01:55:41.597453+00:00 [info] <0.1033.0> Closing all connections in vhost '/' on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-3' because the vhost is stopping 2025-09-11 01:55:41.597483+00:00 [info] <0.564.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4079-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' 2025-09-11 01:55:41.608609+00:00 [info] <0.564.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4079-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' is stopped 2025-09-11 01:55:41.608763+00:00 [info] <0.560.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4079-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' 2025-09-11 01:55:41.612609+00:00 [info] <0.560.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4079-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' is stopped 2025-09-11 01:55:41.613330+00:00 [info] <0.497.0> Management plugin: to stop collect_statistics. 2025-09-11 01:55:46.616155+00:00 [notice] <0.44.0> Application rabbit exited with reason: stopped 2025-09-11 01:55:46.616264+00:00 [info] <0.1032.0> Stopping application 'amqp_client' 2025-09-11 01:55:46.617972+00:00 [notice] <0.44.0> Application amqp_client exited with reason: stopped 2025-09-11 01:55:46.618084+00:00 [info] <0.1032.0> Stopping application 'rabbitmq_prelaunch' 2025-09-11 01:55:46.619641+00:00 [notice] <0.44.0> Application rabbitmq_prelaunch exited with reason: stopped 2025-09-11 01:55:46.619720+00:00 [info] <0.1032.0> Stopping application 'rabbit_common' 2025-09-11 01:55:46.619780+00:00 [notice] <0.44.0> Application rabbit_common exited with reason: stopped 2025-09-11 01:55:46.619813+00:00 [info] <0.1032.0> Stopping application 'prometheus' 2025-09-11 01:55:46.621157+00:00 [notice] <0.44.0> Application prometheus exited with reason: stopped 2025-09-11 01:55:46.621233+00:00 [info] <0.1032.0> Stopping application 'ra' 2025-09-11 01:55:46.623847+00:00 [notice] <0.44.0> Application ra exited with reason: stopped 2025-09-11 01:55:46.623906+00:00 [info] <0.1032.0> Stopping application 'accept' 2025-09-11 01:55:46.623944+00:00 [notice] <0.44.0> Application accept exited with reason: stopped 2025-09-11 01:55:46.624010+00:00 [info] <0.1032.0> Stopping application 'os_mon' 2025-09-11 01:55:46.625316+00:00 [notice] <0.44.0> Application os_mon exited with reason: stopped 2025-09-11 01:55:46.625366+00:00 [info] <0.1032.0> Stopping application 'sysmon_handler' 2025-09-11 01:55:46.626754+00:00 [notice] <0.44.0> Application sysmon_handler exited with reason: stopped 2025-09-11 01:55:46.626820+00:00 [info] <0.1032.0> Stopping application 'osiris' 2025-09-11 01:55:46.628186+00:00 [notice] <0.44.0> Application osiris exited with reason: stopped 2025-09-11 01:55:46.628276+00:00 [info] <0.1032.0> Stopping application 'mnesia' 2025-09-11 01:55:46.630601+00:00 [notice] <0.44.0> Application mnesia exited with reason: stopped 2025-09-11 01:55:46.630687+00:00 [info] <0.1032.0> Successfully stopped RabbitMQ and its dependencies 2025-09-11 01:55:54.096021+00:00 [info] <0.1065.0> Resetting Rabbit forcefully 2025-09-11 01:55:54.615877+00:00 [notice] <0.44.0> Application mnesia exited with reason: stopped 2025-09-11 01:55:54.621114+00:00 [notice] <0.44.0> Application mnesia exited with reason: stopped 2025-09-11 01:55:54.718362+00:00 [info] <0.1071.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-09-11 01:55:54.718480+00:00 [info] <0.1071.0> Successfully synced tables from a peer 2025-09-11 01:55:54.727948+00:00 [info] <0.1071.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-09-11 01:55:54.728051+00:00 [info] <0.1071.0> Successfully synced tables from a peer 2025-09-11 01:55:54.728089+00:00 [warning] <0.1071.0> Feature flags: the previous instance of this node must have failed to write the `feature_flags` file at `/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4079-3-feature_flags`: 2025-09-11 01:55:54.728127+00:00 [warning] <0.1071.0> Feature flags: - list of previously disabled feature flags now marked as such: [maintenance_mode_status] 2025-09-11 01:55:54.729881+00:00 [info] <0.1071.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-09-11 01:55:54.730041+00:00 [info] <0.1071.0> Successfully synced tables from a peer 2025-09-11 01:55:54.741662+00:00 [info] <0.1071.0> Waiting for Mnesia tables for 30000 ms, 0 retries left 2025-09-11 01:55:54.741863+00:00 [info] <0.1071.0> Successfully synced tables from a peer 2025-09-11 01:55:54.748165+00:00 [notice] <0.44.0> Application mnesia exited with reason: stopped 2025-09-11 01:55:54.758890+00:00 [info] <0.1071.0> Clustering with ['contrail@cn-jenkins-deploy-platform-ansible-os-4079-2', 2025-09-11 01:55:54.758890+00:00 [info] <0.1071.0> 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-1'] as disc node 2025-09-11 01:55:54.802197+00:00 [info] <0.1071.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-09-11 01:55:54.802674+00:00 [info] <0.1071.0> Successfully synced tables from a peer 2025-09-11 01:55:55.138557+00:00 [notice] <0.44.0> Application mnesia exited with reason: stopped 2025-09-11 01:55:55.608901+00:00 [info] <0.1415.0> Halting Erlang VM with the following applications: 2025-09-11 01:55:55.608901+00:00 [info] <0.1415.0> amqp10_common 2025-09-11 01:55:55.608901+00:00 [info] <0.1415.0> observer_cli 2025-09-11 01:55:55.608901+00:00 [info] <0.1415.0> stdout_formatter 2025-09-11 01:55:55.608901+00:00 [info] <0.1415.0> seshat 2025-09-11 01:55:55.608901+00:00 [info] <0.1415.0> gen_batch_server 2025-09-11 01:55:55.608901+00:00 [info] <0.1415.0> aten 2025-09-11 01:55:55.608901+00:00 [info] <0.1415.0> ranch 2025-09-11 01:55:55.608901+00:00 [info] <0.1415.0> inets 2025-09-11 01:55:55.608901+00:00 [info] <0.1415.0> systemd 2025-09-11 01:55:55.608901+00:00 [info] <0.1415.0> enough 2025-09-11 01:55:55.608901+00:00 [info] <0.1415.0> cuttlefish 2025-09-11 01:55:55.608901+00:00 [info] <0.1415.0> credentials_obfuscation 2025-09-11 01:55:55.608901+00:00 [info] <0.1415.0> recon 2025-09-11 01:55:55.608901+00:00 [info] <0.1415.0> jsx 2025-09-11 01:55:55.608901+00:00 [info] <0.1415.0> xmerl 2025-09-11 01:55:55.608901+00:00 [info] <0.1415.0> tools 2025-09-11 01:55:55.608901+00:00 [info] <0.1415.0> syntax_tools 2025-09-11 01:55:55.608901+00:00 [info] <0.1415.0> ssl 2025-09-11 01:55:55.608901+00:00 [info] <0.1415.0> public_key 2025-09-11 01:55:55.608901+00:00 [info] <0.1415.0> asn1 2025-09-11 01:55:55.608901+00:00 [info] <0.1415.0> crypto 2025-09-11 01:55:55.608901+00:00 [info] <0.1415.0> compiler 2025-09-11 01:55:55.608901+00:00 [info] <0.1415.0> sasl 2025-09-11 01:55:55.608901+00:00 [info] <0.1415.0> stdlib 2025-09-11 01:55:55.608901+00:00 [info] <0.1415.0> kernel 2025-09-11 01:55:55.608901+00:00 [info] <0.1415.0> 2025-09-11 01:55:59.736251+00:00 [notice] <0.222.0> Logging: configured log handlers are now ACTIVE 2025-09-11 01:55:59.962517+00:00 [info] <0.222.0> ra: starting system quorum_queues 2025-09-11 01:55:59.962593+00:00 [info] <0.222.0> starting Ra system: quorum_queues in directory: /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4079-3/quorum/contrail@cn-jenkins-deploy-platform-ansible-os-4079-3 2025-09-11 01:55:59.981673+00:00 [info] <0.348.0> ra system 'quorum_queues' running pre init for 0 registered servers 2025-09-11 01:55:59.990178+00:00 [info] <0.360.0> ra: meta data store initialised for system quorum_queues. 0 record(s) recovered 2025-09-11 01:56:00.000262+00:00 [notice] <0.383.0> WAL: ra_log_wal init, open tbls: ra_log_open_mem_tables, closed tbls: ra_log_closed_mem_tables 2025-09-11 01:56:00.006083+00:00 [info] <0.222.0> ra: starting system coordination 2025-09-11 01:56:00.006141+00:00 [info] <0.222.0> starting Ra system: coordination in directory: /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4079-3/coordination/contrail@cn-jenkins-deploy-platform-ansible-os-4079-3 2025-09-11 01:56:00.007020+00:00 [info] <0.395.0> ra system 'coordination' running pre init for 0 registered servers 2025-09-11 01:56:00.007526+00:00 [info] <0.396.0> ra: meta data store initialised for system coordination. 0 record(s) recovered 2025-09-11 01:56:00.007672+00:00 [notice] <0.401.0> WAL: ra_coordination_log_wal init, open tbls: ra_coordination_log_open_mem_tables, closed tbls: ra_coordination_log_closed_mem_tables 2025-09-11 01:56:00.011643+00:00 [info] <0.222.0> 2025-09-11 01:56:00.011643+00:00 [info] <0.222.0> Starting RabbitMQ 3.10.7 on Erlang 25.0.4 [jit] 2025-09-11 01:56:00.011643+00:00 [info] <0.222.0> Copyright (c) 2007-2022 VMware, Inc. or its affiliates. 2025-09-11 01:56:00.011643+00:00 [info] <0.222.0> Licensed under the MPL 2.0. Website: https://rabbitmq.com 2025-09-11 01:56:00.012697+00:00 [info] <0.222.0> 2025-09-11 01:56:00.012697+00:00 [info] <0.222.0> node : contrail@cn-jenkins-deploy-platform-ansible-os-4079-3 2025-09-11 01:56:00.012697+00:00 [info] <0.222.0> home dir : /var/lib/rabbitmq 2025-09-11 01:56:00.012697+00:00 [info] <0.222.0> config file(s) : /etc/rabbitmq/rabbitmq.config 2025-09-11 01:56:00.012697+00:00 [info] <0.222.0> : /etc/rabbitmq/conf.d/10-defaults.conf 2025-09-11 01:56:00.012697+00:00 [info] <0.222.0> cookie hash : esZsDxSN6VGbi9JkMSxNZA== 2025-09-11 01:56:00.012697+00:00 [info] <0.222.0> log(s) : /var/log/contrail/config-database-rabbitmq/rabbitmq.log 2025-09-11 01:56:00.012697+00:00 [info] <0.222.0> : /var/log/rabbitmq/contrail@cn-jenkins-deploy-platform-ansible-os-4079-3_upgrade.log 2025-09-11 01:56:00.012697+00:00 [info] <0.222.0> : 2025-09-11 01:56:00.012697+00:00 [info] <0.222.0> database dir : /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4079-3 2025-09-11 01:56:00.189012+00:00 [info] <0.222.0> Feature flags: list of feature flags found: 2025-09-11 01:56:00.189110+00:00 [info] <0.222.0> Feature flags: [x] classic_mirrored_queue_version 2025-09-11 01:56:00.189146+00:00 [info] <0.222.0> Feature flags: [x] drop_unroutable_metric 2025-09-11 01:56:00.189179+00:00 [info] <0.222.0> Feature flags: [x] empty_basic_get_metric 2025-09-11 01:56:00.189197+00:00 [info] <0.222.0> Feature flags: [x] implicit_default_bindings 2025-09-11 01:56:00.189214+00:00 [info] <0.222.0> Feature flags: [ ] maintenance_mode_status 2025-09-11 01:56:00.189268+00:00 [info] <0.222.0> Feature flags: [x] quorum_queue 2025-09-11 01:56:00.189291+00:00 [info] <0.222.0> Feature flags: [x] stream_queue 2025-09-11 01:56:00.189310+00:00 [info] <0.222.0> Feature flags: [x] user_limits 2025-09-11 01:56:00.189340+00:00 [info] <0.222.0> Feature flags: [x] virtual_host_metadata 2025-09-11 01:56:00.189376+00:00 [info] <0.222.0> Feature flags: feature flag states written to disk: yes 2025-09-11 01:56:01.896455+00:00 [info] <0.222.0> Running boot step pre_boot defined by app rabbit 2025-09-11 01:56:01.896539+00:00 [info] <0.222.0> Running boot step rabbit_global_counters defined by app rabbit 2025-09-11 01:56:01.896953+00:00 [info] <0.222.0> Running boot step rabbit_osiris_metrics defined by app rabbit 2025-09-11 01:56:01.897128+00:00 [info] <0.222.0> Running boot step rabbit_core_metrics defined by app rabbit 2025-09-11 01:56:01.897874+00:00 [info] <0.222.0> Running boot step rabbit_alarm defined by app rabbit 2025-09-11 01:56:01.910079+00:00 [info] <0.421.0> Memory high watermark set to 12637 MiB (13251084288 bytes) of 15796 MiB (16563855360 bytes) total 2025-09-11 01:56:01.913446+00:00 [info] <0.423.0> Enabling free disk space monitoring 2025-09-11 01:56:01.913506+00:00 [info] <0.423.0> Disk free limit set to 50MB 2025-09-11 01:56:01.915917+00:00 [info] <0.222.0> Running boot step code_server_cache defined by app rabbit 2025-09-11 01:56:01.916036+00:00 [info] <0.222.0> Running boot step file_handle_cache defined by app rabbit 2025-09-11 01:56:01.916237+00:00 [info] <0.428.0> Limiting to approx 1048479 file handles (943629 sockets) 2025-09-11 01:56:01.916349+00:00 [info] <0.429.0> FHC read buffering: OFF 2025-09-11 01:56:01.916383+00:00 [info] <0.429.0> FHC write buffering: ON 2025-09-11 01:56:01.916649+00:00 [info] <0.222.0> Running boot step worker_pool defined by app rabbit 2025-09-11 01:56:01.916705+00:00 [info] <0.403.0> Will use 4 processes for default worker pool 2025-09-11 01:56:01.916739+00:00 [info] <0.403.0> Starting worker pool 'worker_pool' with 4 processes in it 2025-09-11 01:56:01.917006+00:00 [info] <0.222.0> Running boot step database defined by app rabbit 2025-09-11 01:56:01.922494+00:00 [info] <0.222.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-09-11 01:56:01.926342+00:00 [info] <0.222.0> Successfully synced tables from a peer 2025-09-11 01:56:01.926867+00:00 [info] <0.222.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-09-11 01:56:01.926990+00:00 [info] <0.222.0> Successfully synced tables from a peer 2025-09-11 01:56:01.927034+00:00 [warning] <0.222.0> Feature flags: the previous instance of this node must have failed to write the `feature_flags` file at `/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4079-3-feature_flags`: 2025-09-11 01:56:01.927080+00:00 [warning] <0.222.0> Feature flags: - list of previously enabled feature flags now marked as such: [maintenance_mode_status] 2025-09-11 01:56:01.928646+00:00 [info] <0.222.0> Feature flags: list of feature flags found: 2025-09-11 01:56:01.928699+00:00 [info] <0.222.0> Feature flags: [x] classic_mirrored_queue_version 2025-09-11 01:56:01.928727+00:00 [info] <0.222.0> Feature flags: [x] drop_unroutable_metric 2025-09-11 01:56:01.928752+00:00 [info] <0.222.0> Feature flags: [x] empty_basic_get_metric 2025-09-11 01:56:01.928813+00:00 [info] <0.222.0> Feature flags: [x] implicit_default_bindings 2025-09-11 01:56:01.928849+00:00 [info] <0.222.0> Feature flags: [x] maintenance_mode_status 2025-09-11 01:56:01.928866+00:00 [info] <0.222.0> Feature flags: [x] quorum_queue 2025-09-11 01:56:01.928916+00:00 [info] <0.222.0> Feature flags: [x] stream_queue 2025-09-11 01:56:01.928955+00:00 [info] <0.222.0> Feature flags: [x] user_limits 2025-09-11 01:56:01.928978+00:00 [info] <0.222.0> Feature flags: [x] virtual_host_metadata 2025-09-11 01:56:01.929005+00:00 [info] <0.222.0> Feature flags: feature flag states written to disk: yes 2025-09-11 01:56:01.937888+00:00 [info] <0.222.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-09-11 01:56:01.938100+00:00 [info] <0.222.0> Successfully synced tables from a peer 2025-09-11 01:56:01.957349+00:00 [info] <0.222.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-09-11 01:56:01.957524+00:00 [info] <0.222.0> Successfully synced tables from a peer 2025-09-11 01:56:01.957604+00:00 [info] <0.222.0> Peer discovery backend rabbit_peer_discovery_classic_config does not support registration, skipping registration. 2025-09-11 01:56:01.957780+00:00 [info] <0.222.0> Running boot step database_sync defined by app rabbit 2025-09-11 01:56:01.957868+00:00 [info] <0.222.0> Running boot step feature_flags defined by app rabbit 2025-09-11 01:56:01.958009+00:00 [info] <0.222.0> Running boot step codec_correctness_check defined by app rabbit 2025-09-11 01:56:01.958063+00:00 [info] <0.222.0> Running boot step external_infrastructure defined by app rabbit 2025-09-11 01:56:01.958093+00:00 [info] <0.222.0> Running boot step rabbit_event defined by app rabbit 2025-09-11 01:56:01.958213+00:00 [info] <0.222.0> Running boot step rabbit_registry defined by app rabbit 2025-09-11 01:56:01.958342+00:00 [info] <0.222.0> Running boot step rabbit_auth_mechanism_amqplain defined by app rabbit 2025-09-11 01:56:01.958402+00:00 [info] <0.222.0> Running boot step rabbit_auth_mechanism_cr_demo defined by app rabbit 2025-09-11 01:56:01.958440+00:00 [info] <0.222.0> Running boot step rabbit_auth_mechanism_plain defined by app rabbit 2025-09-11 01:56:01.958505+00:00 [info] <0.222.0> Running boot step rabbit_exchange_type_direct defined by app rabbit 2025-09-11 01:56:01.958577+00:00 [info] <0.222.0> Running boot step rabbit_exchange_type_fanout defined by app rabbit 2025-09-11 01:56:01.958624+00:00 [info] <0.222.0> Running boot step rabbit_exchange_type_headers defined by app rabbit 2025-09-11 01:56:01.958671+00:00 [info] <0.222.0> Running boot step rabbit_exchange_type_topic defined by app rabbit 2025-09-11 01:56:01.958705+00:00 [info] <0.222.0> Running boot step rabbit_mirror_queue_mode_all defined by app rabbit 2025-09-11 01:56:01.958742+00:00 [info] <0.222.0> Running boot step rabbit_mirror_queue_mode_exactly defined by app rabbit 2025-09-11 01:56:01.958779+00:00 [info] <0.222.0> Running boot step rabbit_mirror_queue_mode_nodes defined by app rabbit 2025-09-11 01:56:01.958835+00:00 [info] <0.222.0> Running boot step rabbit_priority_queue defined by app rabbit 2025-09-11 01:56:01.958869+00:00 [info] <0.222.0> Priority queues enabled, real BQ is rabbit_variable_queue 2025-09-11 01:56:01.958941+00:00 [info] <0.222.0> Running boot step rabbit_queue_location_client_local defined by app rabbit 2025-09-11 01:56:01.959010+00:00 [info] <0.222.0> Running boot step rabbit_queue_location_min_masters defined by app rabbit 2025-09-11 01:56:01.959043+00:00 [info] <0.222.0> Running boot step rabbit_queue_location_random defined by app rabbit 2025-09-11 01:56:01.959083+00:00 [info] <0.222.0> Running boot step kernel_ready defined by app rabbit 2025-09-11 01:56:01.959107+00:00 [info] <0.222.0> Running boot step rabbit_sysmon_minder defined by app rabbit 2025-09-11 01:56:01.959178+00:00 [info] <0.222.0> Running boot step rabbit_epmd_monitor defined by app rabbit 2025-09-11 01:56:01.959945+00:00 [info] <0.454.0> epmd monitor knows us, inter-node communication (distribution) port: 25673 2025-09-11 01:56:01.960030+00:00 [info] <0.222.0> Running boot step guid_generator defined by app rabbit 2025-09-11 01:56:01.963558+00:00 [info] <0.222.0> Running boot step rabbit_node_monitor defined by app rabbit 2025-09-11 01:56:01.967725+00:00 [info] <0.458.0> Starting rabbit_node_monitor 2025-09-11 01:56:01.967849+00:00 [info] <0.222.0> Running boot step delegate_sup defined by app rabbit 2025-09-11 01:56:01.968280+00:00 [info] <0.222.0> Running boot step rabbit_memory_monitor defined by app rabbit 2025-09-11 01:56:01.968478+00:00 [info] <0.222.0> Running boot step rabbit_fifo_dlx_sup defined by app rabbit 2025-09-11 01:56:01.968562+00:00 [info] <0.222.0> Running boot step core_initialized defined by app rabbit 2025-09-11 01:56:01.968587+00:00 [info] <0.222.0> Running boot step upgrade_queues defined by app rabbit 2025-09-11 01:56:01.974843+00:00 [info] <0.222.0> message_store upgrades: 1 to apply 2025-09-11 01:56:01.974982+00:00 [info] <0.222.0> message_store upgrades: Applying rabbit_variable_queue:move_messages_to_vhost_store 2025-09-11 01:56:01.975116+00:00 [info] <0.222.0> message_store upgrades: No durable queues found. Skipping message store migration 2025-09-11 01:56:01.975144+00:00 [info] <0.222.0> message_store upgrades: Removing the old message store data 2025-09-11 01:56:01.975493+00:00 [info] <0.222.0> message_store upgrades: All upgrades applied successfully 2025-09-11 01:56:01.986645+00:00 [info] <0.222.0> Running boot step channel_tracking defined by app rabbit 2025-09-11 01:56:01.996034+00:00 [info] <0.222.0> Setting up a table for channel tracking on this node: 'tracked_channel_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-4079-3' 2025-09-11 01:56:01.996640+00:00 [info] <0.222.0> Setting up a table for channel tracking on this node: 'tracked_channel_table_per_user_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-4079-3' 2025-09-11 01:56:01.996855+00:00 [info] <0.222.0> Running boot step rabbit_channel_tracking_handler defined by app rabbit 2025-09-11 01:56:01.996933+00:00 [info] <0.222.0> Running boot step connection_tracking defined by app rabbit 2025-09-11 01:56:01.997476+00:00 [info] <0.222.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-4079-3' 2025-09-11 01:56:01.997989+00:00 [info] <0.222.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-4079-3' 2025-09-11 01:56:01.998452+00:00 [info] <0.222.0> Setting up a table for per-user connection counting on this node: 'tracked_connection_table_per_user_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-4079-3' 2025-09-11 01:56:01.998631+00:00 [info] <0.222.0> Running boot step rabbit_connection_tracking_handler defined by app rabbit 2025-09-11 01:56:01.998685+00:00 [info] <0.222.0> Running boot step rabbit_definitions_hashing defined by app rabbit 2025-09-11 01:56:01.998792+00:00 [info] <0.222.0> Running boot step rabbit_exchange_parameters defined by app rabbit 2025-09-11 01:56:01.998912+00:00 [info] <0.222.0> Running boot step rabbit_mirror_queue_misc defined by app rabbit 2025-09-11 01:56:01.999057+00:00 [info] <0.222.0> Running boot step rabbit_policies defined by app rabbit 2025-09-11 01:56:01.999381+00:00 [info] <0.222.0> Running boot step rabbit_policy defined by app rabbit 2025-09-11 01:56:01.999439+00:00 [info] <0.222.0> Running boot step rabbit_queue_location_validator defined by app rabbit 2025-09-11 01:56:01.999489+00:00 [info] <0.222.0> Running boot step rabbit_quorum_memory_manager defined by app rabbit 2025-09-11 01:56:01.999538+00:00 [info] <0.222.0> Running boot step rabbit_stream_coordinator defined by app rabbit 2025-09-11 01:56:01.999647+00:00 [info] <0.222.0> Running boot step rabbit_vhost_limit defined by app rabbit 2025-09-11 01:56:01.999719+00:00 [info] <0.222.0> Running boot step rabbit_mgmt_reset_handler defined by app rabbitmq_management 2025-09-11 01:56:01.999756+00:00 [info] <0.222.0> Running boot step rabbit_mgmt_db_handler defined by app rabbitmq_management_agent 2025-09-11 01:56:01.999784+00:00 [warning] <0.222.0> force_fine_statistics set to true; ignored. 2025-09-11 01:56:01.999784+00:00 [warning] <0.222.0> Replaced by {rates_mode, none} in the rabbitmq_management application. 2025-09-11 01:56:01.999816+00:00 [info] <0.222.0> Management plugin: using rates mode 'basic' 2025-09-11 01:56:02.000969+00:00 [info] <0.222.0> Running boot step recovery defined by app rabbit 2025-09-11 01:56:02.002250+00:00 [info] <0.501.0> Making sure data directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4079-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists 2025-09-11 01:56:02.002948+00:00 [info] <0.501.0> Setting segment_entry_count for vhost '/' with 0 queues to '2048' 2025-09-11 01:56:02.007118+00:00 [info] <0.501.0> Starting message stores for vhost '/' 2025-09-11 01:56:02.007390+00:00 [info] <0.506.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index 2025-09-11 01:56:02.009695+00:00 [info] <0.501.0> Started message store of type transient for vhost '/' 2025-09-11 01:56:02.009866+00:00 [info] <0.510.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index 2025-09-11 01:56:02.010360+00:00 [warning] <0.510.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": rebuilding indices from scratch 2025-09-11 01:56:02.010978+00:00 [info] <0.501.0> Started message store of type persistent for vhost '/' 2025-09-11 01:56:02.011085+00:00 [info] <0.501.0> Recovering 0 queues of type rabbit_classic_queue took 7ms 2025-09-11 01:56:02.011161+00:00 [info] <0.501.0> Recovering 0 queues of type rabbit_quorum_queue took 0ms 2025-09-11 01:56:02.011190+00:00 [info] <0.501.0> Recovering 0 queues of type rabbit_stream_queue took 0ms 2025-09-11 01:56:02.012249+00:00 [info] <0.501.0> Mirrored queue 'job_status_consumer.dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-3': <0.530.0> 2025-09-11 01:56:02.012437+00:00 [info] <0.501.0> Mirrored queue 'device_manager_ztp.ztp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-3': <0.534.0> 2025-09-11 01:56:02.012603+00:00 [info] <0.501.0> Mirrored queue 'job_request_consumer' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-3': <0.538.0> 2025-09-11 01:56:02.012784+00:00 [info] <0.501.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-4079-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-3': <0.542.0> 2025-09-11 01:56:02.013009+00:00 [info] <0.501.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-4079-3..tftp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-3': <0.546.0> 2025-09-11 01:56:02.013280+00:00 [info] <0.501.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-4079-3..config_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-3': <0.550.0> 2025-09-11 01:56:02.013373+00:00 [info] <0.222.0> Running boot step empty_db_check defined by app rabbit 2025-09-11 01:56:02.013422+00:00 [info] <0.222.0> Will not seed default virtual host and user: have definitions to load... 2025-09-11 01:56:02.013444+00:00 [info] <0.222.0> Running boot step rabbit_looking_glass defined by app rabbit 2025-09-11 01:56:02.013469+00:00 [info] <0.222.0> Running boot step rabbit_core_metrics_gc defined by app rabbit 2025-09-11 01:56:02.013581+00:00 [info] <0.222.0> Running boot step background_gc defined by app rabbit 2025-09-11 01:56:02.013689+00:00 [info] <0.222.0> Running boot step routing_ready defined by app rabbit 2025-09-11 01:56:02.013727+00:00 [info] <0.222.0> Running boot step pre_flight defined by app rabbit 2025-09-11 01:56:02.013748+00:00 [info] <0.222.0> Running boot step notify_cluster defined by app rabbit 2025-09-11 01:56:02.013769+00:00 [info] <0.222.0> Running boot step networking defined by app rabbit 2025-09-11 01:56:02.013790+00:00 [info] <0.222.0> Running boot step definition_import_worker_pool defined by app rabbit 2025-09-11 01:56:02.013865+00:00 [info] <0.403.0> Starting worker pool 'definition_import_pool' with 4 processes in it 2025-09-11 01:56:02.014452+00:00 [info] <0.458.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-1' up 2025-09-11 01:56:02.014452+00:00 [info] <0.222.0> Running boot step cluster_name defined by app rabbit 2025-09-11 01:56:02.014513+00:00 [info] <0.222.0> Running boot step direct_client defined by app rabbit 2025-09-11 01:56:02.014682+00:00 [info] <0.222.0> Running boot step rabbit_maintenance_mode_state defined by app rabbit 2025-09-11 01:56:02.014730+00:00 [info] <0.222.0> Creating table rabbit_node_maintenance_states for feature flag `maintenance_mode_status` 2025-09-11 01:56:02.016998+00:00 [info] <0.222.0> Running boot step rabbit_management_load_definitions defined by app rabbitmq_management 2025-09-11 01:56:02.017439+00:00 [info] <0.222.0> Applying definitions from file at '/etc/rabbitmq/definitions.json' 2025-09-11 01:56:02.017478+00:00 [info] <0.222.0> Asked to import definitions. Acting user: rmq-internal 2025-09-11 01:56:02.017569+00:00 [info] <0.222.0> Importing concurrently 1 users... 2025-09-11 01:56:02.022385+00:00 [info] <0.222.0> Importing concurrently 1 vhosts... 2025-09-11 01:56:02.035961+00:00 [info] <0.458.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-2' up 2025-09-11 01:56:02.036069+00:00 [info] <0.222.0> Importing concurrently 1 permissions... 2025-09-11 01:56:02.039402+00:00 [info] <0.558.0> Successfully set permissions for 'guest' in virtual host '/' to '.*', '.*', '.*' 2025-09-11 01:56:02.039551+00:00 [info] <0.222.0> Importing sequentially 1 policies... 2025-09-11 01:56:02.043547+00:00 [info] <0.597.0> Resetting node maintenance status 2025-09-11 01:56:02.067957+00:00 [info] <0.656.0> Management plugin: HTTP (non-TLS) listener started on port 15673 2025-09-11 01:56:02.068139+00:00 [info] <0.684.0> Statistics database started. 2025-09-11 01:56:02.068200+00:00 [info] <0.683.0> Starting worker pool 'management_worker_pool' with 3 processes in it 2025-09-11 01:56:02.075575+00:00 [info] <0.698.0> Prometheus metrics: HTTP (non-TLS) listener started on port 15692 2025-09-11 01:56:02.075711+00:00 [info] <0.597.0> Ready to start client connection listeners 2025-09-11 01:56:02.076748+00:00 [info] <0.742.0> started TCP listener on 10.0.0.49:5673 2025-09-11 01:56:02.137754+00:00 [info] <0.597.0> Server startup complete; 4 plugins started. 2025-09-11 01:56:02.137754+00:00 [info] <0.597.0> * rabbitmq_prometheus 2025-09-11 01:56:02.137754+00:00 [info] <0.597.0> * rabbitmq_management 2025-09-11 01:56:02.137754+00:00 [info] <0.597.0> * rabbitmq_web_dispatch 2025-09-11 01:56:02.137754+00:00 [info] <0.597.0> * rabbitmq_management_agent 2025-09-11 01:56:47.167838+00:00 [info] <0.1061.0> RabbitMQ is asked to stop... 2025-09-11 01:56:47.203813+00:00 [info] <0.1061.0> Stopping RabbitMQ applications and their dependencies in the following order: 2025-09-11 01:56:47.203813+00:00 [info] <0.1061.0> rabbitmq_management 2025-09-11 01:56:47.203813+00:00 [info] <0.1061.0> rabbitmq_prometheus 2025-09-11 01:56:47.203813+00:00 [info] <0.1061.0> rabbitmq_web_dispatch 2025-09-11 01:56:47.203813+00:00 [info] <0.1061.0> cowboy 2025-09-11 01:56:47.203813+00:00 [info] <0.1061.0> cowlib 2025-09-11 01:56:47.203813+00:00 [info] <0.1061.0> rabbitmq_management_agent 2025-09-11 01:56:47.203813+00:00 [info] <0.1061.0> rabbit 2025-09-11 01:56:47.203813+00:00 [info] <0.1061.0> amqp_client 2025-09-11 01:56:47.203813+00:00 [info] <0.1061.0> rabbitmq_prelaunch 2025-09-11 01:56:47.203813+00:00 [info] <0.1061.0> rabbit_common 2025-09-11 01:56:47.203813+00:00 [info] <0.1061.0> prometheus 2025-09-11 01:56:47.203813+00:00 [info] <0.1061.0> ra 2025-09-11 01:56:47.203813+00:00 [info] <0.1061.0> accept 2025-09-11 01:56:47.203813+00:00 [info] <0.1061.0> os_mon 2025-09-11 01:56:47.203813+00:00 [info] <0.1061.0> sysmon_handler 2025-09-11 01:56:47.203813+00:00 [info] <0.1061.0> osiris 2025-09-11 01:56:47.203813+00:00 [info] <0.1061.0> mnesia 2025-09-11 01:56:47.203813+00:00 [info] <0.1061.0> 2025-09-11 01:56:47.203970+00:00 [info] <0.1061.0> Stopping application 'rabbitmq_management' 2025-09-11 01:56:47.205616+00:00 [warning] <0.648.0> HTTP listener registry could not find context rabbitmq_management_tls 2025-09-11 01:56:47.207196+00:00 [notice] <0.44.0> Application rabbitmq_management exited with reason: stopped 2025-09-11 01:56:47.207308+00:00 [info] <0.1061.0> Stopping application 'rabbitmq_prometheus' 2025-09-11 01:56:47.208230+00:00 [warning] <0.648.0> HTTP listener registry could not find context rabbitmq_prometheus_tls 2025-09-11 01:56:47.209736+00:00 [notice] <0.44.0> Application rabbitmq_prometheus exited with reason: stopped 2025-09-11 01:56:47.209820+00:00 [info] <0.1061.0> Stopping application 'rabbitmq_web_dispatch' 2025-09-11 01:56:47.211584+00:00 [notice] <0.44.0> Application rabbitmq_web_dispatch exited with reason: stopped 2025-09-11 01:56:47.211678+00:00 [info] <0.1061.0> Stopping application 'cowboy' 2025-09-11 01:56:47.213165+00:00 [notice] <0.44.0> Application cowboy exited with reason: stopped 2025-09-11 01:56:47.213247+00:00 [info] <0.1061.0> Stopping application 'cowlib' 2025-09-11 01:56:47.213301+00:00 [notice] <0.44.0> Application cowlib exited with reason: stopped 2025-09-11 01:56:47.213341+00:00 [info] <0.1061.0> Stopping application 'rabbitmq_management_agent' 2025-09-11 01:56:47.215146+00:00 [notice] <0.44.0> Application rabbitmq_management_agent exited with reason: stopped 2025-09-11 01:56:47.215244+00:00 [info] <0.1061.0> Stopping application 'rabbit' 2025-09-11 01:56:47.215351+00:00 [info] <0.222.0> Peer discovery backend rabbit_peer_discovery_classic_config does not support registration, skipping unregistration. 2025-09-11 01:56:47.215462+00:00 [info] <0.742.0> stopped TCP listener on 10.0.0.49:5673 2025-09-11 01:56:47.216541+00:00 [info] <0.1062.0> Closing all connections in vhost '/' on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-3' because the vhost is stopping 2025-09-11 01:56:47.217254+00:00 [info] <0.510.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4079-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' 2025-09-11 01:56:47.223325+00:00 [info] <0.510.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4079-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' is stopped 2025-09-11 01:56:47.223487+00:00 [info] <0.506.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4079-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' 2025-09-11 01:56:47.227475+00:00 [info] <0.506.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4079-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' is stopped 2025-09-11 01:56:47.231806+00:00 [info] <0.449.0> Management plugin: to stop collect_statistics. 2025-09-11 01:56:52.235909+00:00 [notice] <0.44.0> Application rabbit exited with reason: stopped 2025-09-11 01:56:52.236106+00:00 [info] <0.1061.0> Stopping application 'amqp_client' 2025-09-11 01:56:52.238311+00:00 [notice] <0.44.0> Application amqp_client exited with reason: stopped 2025-09-11 01:56:52.238424+00:00 [info] <0.1061.0> Stopping application 'rabbitmq_prelaunch' 2025-09-11 01:56:52.240676+00:00 [notice] <0.44.0> Application rabbitmq_prelaunch exited with reason: stopped 2025-09-11 01:56:52.240799+00:00 [info] <0.1061.0> Stopping application 'rabbit_common' 2025-09-11 01:56:52.240880+00:00 [notice] <0.44.0> Application rabbit_common exited with reason: stopped 2025-09-11 01:56:52.240921+00:00 [info] <0.1061.0> Stopping application 'prometheus' 2025-09-11 01:56:52.243061+00:00 [notice] <0.44.0> Application prometheus exited with reason: stopped 2025-09-11 01:56:52.243146+00:00 [info] <0.1061.0> Stopping application 'ra' 2025-09-11 01:56:52.245940+00:00 [notice] <0.44.0> Application ra exited with reason: stopped 2025-09-11 01:56:52.246031+00:00 [info] <0.1061.0> Stopping application 'accept' 2025-09-11 01:56:52.246063+00:00 [notice] <0.44.0> Application accept exited with reason: stopped 2025-09-11 01:56:52.246086+00:00 [info] <0.1061.0> Stopping application 'os_mon' 2025-09-11 01:56:52.247538+00:00 [notice] <0.44.0> Application os_mon exited with reason: stopped 2025-09-11 01:56:52.247595+00:00 [info] <0.1061.0> Stopping application 'sysmon_handler' 2025-09-11 01:56:52.249069+00:00 [notice] <0.44.0> Application sysmon_handler exited with reason: stopped 2025-09-11 01:56:52.249189+00:00 [info] <0.1061.0> Stopping application 'osiris' 2025-09-11 01:56:52.250797+00:00 [notice] <0.44.0> Application osiris exited with reason: stopped 2025-09-11 01:56:52.250854+00:00 [info] <0.1061.0> Stopping application 'mnesia' 2025-09-11 01:56:52.256655+00:00 [notice] <0.44.0> Application mnesia exited with reason: stopped 2025-09-11 01:56:52.256742+00:00 [info] <0.1061.0> Successfully stopped RabbitMQ and its dependencies 2025-09-11 01:56:58.674926+00:00 [info] <0.1153.0> Resetting Rabbit forcefully 2025-09-11 01:56:59.166101+00:00 [notice] <0.44.0> Application mnesia exited with reason: stopped 2025-09-11 01:56:59.175169+00:00 [notice] <0.44.0> Application mnesia exited with reason: stopped 2025-09-11 01:56:59.269924+00:00 [info] <0.1159.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-09-11 01:56:59.270062+00:00 [info] <0.1159.0> Successfully synced tables from a peer 2025-09-11 01:56:59.279691+00:00 [info] <0.1159.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-09-11 01:56:59.279801+00:00 [info] <0.1159.0> Successfully synced tables from a peer 2025-09-11 01:56:59.279841+00:00 [warning] <0.1159.0> Feature flags: the previous instance of this node must have failed to write the `feature_flags` file at `/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4079-3-feature_flags`: 2025-09-11 01:56:59.279882+00:00 [warning] <0.1159.0> Feature flags: - list of previously disabled feature flags now marked as such: [maintenance_mode_status] 2025-09-11 01:56:59.285047+00:00 [info] <0.1159.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-09-11 01:56:59.285208+00:00 [info] <0.1159.0> Successfully synced tables from a peer 2025-09-11 01:56:59.298098+00:00 [info] <0.1159.0> Waiting for Mnesia tables for 30000 ms, 0 retries left 2025-09-11 01:56:59.298227+00:00 [info] <0.1159.0> Successfully synced tables from a peer 2025-09-11 01:56:59.304502+00:00 [notice] <0.44.0> Application mnesia exited with reason: stopped 2025-09-11 01:56:59.319565+00:00 [info] <0.1159.0> Clustering with ['contrail@cn-jenkins-deploy-platform-ansible-os-4079-2', 2025-09-11 01:56:59.319565+00:00 [info] <0.1159.0> 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-1'] as disc node 2025-09-11 01:56:59.346769+00:00 [info] <0.1159.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-09-11 01:56:59.346938+00:00 [info] <0.1159.0> Successfully synced tables from a peer 2025-09-11 01:56:59.650443+00:00 [info] <0.1159.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-09-11 01:56:59.650799+00:00 [info] <0.1159.0> Successfully synced tables from a peer 2025-09-11 01:56:59.650857+00:00 [warning] <0.1159.0> Feature flags: the previous instance of this node must have failed to write the `feature_flags` file at `/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4079-3-feature_flags`: 2025-09-11 01:56:59.650906+00:00 [warning] <0.1159.0> Feature flags: - list of previously enabled feature flags now marked as such: [maintenance_mode_status] 2025-09-11 01:56:59.653700+00:00 [info] <0.1159.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-09-11 01:56:59.653911+00:00 [info] <0.1159.0> Successfully synced tables from a peer 2025-09-11 01:56:59.669829+00:00 [info] <0.1159.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-09-11 01:56:59.670134+00:00 [info] <0.1159.0> Successfully synced tables from a peer 2025-09-11 01:56:59.675336+00:00 [notice] <0.44.0> Application mnesia exited with reason: stopped 2025-09-11 01:56:59.675611+00:00 [error] <0.1159.0> Failed to create a tracked connection table for node 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-3': {node_not_running, 2025-09-11 01:56:59.675611+00:00 [error] <0.1159.0> 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-3'} 2025-09-11 01:56:59.676309+00:00 [info] <0.1159.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-4079-3' 2025-09-11 01:56:59.676475+00:00 [error] <0.1159.0> Failed to create a per-vhost tracked connection table for node 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-3': {node_not_running, 2025-09-11 01:56:59.676475+00:00 [error] <0.1159.0> 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-3'} 2025-09-11 01:56:59.676595+00:00 [info] <0.1159.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-4079-3' 2025-09-11 01:56:59.676696+00:00 [error] <0.1159.0> Failed to create a per-user tracked connection table for node 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-3': {node_not_running, 2025-09-11 01:56:59.676696+00:00 [error] <0.1159.0> 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-3'} 2025-09-11 01:56:59.676827+00:00 [info] <0.1159.0> Setting up a table for per-user connection counting on this node: 'tracked_connection_table_per_user_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-4079-3' 2025-09-11 01:57:00.154777+00:00 [info] <0.1522.0> Halting Erlang VM with the following applications: 2025-09-11 01:57:00.154777+00:00 [info] <0.1522.0> amqp10_common 2025-09-11 01:57:00.154777+00:00 [info] <0.1522.0> observer_cli 2025-09-11 01:57:00.154777+00:00 [info] <0.1522.0> stdout_formatter 2025-09-11 01:57:00.154777+00:00 [info] <0.1522.0> seshat 2025-09-11 01:57:00.154777+00:00 [info] <0.1522.0> gen_batch_server 2025-09-11 01:57:00.154777+00:00 [info] <0.1522.0> aten 2025-09-11 01:57:00.154777+00:00 [info] <0.1522.0> ranch 2025-09-11 01:57:00.154777+00:00 [info] <0.1522.0> inets 2025-09-11 01:57:00.154777+00:00 [info] <0.1522.0> systemd 2025-09-11 01:57:00.154777+00:00 [info] <0.1522.0> enough 2025-09-11 01:57:00.154777+00:00 [info] <0.1522.0> cuttlefish 2025-09-11 01:57:00.154777+00:00 [info] <0.1522.0> credentials_obfuscation 2025-09-11 01:57:00.154777+00:00 [info] <0.1522.0> recon 2025-09-11 01:57:00.154777+00:00 [info] <0.1522.0> jsx 2025-09-11 01:57:00.154777+00:00 [info] <0.1522.0> xmerl 2025-09-11 01:57:00.154777+00:00 [info] <0.1522.0> tools 2025-09-11 01:57:00.154777+00:00 [info] <0.1522.0> syntax_tools 2025-09-11 01:57:00.154777+00:00 [info] <0.1522.0> ssl 2025-09-11 01:57:00.154777+00:00 [info] <0.1522.0> public_key 2025-09-11 01:57:00.154777+00:00 [info] <0.1522.0> asn1 2025-09-11 01:57:00.154777+00:00 [info] <0.1522.0> crypto 2025-09-11 01:57:00.154777+00:00 [info] <0.1522.0> compiler 2025-09-11 01:57:00.154777+00:00 [info] <0.1522.0> sasl 2025-09-11 01:57:00.154777+00:00 [info] <0.1522.0> stdlib 2025-09-11 01:57:00.154777+00:00 [info] <0.1522.0> kernel 2025-09-11 01:57:00.154777+00:00 [info] <0.1522.0> 2025-09-11 01:57:04.113114+00:00 [notice] <0.222.0> Logging: configured log handlers are now ACTIVE 2025-09-11 01:57:05.608721+00:00 [info] <0.222.0> upgrades: Backing up mnesia dir to "/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4079-3-upgrade-backup" 2025-09-11 01:57:05.612368+00:00 [info] <0.222.0> upgrades: Mnesia dir backed up to "/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4079-3-upgrade-backup" 2025-09-11 01:57:05.618808+00:00 [notice] <0.44.0> Application mnesia exited with reason: stopped 2025-09-11 01:57:05.620442+00:00 [info] <0.222.0> Feature flags: list of feature flags found: 2025-09-11 01:57:05.620491+00:00 [info] <0.222.0> Feature flags: [x] classic_mirrored_queue_version 2025-09-11 01:57:05.620530+00:00 [info] <0.222.0> Feature flags: [x] drop_unroutable_metric 2025-09-11 01:57:05.620548+00:00 [info] <0.222.0> Feature flags: [x] empty_basic_get_metric 2025-09-11 01:57:05.620564+00:00 [info] <0.222.0> Feature flags: [x] implicit_default_bindings 2025-09-11 01:57:05.620612+00:00 [info] <0.222.0> Feature flags: [x] maintenance_mode_status 2025-09-11 01:57:05.620633+00:00 [info] <0.222.0> Feature flags: [x] quorum_queue 2025-09-11 01:57:05.620648+00:00 [info] <0.222.0> Feature flags: [x] stream_queue 2025-09-11 01:57:05.620665+00:00 [info] <0.222.0> Feature flags: [x] user_limits 2025-09-11 01:57:05.620718+00:00 [info] <0.222.0> Feature flags: [x] virtual_host_metadata 2025-09-11 01:57:05.620738+00:00 [info] <0.222.0> Feature flags: feature flag states written to disk: yes 2025-09-11 01:57:05.664982+00:00 [info] <0.222.0> ra: starting system quorum_queues 2025-09-11 01:57:05.665058+00:00 [info] <0.222.0> starting Ra system: quorum_queues in directory: /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4079-3/quorum/contrail@cn-jenkins-deploy-platform-ansible-os-4079-3 2025-09-11 01:57:05.666062+00:00 [info] <0.313.0> ra system 'quorum_queues' running pre init for 0 registered servers 2025-09-11 01:57:05.667802+00:00 [info] <0.314.0> ra: meta data store initialised for system quorum_queues. 0 record(s) recovered 2025-09-11 01:57:05.667973+00:00 [notice] <0.319.0> WAL: ra_log_wal init, open tbls: ra_log_open_mem_tables, closed tbls: ra_log_closed_mem_tables 2025-09-11 01:57:05.671161+00:00 [info] <0.222.0> ra: starting system coordination 2025-09-11 01:57:05.671210+00:00 [info] <0.222.0> starting Ra system: coordination in directory: /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4079-3/coordination/contrail@cn-jenkins-deploy-platform-ansible-os-4079-3 2025-09-11 01:57:05.671906+00:00 [info] <0.330.0> ra system 'coordination' running pre init for 0 registered servers 2025-09-11 01:57:05.672345+00:00 [info] <0.331.0> ra: meta data store initialised for system coordination. 0 record(s) recovered 2025-09-11 01:57:05.672538+00:00 [notice] <0.336.0> WAL: ra_coordination_log_wal init, open tbls: ra_coordination_log_open_mem_tables, closed tbls: ra_coordination_log_closed_mem_tables 2025-09-11 01:57:05.676431+00:00 [info] <0.222.0> 2025-09-11 01:57:05.676431+00:00 [info] <0.222.0> Starting RabbitMQ 3.10.7 on Erlang 25.0.4 [jit] 2025-09-11 01:57:05.676431+00:00 [info] <0.222.0> Copyright (c) 2007-2022 VMware, Inc. or its affiliates. 2025-09-11 01:57:05.676431+00:00 [info] <0.222.0> Licensed under the MPL 2.0. Website: https://rabbitmq.com 2025-09-11 01:57:05.677821+00:00 [info] <0.222.0> 2025-09-11 01:57:05.677821+00:00 [info] <0.222.0> node : contrail@cn-jenkins-deploy-platform-ansible-os-4079-3 2025-09-11 01:57:05.677821+00:00 [info] <0.222.0> home dir : /var/lib/rabbitmq 2025-09-11 01:57:05.677821+00:00 [info] <0.222.0> config file(s) : /etc/rabbitmq/rabbitmq.config 2025-09-11 01:57:05.677821+00:00 [info] <0.222.0> : /etc/rabbitmq/conf.d/10-defaults.conf 2025-09-11 01:57:05.677821+00:00 [info] <0.222.0> cookie hash : esZsDxSN6VGbi9JkMSxNZA== 2025-09-11 01:57:05.677821+00:00 [info] <0.222.0> log(s) : /var/log/contrail/config-database-rabbitmq/rabbitmq.log 2025-09-11 01:57:05.677821+00:00 [info] <0.222.0> : /var/log/rabbitmq/contrail@cn-jenkins-deploy-platform-ansible-os-4079-3_upgrade.log 2025-09-11 01:57:05.677821+00:00 [info] <0.222.0> : 2025-09-11 01:57:05.677821+00:00 [info] <0.222.0> database dir : /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4079-3 2025-09-11 01:57:05.865345+00:00 [info] <0.222.0> Feature flag `drop_unroutable_metric`: mark as enabled=true 2025-09-11 01:57:05.866819+00:00 [info] <0.222.0> Feature flag `empty_basic_get_metric`: mark as enabled=true 2025-09-11 01:57:06.035604+00:00 [info] <0.222.0> Running boot step pre_boot defined by app rabbit 2025-09-11 01:57:06.035697+00:00 [info] <0.222.0> Running boot step rabbit_global_counters defined by app rabbit 2025-09-11 01:57:06.036116+00:00 [info] <0.222.0> Running boot step rabbit_osiris_metrics defined by app rabbit 2025-09-11 01:57:06.036255+00:00 [info] <0.222.0> Running boot step rabbit_core_metrics defined by app rabbit 2025-09-11 01:57:06.036671+00:00 [info] <0.222.0> Running boot step rabbit_alarm defined by app rabbit 2025-09-11 01:57:06.040065+00:00 [info] <0.418.0> Memory high watermark set to 12637 MiB (13251084288 bytes) of 15796 MiB (16563855360 bytes) total 2025-09-11 01:57:06.043453+00:00 [info] <0.420.0> Enabling free disk space monitoring 2025-09-11 01:57:06.043506+00:00 [info] <0.420.0> Disk free limit set to 50MB 2025-09-11 01:57:06.045921+00:00 [info] <0.222.0> Running boot step code_server_cache defined by app rabbit 2025-09-11 01:57:06.046035+00:00 [info] <0.222.0> Running boot step file_handle_cache defined by app rabbit 2025-09-11 01:57:06.046193+00:00 [info] <0.425.0> Limiting to approx 1048479 file handles (943629 sockets) 2025-09-11 01:57:06.046263+00:00 [info] <0.426.0> FHC read buffering: OFF 2025-09-11 01:57:06.046294+00:00 [info] <0.426.0> FHC write buffering: ON 2025-09-11 01:57:06.046645+00:00 [info] <0.222.0> Running boot step worker_pool defined by app rabbit 2025-09-11 01:57:06.046720+00:00 [info] <0.347.0> Will use 4 processes for default worker pool 2025-09-11 01:57:06.046754+00:00 [info] <0.347.0> Starting worker pool 'worker_pool' with 4 processes in it 2025-09-11 01:57:06.047035+00:00 [info] <0.222.0> Running boot step database defined by app rabbit 2025-09-11 01:57:06.048960+00:00 [info] <0.222.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-09-11 01:57:06.052419+00:00 [info] <0.222.0> Successfully synced tables from a peer 2025-09-11 01:57:06.052886+00:00 [info] <0.222.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-09-11 01:57:06.052969+00:00 [info] <0.222.0> Successfully synced tables from a peer 2025-09-11 01:57:06.053002+00:00 [warning] <0.222.0> Feature flags: the previous instance of this node must have failed to write the `feature_flags` file at `/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4079-3-feature_flags`: 2025-09-11 01:57:06.053055+00:00 [warning] <0.222.0> Feature flags: - list of previously disabled feature flags now marked as such: [empty_basic_get_metric] 2025-09-11 01:57:06.055018+00:00 [info] <0.222.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-09-11 01:57:06.055102+00:00 [info] <0.222.0> Successfully synced tables from a peer 2025-09-11 01:57:06.068109+00:00 [info] <0.222.0> upgrades: Mnesia backup removed 2025-09-11 01:57:06.068176+00:00 [info] <0.222.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-09-11 01:57:06.068260+00:00 [info] <0.222.0> Successfully synced tables from a peer 2025-09-11 01:57:06.068293+00:00 [info] <0.222.0> Peer discovery backend rabbit_peer_discovery_classic_config does not support registration, skipping registration. 2025-09-11 01:57:06.068352+00:00 [info] <0.222.0> Running boot step database_sync defined by app rabbit 2025-09-11 01:57:06.068442+00:00 [info] <0.222.0> Running boot step feature_flags defined by app rabbit 2025-09-11 01:57:06.068564+00:00 [info] <0.222.0> Running boot step codec_correctness_check defined by app rabbit 2025-09-11 01:57:06.068593+00:00 [info] <0.222.0> Running boot step external_infrastructure defined by app rabbit 2025-09-11 01:57:06.068612+00:00 [info] <0.222.0> Running boot step rabbit_event defined by app rabbit 2025-09-11 01:57:06.068694+00:00 [info] <0.222.0> Running boot step rabbit_registry defined by app rabbit 2025-09-11 01:57:06.068736+00:00 [info] <0.222.0> Running boot step rabbit_auth_mechanism_amqplain defined by app rabbit 2025-09-11 01:57:06.068774+00:00 [info] <0.222.0> Running boot step rabbit_auth_mechanism_cr_demo defined by app rabbit 2025-09-11 01:57:06.068836+00:00 [info] <0.222.0> Running boot step rabbit_auth_mechanism_plain defined by app rabbit 2025-09-11 01:57:06.068888+00:00 [info] <0.222.0> Running boot step rabbit_exchange_type_direct defined by app rabbit 2025-09-11 01:57:06.068963+00:00 [info] <0.222.0> Running boot step rabbit_exchange_type_fanout defined by app rabbit 2025-09-11 01:57:06.068998+00:00 [info] <0.222.0> Running boot step rabbit_exchange_type_headers defined by app rabbit 2025-09-11 01:57:06.069054+00:00 [info] <0.222.0> Running boot step rabbit_exchange_type_topic defined by app rabbit 2025-09-11 01:57:06.069127+00:00 [info] <0.222.0> Running boot step rabbit_mirror_queue_mode_all defined by app rabbit 2025-09-11 01:57:06.069165+00:00 [info] <0.222.0> Running boot step rabbit_mirror_queue_mode_exactly defined by app rabbit 2025-09-11 01:57:06.069202+00:00 [info] <0.222.0> Running boot step rabbit_mirror_queue_mode_nodes defined by app rabbit 2025-09-11 01:57:06.069239+00:00 [info] <0.222.0> Running boot step rabbit_priority_queue defined by app rabbit 2025-09-11 01:57:06.069265+00:00 [info] <0.222.0> Priority queues enabled, real BQ is rabbit_variable_queue 2025-09-11 01:57:06.069338+00:00 [info] <0.222.0> Running boot step rabbit_queue_location_client_local defined by app rabbit 2025-09-11 01:57:06.069385+00:00 [info] <0.222.0> Running boot step rabbit_queue_location_min_masters defined by app rabbit 2025-09-11 01:57:06.069422+00:00 [info] <0.222.0> Running boot step rabbit_queue_location_random defined by app rabbit 2025-09-11 01:57:06.069462+00:00 [info] <0.222.0> Running boot step kernel_ready defined by app rabbit 2025-09-11 01:57:06.069483+00:00 [info] <0.222.0> Running boot step rabbit_sysmon_minder defined by app rabbit 2025-09-11 01:57:06.069627+00:00 [info] <0.222.0> Running boot step rabbit_epmd_monitor defined by app rabbit 2025-09-11 01:57:06.070188+00:00 [info] <0.450.0> epmd monitor knows us, inter-node communication (distribution) port: 25673 2025-09-11 01:57:06.070279+00:00 [info] <0.222.0> Running boot step guid_generator defined by app rabbit 2025-09-11 01:57:06.074026+00:00 [info] <0.222.0> Running boot step rabbit_node_monitor defined by app rabbit 2025-09-11 01:57:06.075077+00:00 [info] <0.454.0> Starting rabbit_node_monitor 2025-09-11 01:57:06.075208+00:00 [info] <0.222.0> Running boot step delegate_sup defined by app rabbit 2025-09-11 01:57:06.075730+00:00 [info] <0.222.0> Running boot step rabbit_memory_monitor defined by app rabbit 2025-09-11 01:57:06.075858+00:00 [info] <0.222.0> Running boot step rabbit_fifo_dlx_sup defined by app rabbit 2025-09-11 01:57:06.075913+00:00 [info] <0.222.0> Running boot step core_initialized defined by app rabbit 2025-09-11 01:57:06.075940+00:00 [info] <0.222.0> Running boot step upgrade_queues defined by app rabbit 2025-09-11 01:57:06.082094+00:00 [info] <0.222.0> message_store upgrades: 1 to apply 2025-09-11 01:57:06.082245+00:00 [info] <0.222.0> message_store upgrades: Applying rabbit_variable_queue:move_messages_to_vhost_store 2025-09-11 01:57:06.082376+00:00 [info] <0.222.0> message_store upgrades: No durable queues found. Skipping message store migration 2025-09-11 01:57:06.082433+00:00 [info] <0.222.0> message_store upgrades: Removing the old message store data 2025-09-11 01:57:06.082772+00:00 [info] <0.222.0> message_store upgrades: All upgrades applied successfully 2025-09-11 01:57:06.091035+00:00 [info] <0.222.0> Running boot step channel_tracking defined by app rabbit 2025-09-11 01:57:06.098448+00:00 [info] <0.222.0> Setting up a table for channel tracking on this node: 'tracked_channel_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-4079-3' 2025-09-11 01:57:06.104703+00:00 [info] <0.222.0> Setting up a table for channel tracking on this node: 'tracked_channel_table_per_user_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-4079-3' 2025-09-11 01:57:06.104903+00:00 [info] <0.222.0> Running boot step rabbit_channel_tracking_handler defined by app rabbit 2025-09-11 01:57:06.104973+00:00 [info] <0.222.0> Running boot step connection_tracking defined by app rabbit 2025-09-11 01:57:06.121412+00:00 [info] <0.222.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-4079-3' 2025-09-11 01:57:06.128092+00:00 [info] <0.222.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-4079-3' 2025-09-11 01:57:06.133462+00:00 [info] <0.222.0> Setting up a table for per-user connection counting on this node: 'tracked_connection_table_per_user_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-4079-3' 2025-09-11 01:57:06.133712+00:00 [info] <0.222.0> Running boot step rabbit_connection_tracking_handler defined by app rabbit 2025-09-11 01:57:06.133796+00:00 [info] <0.222.0> Running boot step rabbit_definitions_hashing defined by app rabbit 2025-09-11 01:57:06.133889+00:00 [info] <0.222.0> Running boot step rabbit_exchange_parameters defined by app rabbit 2025-09-11 01:57:06.133993+00:00 [info] <0.222.0> Running boot step rabbit_mirror_queue_misc defined by app rabbit 2025-09-11 01:57:06.134143+00:00 [info] <0.222.0> Running boot step rabbit_policies defined by app rabbit 2025-09-11 01:57:06.134470+00:00 [info] <0.222.0> Running boot step rabbit_policy defined by app rabbit 2025-09-11 01:57:06.134513+00:00 [info] <0.222.0> Running boot step rabbit_queue_location_validator defined by app rabbit 2025-09-11 01:57:06.134577+00:00 [info] <0.222.0> Running boot step rabbit_quorum_memory_manager defined by app rabbit 2025-09-11 01:57:06.134620+00:00 [info] <0.222.0> Running boot step rabbit_stream_coordinator defined by app rabbit 2025-09-11 01:57:06.134745+00:00 [info] <0.222.0> Running boot step rabbit_vhost_limit defined by app rabbit 2025-09-11 01:57:06.134803+00:00 [info] <0.222.0> Running boot step rabbit_mgmt_reset_handler defined by app rabbitmq_management 2025-09-11 01:57:06.134845+00:00 [info] <0.222.0> Running boot step rabbit_mgmt_db_handler defined by app rabbitmq_management_agent 2025-09-11 01:57:06.134889+00:00 [warning] <0.222.0> force_fine_statistics set to true; ignored. 2025-09-11 01:57:06.134889+00:00 [warning] <0.222.0> Replaced by {rates_mode, none} in the rabbitmq_management application. 2025-09-11 01:57:06.134985+00:00 [info] <0.222.0> Management plugin: using rates mode 'basic' 2025-09-11 01:57:06.136571+00:00 [info] <0.222.0> Running boot step recovery defined by app rabbit 2025-09-11 01:57:06.142044+00:00 [info] <0.512.0> Making sure data directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-4079-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists 2025-09-11 01:57:06.142864+00:00 [info] <0.512.0> Setting segment_entry_count for vhost '/' with 0 queues to '2048' 2025-09-11 01:57:06.150301+00:00 [info] <0.512.0> Starting message stores for vhost '/' 2025-09-11 01:57:06.150601+00:00 [info] <0.517.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index 2025-09-11 01:57:06.151651+00:00 [info] <0.512.0> Started message store of type transient for vhost '/' 2025-09-11 01:57:06.151870+00:00 [info] <0.521.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index 2025-09-11 01:57:06.152329+00:00 [warning] <0.521.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": rebuilding indices from scratch 2025-09-11 01:57:06.152818+00:00 [info] <0.512.0> Started message store of type persistent for vhost '/' 2025-09-11 01:57:06.152966+00:00 [info] <0.512.0> Recovering 0 queues of type rabbit_classic_queue took 9ms 2025-09-11 01:57:06.152999+00:00 [info] <0.512.0> Recovering 0 queues of type rabbit_quorum_queue took 0ms 2025-09-11 01:57:06.153021+00:00 [info] <0.512.0> Recovering 0 queues of type rabbit_stream_queue took 0ms 2025-09-11 01:57:06.154130+00:00 [info] <0.512.0> Mirrored queue 'job_status_consumer.dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-3': <0.541.0> 2025-09-11 01:57:06.154417+00:00 [info] <0.512.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-4079-3.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-3': <0.545.0> 2025-09-11 01:57:06.154669+00:00 [info] <0.512.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-4079-2' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-3': <0.549.0> 2025-09-11 01:57:06.154882+00:00 [info] <0.512.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-4079-3' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-3': <0.553.0> 2025-09-11 01:57:06.155091+00:00 [info] <0.512.0> Mirrored queue 'device_manager_ztp.ztp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-3': <0.557.0> 2025-09-11 01:57:06.155292+00:00 [info] <0.512.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-4079-1' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-3': <0.561.0> 2025-09-11 01:57:06.155458+00:00 [info] <0.512.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-4079-2.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-3': <0.565.0> 2025-09-11 01:57:06.155641+00:00 [info] <0.512.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-4079-2..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-3': <0.569.0> 2025-09-11 01:57:06.155830+00:00 [info] <0.512.0> Mirrored queue 'job_request_consumer' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-3': <0.573.0> 2025-09-11 01:57:06.156014+00:00 [info] <0.512.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-4079-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-3': <0.577.0> 2025-09-11 01:57:06.156281+00:00 [info] <0.512.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-4079-3..tftp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-3': <0.581.0> 2025-09-11 01:57:06.156762+00:00 [info] <0.512.0> Mirrored queue 'device_manager.cn-jenkins-deploy-platform-ansible-os-4079-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-3': <0.586.0> 2025-09-11 01:57:06.157138+00:00 [info] <0.512.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-4079-1..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-3': <0.592.0> 2025-09-11 01:57:06.157360+00:00 [info] <0.512.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-4079-3..config_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-3': <0.597.0> 2025-09-11 01:57:06.157763+00:00 [info] <0.512.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-4079-3..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-3': <0.601.0> 2025-09-11 01:57:06.158033+00:00 [info] <0.512.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-4079-2' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-3': <0.605.0> 2025-09-11 01:57:06.158356+00:00 [info] <0.512.0> Mirrored queue 'svc_monitor.cn-jenkins-deploy-platform-ansible-os-4079-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-3': <0.609.0> 2025-09-11 01:57:06.158534+00:00 [info] <0.512.0> Mirrored queue 'schema_transformer.cn-jenkins-deploy-platform-ansible-os-4079-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-3': <0.613.0> 2025-09-11 01:57:06.158733+00:00 [info] <0.512.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-4079-1.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-3': <0.618.0> 2025-09-11 01:57:06.159180+00:00 [info] <0.512.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-4079-3' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-3': <0.623.0> 2025-09-11 01:57:06.159806+00:00 [info] <0.512.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-4079-1' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-3': <0.632.0> 2025-09-11 01:57:06.160287+00:00 [info] <0.222.0> Running boot step empty_db_check defined by app rabbit 2025-09-11 01:57:06.160347+00:00 [info] <0.222.0> Will not seed default virtual host and user: have definitions to load... 2025-09-11 01:57:06.160372+00:00 [info] <0.222.0> Running boot step rabbit_looking_glass defined by app rabbit 2025-09-11 01:57:06.160420+00:00 [info] <0.222.0> Running boot step rabbit_core_metrics_gc defined by app rabbit 2025-09-11 01:57:06.160771+00:00 [info] <0.222.0> Running boot step background_gc defined by app rabbit 2025-09-11 01:57:06.160932+00:00 [info] <0.222.0> Running boot step routing_ready defined by app rabbit 2025-09-11 01:57:06.160998+00:00 [info] <0.222.0> Running boot step pre_flight defined by app rabbit 2025-09-11 01:57:06.161048+00:00 [info] <0.222.0> Running boot step notify_cluster defined by app rabbit 2025-09-11 01:57:06.161083+00:00 [info] <0.222.0> Running boot step networking defined by app rabbit 2025-09-11 01:57:06.161113+00:00 [info] <0.222.0> Running boot step definition_import_worker_pool defined by app rabbit 2025-09-11 01:57:06.161245+00:00 [info] <0.347.0> Starting worker pool 'definition_import_pool' with 4 processes in it 2025-09-11 01:57:06.161968+00:00 [info] <0.454.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-1' up 2025-09-11 01:57:06.162593+00:00 [info] <0.222.0> Running boot step cluster_name defined by app rabbit 2025-09-11 01:57:06.162645+00:00 [info] <0.222.0> Running boot step direct_client defined by app rabbit 2025-09-11 01:57:06.162797+00:00 [info] <0.222.0> Running boot step rabbit_maintenance_mode_state defined by app rabbit 2025-09-11 01:57:06.162857+00:00 [info] <0.222.0> Creating table rabbit_node_maintenance_states for feature flag `maintenance_mode_status` 2025-09-11 01:57:06.164007+00:00 [info] <0.222.0> Running boot step rabbit_management_load_definitions defined by app rabbitmq_management 2025-09-11 01:57:06.164316+00:00 [info] <0.222.0> Applying definitions from file at '/etc/rabbitmq/definitions.json' 2025-09-11 01:57:06.164387+00:00 [info] <0.222.0> Asked to import definitions. Acting user: rmq-internal 2025-09-11 01:57:06.164493+00:00 [info] <0.222.0> Importing concurrently 1 users... 2025-09-11 01:57:06.171870+00:00 [info] <0.222.0> Importing concurrently 1 vhosts... 2025-09-11 01:57:06.179696+00:00 [info] <0.222.0> Importing concurrently 1 permissions... 2025-09-11 01:57:06.189519+00:00 [info] <0.642.0> Successfully set permissions for 'guest' in virtual host '/' to '.*', '.*', '.*' 2025-09-11 01:57:06.189687+00:00 [info] <0.222.0> Importing sequentially 1 policies... 2025-09-11 01:57:06.218049+00:00 [info] <0.454.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-4079-2' up 2025-09-11 01:57:06.218287+00:00 [info] <0.731.0> Resetting node maintenance status 2025-09-11 01:57:06.241289+00:00 [info] <0.791.0> Management plugin: HTTP (non-TLS) listener started on port 15673 2025-09-11 01:57:06.241470+00:00 [info] <0.819.0> Statistics database started. 2025-09-11 01:57:06.241559+00:00 [info] <0.818.0> Starting worker pool 'management_worker_pool' with 3 processes in it 2025-09-11 01:57:06.248757+00:00 [info] <0.833.0> Prometheus metrics: HTTP (non-TLS) listener started on port 15692 2025-09-11 01:57:06.248927+00:00 [info] <0.731.0> Ready to start client connection listeners 2025-09-11 01:57:06.250121+00:00 [info] <0.877.0> started TCP listener on 10.0.0.49:5673 2025-09-11 01:57:06.311653+00:00 [info] <0.731.0> Server startup complete; 4 plugins started. 2025-09-11 01:57:06.311653+00:00 [info] <0.731.0> * rabbitmq_prometheus 2025-09-11 01:57:06.311653+00:00 [info] <0.731.0> * rabbitmq_management 2025-09-11 01:57:06.311653+00:00 [info] <0.731.0> * rabbitmq_web_dispatch 2025-09-11 01:57:06.311653+00:00 [info] <0.731.0> * rabbitmq_management_agent