2025-03-20 00:44:22.958 [info] <0.8.0> Log file opened with Lager 2025-03-20 00:44:25.547 [info] <0.8.0> Feature flags: list of feature flags found: 2025-03-20 00:44:25.547 [info] <0.8.0> Feature flags: feature flag states written to disk: yes 2025-03-20 00:44:25.582 [info] <0.240.0> Starting RabbitMQ 3.7.28 on Erlang 21.3.8.21 Copyright (c) 2007-2020 Pivotal Software, Inc. Licensed under the MPL. See https://www.rabbitmq.com/ 2025-03-20 00:44:25.583 [info] <0.240.0> node : contrail@cn-jenkins-deploy-platform-ansible-os-2923-3 home dir : /var/lib/rabbitmq config file(s) : /etc/rabbitmq/rabbitmq.config cookie hash : esZsDxSN6VGbi9JkMSxNZA== log(s) : /var/log/contrail/config-database-rabbitmq/rabbitmq.log : /var/log/rabbitmq/contrail@cn-jenkins-deploy-platform-ansible-os-2923-3_upgrade.log database dir : /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2923-3 2025-03-20 00:44:25.600 [info] <0.240.0> Running boot step pre_boot defined by app rabbit 2025-03-20 00:44:25.600 [info] <0.240.0> Running boot step rabbit_core_metrics defined by app rabbit 2025-03-20 00:44:25.601 [info] <0.240.0> Running boot step rabbit_alarm defined by app rabbit 2025-03-20 00:44:25.612 [info] <0.246.0> Memory high watermark set to 12637 MiB (13251077734 bytes) of 15796 MiB (16563847168 bytes) total 2025-03-20 00:44:25.620 [info] <0.248.0> Enabling free disk space monitoring 2025-03-20 00:44:25.620 [info] <0.248.0> Disk free limit set to 50MB 2025-03-20 00:44:25.626 [info] <0.240.0> Running boot step code_server_cache defined by app rabbit 2025-03-20 00:44:25.626 [info] <0.240.0> Running boot step file_handle_cache defined by app rabbit 2025-03-20 00:44:25.626 [info] <0.251.0> Limiting to approx 65436 file handles (58890 sockets) 2025-03-20 00:44:25.626 [info] <0.252.0> FHC read buffering: OFF 2025-03-20 00:44:25.626 [info] <0.252.0> FHC write buffering: ON 2025-03-20 00:44:25.627 [info] <0.240.0> Running boot step worker_pool defined by app rabbit 2025-03-20 00:44:25.627 [info] <0.241.0> Will use 4 processes for default worker pool 2025-03-20 00:44:25.627 [info] <0.241.0> Starting worker pool 'worker_pool' with 4 processes in it 2025-03-20 00:44:25.627 [info] <0.240.0> Running boot step database defined by app rabbit 2025-03-20 00:44:25.627 [info] <0.240.0> Node database directory at /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2923-3 is empty. Assuming we need to join an existing cluster or initialise from scratch... 2025-03-20 00:44:25.627 [info] <0.240.0> Configured peer discovery backend: rabbit_peer_discovery_classic_config 2025-03-20 00:44:25.627 [info] <0.240.0> Will try to lock with peer discovery backend rabbit_peer_discovery_classic_config 2025-03-20 00:44:25.627 [info] <0.240.0> Peer discovery backend does not support locking, falling back to randomized delay 2025-03-20 00:44:25.627 [info] <0.240.0> Peer discovery backend rabbit_peer_discovery_classic_config supports registration. 2025-03-20 00:44:25.627 [info] <0.240.0> Will wait for 51728 milliseconds before proceeding with registration... 2025-03-20 00:45:09.939 [info] <0.319.0> RabbitMQ hasn't finished starting yet. Waiting for startup to finish before stopping... 2025-03-20 00:45:17.357 [info] <0.240.0> All discovered existing cluster peers: contrail@cn-jenkins-deploy-platform-ansible-os-2923-1, contrail@cn-jenkins-deploy-platform-ansible-os-2923-2, contrail@cn-jenkins-deploy-platform-ansible-os-2923-3 2025-03-20 00:45:17.357 [info] <0.240.0> Peer nodes we can cluster with: contrail@cn-jenkins-deploy-platform-ansible-os-2923-1, contrail@cn-jenkins-deploy-platform-ansible-os-2923-2 2025-03-20 00:45:17.359 [info] <0.240.0> Node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-1' selected for auto-clustering 2025-03-20 00:45:17.378 [info] <0.240.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-03-20 00:45:17.378 [info] <0.240.0> Successfully synced tables from a peer 2025-03-20 00:45:17.757 [info] <0.240.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-03-20 00:45:17.758 [info] <0.240.0> Successfully synced tables from a peer 2025-03-20 00:45:17.789 [info] <0.240.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-03-20 00:45:17.790 [info] <0.240.0> Successfully synced tables from a peer 2025-03-20 00:45:17.799 [info] <0.240.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2923-3' 2025-03-20 00:45:17.807 [info] <0.240.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2923-3' 2025-03-20 00:45:17.808 [info] <0.240.0> Will register with peer discovery backend rabbit_peer_discovery_classic_config 2025-03-20 00:45:17.808 [info] <0.240.0> Running boot step database_sync defined by app rabbit 2025-03-20 00:45:17.808 [info] <0.240.0> Running boot step feature_flags defined by app rabbit 2025-03-20 00:45:17.809 [info] <0.240.0> Running boot step codec_correctness_check defined by app rabbit 2025-03-20 00:45:17.809 [info] <0.240.0> Running boot step external_infrastructure defined by app rabbit 2025-03-20 00:45:17.809 [info] <0.240.0> Running boot step rabbit_registry defined by app rabbit 2025-03-20 00:45:17.809 [info] <0.240.0> Running boot step rabbit_auth_mechanism_cr_demo defined by app rabbit 2025-03-20 00:45:17.809 [info] <0.240.0> Running boot step rabbit_queue_location_random defined by app rabbit 2025-03-20 00:45:17.809 [info] <0.240.0> Running boot step rabbit_event defined by app rabbit 2025-03-20 00:45:17.809 [info] <0.240.0> Running boot step rabbit_auth_mechanism_amqplain defined by app rabbit 2025-03-20 00:45:17.809 [info] <0.240.0> Running boot step rabbit_auth_mechanism_plain defined by app rabbit 2025-03-20 00:45:17.809 [info] <0.240.0> Running boot step rabbit_exchange_type_direct defined by app rabbit 2025-03-20 00:45:17.809 [info] <0.240.0> Running boot step rabbit_exchange_type_fanout defined by app rabbit 2025-03-20 00:45:17.810 [info] <0.240.0> Running boot step rabbit_exchange_type_headers defined by app rabbit 2025-03-20 00:45:17.810 [info] <0.240.0> Running boot step rabbit_exchange_type_topic defined by app rabbit 2025-03-20 00:45:17.810 [info] <0.240.0> Running boot step rabbit_mirror_queue_mode_all defined by app rabbit 2025-03-20 00:45:17.810 [info] <0.240.0> Running boot step rabbit_mirror_queue_mode_exactly defined by app rabbit 2025-03-20 00:45:17.810 [info] <0.240.0> Running boot step rabbit_mirror_queue_mode_nodes defined by app rabbit 2025-03-20 00:45:17.810 [info] <0.240.0> Running boot step rabbit_priority_queue defined by app rabbit 2025-03-20 00:45:17.810 [info] <0.240.0> Priority queues enabled, real BQ is rabbit_variable_queue 2025-03-20 00:45:17.810 [info] <0.240.0> Running boot step rabbit_queue_location_client_local defined by app rabbit 2025-03-20 00:45:17.810 [info] <0.240.0> Running boot step rabbit_queue_location_min_masters defined by app rabbit 2025-03-20 00:45:17.810 [info] <0.240.0> Running boot step kernel_ready defined by app rabbit 2025-03-20 00:45:17.810 [info] <0.240.0> Running boot step rabbit_sysmon_minder defined by app rabbit 2025-03-20 00:45:17.810 [info] <0.240.0> Running boot step rabbit_epmd_monitor defined by app rabbit 2025-03-20 00:45:17.811 [info] <0.488.0> epmd monitor knows us, inter-node communication (distribution) port: 25673 2025-03-20 00:45:17.811 [info] <0.240.0> Running boot step guid_generator defined by app rabbit 2025-03-20 00:45:17.814 [info] <0.240.0> Running boot step rabbit_node_monitor defined by app rabbit 2025-03-20 00:45:17.817 [info] <0.492.0> Starting rabbit_node_monitor 2025-03-20 00:45:17.817 [info] <0.240.0> Running boot step delegate_sup defined by app rabbit 2025-03-20 00:45:17.818 [info] <0.240.0> Running boot step rabbit_memory_monitor defined by app rabbit 2025-03-20 00:45:17.818 [info] <0.240.0> Running boot step core_initialized defined by app rabbit 2025-03-20 00:45:17.818 [info] <0.240.0> Running boot step upgrade_queues defined by app rabbit 2025-03-20 00:45:17.845 [info] <0.240.0> message_store upgrades: 1 to apply 2025-03-20 00:45:17.845 [info] <0.240.0> message_store upgrades: Applying rabbit_variable_queue:move_messages_to_vhost_store 2025-03-20 00:45:17.845 [info] <0.240.0> message_store upgrades: No durable queues found. Skipping message store migration 2025-03-20 00:45:17.845 [info] <0.240.0> message_store upgrades: Removing the old message store data 2025-03-20 00:45:17.850 [info] <0.240.0> message_store upgrades: All upgrades applied successfully 2025-03-20 00:45:17.877 [info] <0.240.0> Running boot step rabbit_connection_tracking defined by app rabbit 2025-03-20 00:45:17.877 [info] <0.240.0> Running boot step rabbit_connection_tracking_handler defined by app rabbit 2025-03-20 00:45:17.877 [info] <0.240.0> Running boot step rabbit_exchange_parameters defined by app rabbit 2025-03-20 00:45:17.878 [info] <0.240.0> Running boot step rabbit_mirror_queue_misc defined by app rabbit 2025-03-20 00:45:17.878 [info] <0.240.0> Running boot step rabbit_policies defined by app rabbit 2025-03-20 00:45:17.879 [info] <0.240.0> Running boot step rabbit_policy defined by app rabbit 2025-03-20 00:45:17.880 [info] <0.240.0> Running boot step rabbit_queue_location_validator defined by app rabbit 2025-03-20 00:45:17.880 [info] <0.240.0> Running boot step rabbit_vhost_limit defined by app rabbit 2025-03-20 00:45:17.880 [info] <0.240.0> Running boot step rabbit_mgmt_reset_handler defined by app rabbitmq_management 2025-03-20 00:45:17.881 [info] <0.240.0> Running boot step rabbit_mgmt_db_handler defined by app rabbitmq_management_agent 2025-03-20 00:45:17.881 [warning] <0.240.0> force_fine_statistics set to true; ignored. Replaced by {rates_mode, none} in the rabbitmq_management application. 2025-03-20 00:45:17.881 [info] <0.240.0> Management plugin: using rates mode 'basic' 2025-03-20 00:45:17.883 [info] <0.240.0> Running boot step recovery defined by app rabbit 2025-03-20 00:45:17.885 [info] <0.529.0> Making sure data directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2923-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists 2025-03-20 00:45:17.894 [info] <0.529.0> Starting message stores for vhost '/' 2025-03-20 00:45:17.894 [info] <0.533.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index 2025-03-20 00:45:17.909 [info] <0.529.0> Started message store of type transient for vhost '/' 2025-03-20 00:45:17.909 [info] <0.536.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index 2025-03-20 00:45:17.917 [warning] <0.536.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": rebuilding indices from scratch 2025-03-20 00:45:17.917 [info] <0.529.0> Started message store of type persistent for vhost '/' 2025-03-20 00:45:17.918 [info] <0.240.0> Running boot step load_definitions defined by app rabbitmq_management 2025-03-20 00:45:17.919 [info] <0.240.0> Applying definitions from /etc/rabbitmq/definitions.json 2025-03-20 00:45:17.919 [info] <0.240.0> Asked to import definitions. Acting user: rmq-internal 2025-03-20 00:45:17.919 [info] <0.240.0> Importing users... 2025-03-20 00:45:17.925 [info] <0.240.0> Setting user tags for user 'guest' to [administrator] 2025-03-20 00:45:17.932 [info] <0.240.0> Importing vhosts... 2025-03-20 00:45:17.932 [info] <0.240.0> Importing user permissions... 2025-03-20 00:45:17.932 [info] <0.240.0> Setting permissions for 'guest' in '/' to '.*', '.*', '.*' 2025-03-20 00:45:17.938 [info] <0.240.0> Importing topic permissions... 2025-03-20 00:45:17.938 [info] <0.240.0> Importing parameters... 2025-03-20 00:45:17.938 [info] <0.240.0> Importing global parameters... 2025-03-20 00:45:17.938 [info] <0.240.0> Importing policies... 2025-03-20 00:45:17.947 [info] <0.240.0> Importing queues... 2025-03-20 00:45:17.947 [info] <0.240.0> Importing exchanges... 2025-03-20 00:45:17.947 [info] <0.240.0> Importing bindings... 2025-03-20 00:45:17.947 [info] <0.240.0> Running boot step empty_db_check defined by app rabbit 2025-03-20 00:45:17.948 [info] <0.240.0> Running boot step rabbit_looking_glass defined by app rabbit 2025-03-20 00:45:17.948 [info] <0.240.0> Running boot step rabbit_core_metrics_gc defined by app rabbit 2025-03-20 00:45:17.948 [info] <0.240.0> Running boot step background_gc defined by app rabbit 2025-03-20 00:45:17.948 [info] <0.240.0> Running boot step connection_tracking defined by app rabbit 2025-03-20 00:45:17.949 [info] <0.240.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2923-3' 2025-03-20 00:45:17.950 [info] <0.240.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2923-3' 2025-03-20 00:45:17.950 [info] <0.240.0> Running boot step routing_ready defined by app rabbit 2025-03-20 00:45:17.950 [info] <0.240.0> Running boot step pre_flight defined by app rabbit 2025-03-20 00:45:17.950 [info] <0.240.0> Running boot step notify_cluster defined by app rabbit 2025-03-20 00:45:17.950 [info] <0.240.0> Running boot step networking defined by app rabbit 2025-03-20 00:45:17.951 [info] <0.492.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-2' up 2025-03-20 00:45:17.953 [info] <0.577.0> started TCP listener on [::]:5673 2025-03-20 00:45:17.953 [info] <0.240.0> Running boot step cluster_name defined by app rabbit 2025-03-20 00:45:17.953 [info] <0.240.0> Running boot step direct_client defined by app rabbit 2025-03-20 00:45:17.953 [info] <0.240.0> Running boot step os_signal_handler defined by app rabbit 2025-03-20 00:45:17.953 [info] <0.579.0> Swapping OS signal event handler (erl_signal_server) for our own 2025-03-20 00:45:17.961 [info] <0.492.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-1' up 2025-03-20 00:45:17.996 [info] <0.631.0> Management plugin: HTTP (non-TLS) listener started on port 15673 2025-03-20 00:45:17.996 [info] <0.737.0> Statistics database started. 2025-03-20 00:45:17.996 [info] <0.736.0> Starting worker pool 'management_worker_pool' with 3 processes in it 2025-03-20 00:45:17.997 [notice] <0.106.0> Changed loghwm of /var/log/contrail/config-database-rabbitmq/rabbitmq.log to 50 2025-03-20 00:45:18.136 [info] <0.8.0> Server startup complete; 3 plugins started. * rabbitmq_management * rabbitmq_web_dispatch * rabbitmq_management_agent 2025-03-20 00:45:18.221 [info] <0.319.0> RabbitMQ is asked to stop... 2025-03-20 00:45:18.310 [info] <0.319.0> Stopping RabbitMQ applications and their dependencies in the following order: rabbitmq_management amqp_client rabbitmq_web_dispatch cowboy cowlib rabbitmq_management_agent rabbit mnesia rabbit_common sysmon_handler os_mon 2025-03-20 00:45:18.310 [info] <0.319.0> Stopping application 'rabbitmq_management' 2025-03-20 00:45:18.311 [warning] <0.627.0> RabbitMQ HTTP listener registry could not find context rabbitmq_management_tls 2025-03-20 00:45:18.313 [info] <0.43.0> Application rabbitmq_management exited with reason: stopped 2025-03-20 00:45:18.313 [info] <0.319.0> Stopping application 'amqp_client' 2025-03-20 00:45:18.315 [info] <0.43.0> Application amqp_client exited with reason: stopped 2025-03-20 00:45:18.315 [info] <0.319.0> Stopping application 'rabbitmq_web_dispatch' 2025-03-20 00:45:18.316 [info] <0.43.0> Application rabbitmq_web_dispatch exited with reason: stopped 2025-03-20 00:45:18.316 [info] <0.319.0> Stopping application 'cowboy' 2025-03-20 00:45:18.317 [info] <0.319.0> Stopping application 'cowlib' 2025-03-20 00:45:18.317 [info] <0.43.0> Application cowboy exited with reason: stopped 2025-03-20 00:45:18.318 [info] <0.319.0> Stopping application 'rabbitmq_management_agent' 2025-03-20 00:45:18.318 [info] <0.43.0> Application cowlib exited with reason: stopped 2025-03-20 00:45:18.319 [info] <0.319.0> Stopping application 'rabbit' 2025-03-20 00:45:18.319 [info] <0.240.0> Will unregister with peer discovery backend rabbit_peer_discovery_classic_config 2025-03-20 00:45:18.319 [info] <0.43.0> Application rabbitmq_management_agent exited with reason: stopped 2025-03-20 00:45:18.319 [info] <0.577.0> stopped TCP listener on [::]:5673 2025-03-20 00:45:18.320 [info] <0.517.0> Closing all connections in vhost '/' on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3' because the vhost is stopping 2025-03-20 00:45:18.320 [info] <0.536.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2923-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' 2025-03-20 00:45:18.324 [info] <0.536.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2923-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' is stopped 2025-03-20 00:45:18.325 [info] <0.533.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2923-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' 2025-03-20 00:45:18.330 [info] <0.533.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2923-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' is stopped 2025-03-20 00:45:18.332 [info] <0.319.0> Stopping application 'mnesia' 2025-03-20 00:45:18.332 [info] <0.43.0> Application rabbit exited with reason: stopped 2025-03-20 00:45:18.335 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-03-20 00:45:18.335 [info] <0.319.0> Stopping application 'rabbit_common' 2025-03-20 00:45:18.335 [info] <0.319.0> Stopping application 'sysmon_handler' 2025-03-20 00:45:18.335 [info] <0.43.0> Application rabbit_common exited with reason: stopped 2025-03-20 00:45:18.336 [info] <0.319.0> Stopping application 'os_mon' 2025-03-20 00:45:18.336 [info] <0.43.0> Application sysmon_handler exited with reason: stopped 2025-03-20 00:45:18.338 [info] <0.319.0> Successfully stopped RabbitMQ and its dependencies 2025-03-20 00:45:18.338 [info] <0.43.0> Application os_mon exited with reason: stopped 2025-03-20 00:45:28.077 [info] <0.761.0> Resetting Rabbit forcefully 2025-03-20 00:45:28.957 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-03-20 00:45:28.961 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-03-20 00:45:29.049 [info] <0.767.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-03-20 00:45:29.049 [info] <0.767.0> Successfully synced tables from a peer 2025-03-20 00:45:29.078 [info] <0.767.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-03-20 00:45:29.078 [info] <0.767.0> Successfully synced tables from a peer 2025-03-20 00:45:29.109 [info] <0.767.0> Waiting for Mnesia tables for 30000 ms, 0 retries left 2025-03-20 00:45:29.109 [info] <0.767.0> Successfully synced tables from a peer 2025-03-20 00:45:29.115 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-03-20 00:45:29.123 [info] <0.767.0> Clustering with ['contrail@cn-jenkins-deploy-platform-ansible-os-2923-2','contrail@cn-jenkins-deploy-platform-ansible-os-2923-1'] as disc node 2025-03-20 00:45:29.143 [info] <0.767.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-03-20 00:45:29.143 [info] <0.767.0> Successfully synced tables from a peer 2025-03-20 00:45:29.439 [info] <0.767.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-03-20 00:45:29.439 [info] <0.767.0> Successfully synced tables from a peer 2025-03-20 00:45:29.472 [info] <0.767.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-03-20 00:45:29.472 [info] <0.767.0> Successfully synced tables from a peer 2025-03-20 00:45:29.477 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-03-20 00:45:29.477 [error] <0.767.0> Failed to create a tracked connection table for node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': {node_not_running,'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3'} 2025-03-20 00:45:29.477 [info] <0.767.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2923-3' 2025-03-20 00:45:29.478 [error] <0.767.0> Failed to create a per-vhost tracked connection table for node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': {node_not_running,'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3'} 2025-03-20 00:45:29.478 [info] <0.767.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2923-3' 2025-03-20 00:45:30.353 [info] <0.1128.0> RabbitMQ is asked to stop... 2025-03-20 00:45:30.413 [info] <0.1128.0> Stopping RabbitMQ applications and their dependencies in the following order: rabbitmq_management amqp_client rabbitmq_web_dispatch rabbitmq_management_agent rabbit mnesia rabbit_common sysmon_handler os_mon 2025-03-20 00:45:30.413 [info] <0.1128.0> Stopping application 'rabbitmq_management' 2025-03-20 00:45:30.413 [info] <0.1128.0> Stopping application 'amqp_client' 2025-03-20 00:45:30.413 [info] <0.1128.0> Stopping application 'rabbitmq_web_dispatch' 2025-03-20 00:45:30.413 [info] <0.1128.0> Stopping application 'rabbitmq_management_agent' 2025-03-20 00:45:30.413 [info] <0.1128.0> Stopping application 'rabbit' 2025-03-20 00:45:30.413 [info] <0.1128.0> Stopping application 'mnesia' 2025-03-20 00:45:30.413 [info] <0.1128.0> Stopping application 'rabbit_common' 2025-03-20 00:45:30.413 [info] <0.1128.0> Stopping application 'sysmon_handler' 2025-03-20 00:45:30.413 [info] <0.1128.0> Stopping application 'os_mon' 2025-03-20 00:45:30.413 [info] <0.1128.0> Successfully stopped RabbitMQ and its dependencies 2025-03-20 00:45:30.413 [info] <0.1128.0> Halting Erlang VM with the following applications: ranch ssl public_key asn1 observer_cli tools credentials_obfuscation crypto stdout_formatter recon inets jsx xmerl lager goldrush compiler syntax_tools sasl stdlib kernel 2025-03-20 00:45:35.437 [info] <0.8.0> Log file opened with Lager 2025-03-20 00:45:36.886 [info] <0.8.0> Feature flags: list of feature flags found: 2025-03-20 00:45:36.886 [info] <0.8.0> Feature flags: feature flag states written to disk: yes 2025-03-20 00:45:36.966 [info] <0.8.0> upgrades: Backing up mnesia dir to "/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2923-3-upgrade-backup" 2025-03-20 00:45:36.968 [info] <0.8.0> upgrades: Mnesia dir backed up to "/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2923-3-upgrade-backup" 2025-03-20 00:45:36.977 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-03-20 00:45:37.023 [info] <0.301.0> Starting RabbitMQ 3.7.28 on Erlang 21.3.8.21 Copyright (c) 2007-2020 Pivotal Software, Inc. Licensed under the MPL. See https://www.rabbitmq.com/ 2025-03-20 00:45:37.024 [info] <0.301.0> node : contrail@cn-jenkins-deploy-platform-ansible-os-2923-3 home dir : /var/lib/rabbitmq config file(s) : /etc/rabbitmq/rabbitmq.config cookie hash : esZsDxSN6VGbi9JkMSxNZA== log(s) : /var/log/contrail/config-database-rabbitmq/rabbitmq.log : /var/log/rabbitmq/contrail@cn-jenkins-deploy-platform-ansible-os-2923-3_upgrade.log database dir : /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2923-3 2025-03-20 00:45:37.037 [info] <0.301.0> Running boot step pre_boot defined by app rabbit 2025-03-20 00:45:37.037 [info] <0.301.0> Running boot step rabbit_core_metrics defined by app rabbit 2025-03-20 00:45:37.038 [info] <0.301.0> Running boot step rabbit_alarm defined by app rabbit 2025-03-20 00:45:37.042 [info] <0.314.0> Memory high watermark set to 12637 MiB (13251077734 bytes) of 15796 MiB (16563847168 bytes) total 2025-03-20 00:45:37.048 [info] <0.320.0> Enabling free disk space monitoring 2025-03-20 00:45:37.049 [info] <0.320.0> Disk free limit set to 50MB 2025-03-20 00:45:37.054 [info] <0.301.0> Running boot step code_server_cache defined by app rabbit 2025-03-20 00:45:37.054 [info] <0.301.0> Running boot step file_handle_cache defined by app rabbit 2025-03-20 00:45:37.054 [info] <0.335.0> FHC read buffering: OFF 2025-03-20 00:45:37.054 [info] <0.335.0> FHC write buffering: ON 2025-03-20 00:45:37.054 [info] <0.334.0> Limiting to approx 65436 file handles (58890 sockets) 2025-03-20 00:45:37.054 [info] <0.301.0> Running boot step worker_pool defined by app rabbit 2025-03-20 00:45:37.054 [info] <0.302.0> Will use 4 processes for default worker pool 2025-03-20 00:45:37.054 [info] <0.302.0> Starting worker pool 'worker_pool' with 4 processes in it 2025-03-20 00:45:37.055 [info] <0.301.0> Running boot step database defined by app rabbit 2025-03-20 00:45:37.073 [info] <0.301.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-03-20 00:45:37.098 [info] <0.301.0> Successfully synced tables from a peer 2025-03-20 00:45:37.099 [info] <0.301.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-03-20 00:45:37.099 [info] <0.301.0> Successfully synced tables from a peer 2025-03-20 00:45:37.129 [info] <0.301.0> upgrades: Mnesia backup removed 2025-03-20 00:45:37.129 [info] <0.301.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-03-20 00:45:37.129 [info] <0.301.0> Successfully synced tables from a peer 2025-03-20 00:45:37.129 [info] <0.301.0> Will register with peer discovery backend rabbit_peer_discovery_classic_config 2025-03-20 00:45:37.203 [info] <0.301.0> Running boot step database_sync defined by app rabbit 2025-03-20 00:45:37.203 [info] <0.301.0> Running boot step feature_flags defined by app rabbit 2025-03-20 00:45:37.203 [info] <0.301.0> Running boot step codec_correctness_check defined by app rabbit 2025-03-20 00:45:37.203 [info] <0.301.0> Running boot step external_infrastructure defined by app rabbit 2025-03-20 00:45:37.203 [info] <0.301.0> Running boot step rabbit_registry defined by app rabbit 2025-03-20 00:45:37.203 [info] <0.301.0> Running boot step rabbit_auth_mechanism_cr_demo defined by app rabbit 2025-03-20 00:45:37.203 [info] <0.301.0> Running boot step rabbit_queue_location_random defined by app rabbit 2025-03-20 00:45:37.203 [info] <0.301.0> Running boot step rabbit_event defined by app rabbit 2025-03-20 00:45:37.204 [info] <0.301.0> Running boot step rabbit_auth_mechanism_amqplain defined by app rabbit 2025-03-20 00:45:37.204 [info] <0.301.0> Running boot step rabbit_auth_mechanism_plain defined by app rabbit 2025-03-20 00:45:37.204 [info] <0.301.0> Running boot step rabbit_exchange_type_direct defined by app rabbit 2025-03-20 00:45:37.204 [info] <0.301.0> Running boot step rabbit_exchange_type_fanout defined by app rabbit 2025-03-20 00:45:37.204 [info] <0.301.0> Running boot step rabbit_exchange_type_headers defined by app rabbit 2025-03-20 00:45:37.204 [info] <0.301.0> Running boot step rabbit_exchange_type_topic defined by app rabbit 2025-03-20 00:45:37.204 [info] <0.301.0> Running boot step rabbit_mirror_queue_mode_all defined by app rabbit 2025-03-20 00:45:37.204 [info] <0.301.0> Running boot step rabbit_mirror_queue_mode_exactly defined by app rabbit 2025-03-20 00:45:37.204 [info] <0.301.0> Running boot step rabbit_mirror_queue_mode_nodes defined by app rabbit 2025-03-20 00:45:37.205 [info] <0.301.0> Running boot step rabbit_priority_queue defined by app rabbit 2025-03-20 00:45:37.205 [info] <0.301.0> Priority queues enabled, real BQ is rabbit_variable_queue 2025-03-20 00:45:37.205 [info] <0.301.0> Running boot step rabbit_queue_location_client_local defined by app rabbit 2025-03-20 00:45:37.205 [info] <0.301.0> Running boot step rabbit_queue_location_min_masters defined by app rabbit 2025-03-20 00:45:37.205 [info] <0.301.0> Running boot step kernel_ready defined by app rabbit 2025-03-20 00:45:37.205 [info] <0.301.0> Running boot step rabbit_sysmon_minder defined by app rabbit 2025-03-20 00:45:37.205 [info] <0.301.0> Running boot step rabbit_epmd_monitor defined by app rabbit 2025-03-20 00:45:37.206 [info] <0.410.0> epmd monitor knows us, inter-node communication (distribution) port: 25673 2025-03-20 00:45:37.206 [info] <0.301.0> Running boot step guid_generator defined by app rabbit 2025-03-20 00:45:37.210 [info] <0.301.0> Running boot step rabbit_node_monitor defined by app rabbit 2025-03-20 00:45:37.211 [info] <0.414.0> Starting rabbit_node_monitor 2025-03-20 00:45:37.211 [info] <0.301.0> Running boot step delegate_sup defined by app rabbit 2025-03-20 00:45:37.212 [info] <0.301.0> Running boot step rabbit_memory_monitor defined by app rabbit 2025-03-20 00:45:37.212 [info] <0.301.0> Running boot step core_initialized defined by app rabbit 2025-03-20 00:45:37.212 [info] <0.301.0> Running boot step upgrade_queues defined by app rabbit 2025-03-20 00:45:37.233 [info] <0.301.0> message_store upgrades: 1 to apply 2025-03-20 00:45:37.234 [info] <0.301.0> message_store upgrades: Applying rabbit_variable_queue:move_messages_to_vhost_store 2025-03-20 00:45:37.234 [info] <0.301.0> message_store upgrades: No durable queues found. Skipping message store migration 2025-03-20 00:45:37.234 [info] <0.301.0> message_store upgrades: Removing the old message store data 2025-03-20 00:45:37.234 [info] <0.301.0> message_store upgrades: All upgrades applied successfully 2025-03-20 00:45:37.261 [info] <0.301.0> Running boot step rabbit_connection_tracking defined by app rabbit 2025-03-20 00:45:37.261 [info] <0.301.0> Running boot step rabbit_connection_tracking_handler defined by app rabbit 2025-03-20 00:45:37.261 [info] <0.301.0> Running boot step rabbit_exchange_parameters defined by app rabbit 2025-03-20 00:45:37.261 [info] <0.301.0> Running boot step rabbit_mirror_queue_misc defined by app rabbit 2025-03-20 00:45:37.261 [info] <0.301.0> Running boot step rabbit_policies defined by app rabbit 2025-03-20 00:45:37.262 [info] <0.301.0> Running boot step rabbit_policy defined by app rabbit 2025-03-20 00:45:37.262 [info] <0.301.0> Running boot step rabbit_queue_location_validator defined by app rabbit 2025-03-20 00:45:37.262 [info] <0.301.0> Running boot step rabbit_vhost_limit defined by app rabbit 2025-03-20 00:45:37.262 [info] <0.301.0> Running boot step rabbit_mgmt_reset_handler defined by app rabbitmq_management 2025-03-20 00:45:37.262 [info] <0.301.0> Running boot step rabbit_mgmt_db_handler defined by app rabbitmq_management_agent 2025-03-20 00:45:37.262 [warning] <0.301.0> force_fine_statistics set to true; ignored. Replaced by {rates_mode, none} in the rabbitmq_management application. 2025-03-20 00:45:37.262 [info] <0.301.0> Management plugin: using rates mode 'basic' 2025-03-20 00:45:37.264 [info] <0.301.0> Running boot step recovery defined by app rabbit 2025-03-20 00:45:37.265 [info] <0.451.0> Making sure data directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2923-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists 2025-03-20 00:45:37.269 [info] <0.451.0> Starting message stores for vhost '/' 2025-03-20 00:45:37.269 [info] <0.455.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index 2025-03-20 00:45:37.270 [info] <0.451.0> Started message store of type transient for vhost '/' 2025-03-20 00:45:37.270 [info] <0.458.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index 2025-03-20 00:45:37.271 [warning] <0.458.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": rebuilding indices from scratch 2025-03-20 00:45:37.271 [info] <0.451.0> Started message store of type persistent for vhost '/' 2025-03-20 00:45:37.272 [info] <0.301.0> Running boot step load_definitions defined by app rabbitmq_management 2025-03-20 00:45:37.272 [info] <0.301.0> Applying definitions from /etc/rabbitmq/definitions.json 2025-03-20 00:45:37.272 [info] <0.301.0> Asked to import definitions. Acting user: rmq-internal 2025-03-20 00:45:37.272 [info] <0.301.0> Importing users... 2025-03-20 00:45:37.277 [info] <0.301.0> Setting user tags for user 'guest' to [administrator] 2025-03-20 00:45:37.281 [info] <0.301.0> Importing vhosts... 2025-03-20 00:45:37.281 [info] <0.301.0> Importing user permissions... 2025-03-20 00:45:37.282 [info] <0.301.0> Setting permissions for 'guest' in '/' to '.*', '.*', '.*' 2025-03-20 00:45:37.286 [info] <0.301.0> Importing topic permissions... 2025-03-20 00:45:37.286 [info] <0.301.0> Importing parameters... 2025-03-20 00:45:37.286 [info] <0.301.0> Importing global parameters... 2025-03-20 00:45:37.286 [info] <0.301.0> Importing policies... 2025-03-20 00:45:37.290 [info] <0.301.0> Importing queues... 2025-03-20 00:45:37.290 [info] <0.301.0> Importing exchanges... 2025-03-20 00:45:37.290 [info] <0.301.0> Importing bindings... 2025-03-20 00:45:37.290 [info] <0.301.0> Running boot step empty_db_check defined by app rabbit 2025-03-20 00:45:37.290 [info] <0.301.0> Running boot step rabbit_looking_glass defined by app rabbit 2025-03-20 00:45:37.290 [info] <0.301.0> Running boot step rabbit_core_metrics_gc defined by app rabbit 2025-03-20 00:45:37.290 [info] <0.301.0> Running boot step background_gc defined by app rabbit 2025-03-20 00:45:37.291 [info] <0.301.0> Running boot step connection_tracking defined by app rabbit 2025-03-20 00:45:37.299 [info] <0.301.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2923-3' 2025-03-20 00:45:37.306 [info] <0.301.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2923-3' 2025-03-20 00:45:37.306 [info] <0.301.0> Running boot step routing_ready defined by app rabbit 2025-03-20 00:45:37.306 [info] <0.301.0> Running boot step pre_flight defined by app rabbit 2025-03-20 00:45:37.307 [info] <0.301.0> Running boot step notify_cluster defined by app rabbit 2025-03-20 00:45:37.307 [info] <0.301.0> Running boot step networking defined by app rabbit 2025-03-20 00:45:37.307 [info] <0.414.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-1' up 2025-03-20 00:45:37.309 [info] <0.505.0> started TCP listener on [::]:5673 2025-03-20 00:45:37.309 [info] <0.301.0> Running boot step cluster_name defined by app rabbit 2025-03-20 00:45:37.309 [info] <0.301.0> Running boot step direct_client defined by app rabbit 2025-03-20 00:45:37.309 [info] <0.301.0> Running boot step os_signal_handler defined by app rabbit 2025-03-20 00:45:37.309 [info] <0.507.0> Swapping OS signal event handler (erl_signal_server) for our own 2025-03-20 00:45:37.315 [info] <0.414.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-2' up 2025-03-20 00:45:37.363 [info] <0.559.0> Management plugin: HTTP (non-TLS) listener started on port 15673 2025-03-20 00:45:37.363 [info] <0.665.0> Statistics database started. 2025-03-20 00:45:37.363 [info] <0.664.0> Starting worker pool 'management_worker_pool' with 3 processes in it 2025-03-20 00:45:37.364 [notice] <0.106.0> Changed loghwm of /var/log/contrail/config-database-rabbitmq/rabbitmq.log to 50 2025-03-20 00:45:37.440 [info] <0.8.0> Server startup complete; 3 plugins started. * rabbitmq_management * rabbitmq_management_agent * rabbitmq_web_dispatch 2025-03-20 01:53:07.777 [info] <0.8.0> Log file opened with Lager 2025-03-20 01:53:16.564 [info] <0.8.0> Feature flags: list of feature flags found: 2025-03-20 01:53:16.564 [info] <0.8.0> Feature flags: feature flag states written to disk: yes 2025-03-20 01:53:16.788 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-03-20 01:53:16.911 [info] <0.298.0> Starting RabbitMQ 3.7.28 on Erlang 21.3.8.21 Copyright (c) 2007-2020 Pivotal Software, Inc. Licensed under the MPL. See https://www.rabbitmq.com/ 2025-03-20 01:53:16.917 [info] <0.298.0> node : contrail@cn-jenkins-deploy-platform-ansible-os-2923-3 home dir : /var/lib/rabbitmq config file(s) : /etc/rabbitmq/rabbitmq.config cookie hash : esZsDxSN6VGbi9JkMSxNZA== log(s) : /var/log/contrail/config-database-rabbitmq/rabbitmq.log : /var/log/rabbitmq/contrail@cn-jenkins-deploy-platform-ansible-os-2923-3_upgrade.log database dir : /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2923-3 2025-03-20 01:53:16.953 [info] <0.298.0> Running boot step pre_boot defined by app rabbit 2025-03-20 01:53:16.953 [info] <0.298.0> Running boot step rabbit_core_metrics defined by app rabbit 2025-03-20 01:53:16.953 [info] <0.298.0> Running boot step rabbit_alarm defined by app rabbit 2025-03-20 01:53:16.989 [info] <0.321.0> Memory high watermark set to 12637 MiB (13251077734 bytes) of 15796 MiB (16563847168 bytes) total 2025-03-20 01:53:17.016 [info] <0.347.0> Enabling free disk space monitoring 2025-03-20 01:53:17.016 [info] <0.347.0> Disk free limit set to 50MB 2025-03-20 01:53:17.044 [info] <0.298.0> Running boot step code_server_cache defined by app rabbit 2025-03-20 01:53:17.044 [info] <0.298.0> Running boot step file_handle_cache defined by app rabbit 2025-03-20 01:53:17.044 [info] <0.382.0> FHC read buffering: OFF 2025-03-20 01:53:17.044 [info] <0.382.0> FHC write buffering: ON 2025-03-20 01:53:17.044 [info] <0.381.0> Limiting to approx 65436 file handles (58890 sockets) 2025-03-20 01:53:17.054 [info] <0.298.0> Running boot step worker_pool defined by app rabbit 2025-03-20 01:53:17.054 [info] <0.299.0> Will use 4 processes for default worker pool 2025-03-20 01:53:17.054 [info] <0.299.0> Starting worker pool 'worker_pool' with 4 processes in it 2025-03-20 01:53:17.055 [info] <0.298.0> Running boot step database defined by app rabbit 2025-03-20 01:53:17.079 [info] <0.298.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-03-20 01:53:17.084 [info] <0.298.0> Successfully synced tables from a peer 2025-03-20 01:53:17.087 [info] <0.298.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-03-20 01:53:17.088 [info] <0.298.0> Successfully synced tables from a peer 2025-03-20 01:53:17.184 [info] <0.298.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-03-20 01:53:17.184 [info] <0.298.0> Successfully synced tables from a peer 2025-03-20 01:53:17.184 [info] <0.298.0> Will register with peer discovery backend rabbit_peer_discovery_classic_config 2025-03-20 01:53:17.185 [info] <0.298.0> Running boot step database_sync defined by app rabbit 2025-03-20 01:53:17.185 [info] <0.298.0> Running boot step feature_flags defined by app rabbit 2025-03-20 01:53:17.185 [info] <0.298.0> Running boot step codec_correctness_check defined by app rabbit 2025-03-20 01:53:17.185 [info] <0.298.0> Running boot step external_infrastructure defined by app rabbit 2025-03-20 01:53:17.185 [info] <0.298.0> Running boot step rabbit_registry defined by app rabbit 2025-03-20 01:53:17.185 [info] <0.298.0> Running boot step rabbit_auth_mechanism_cr_demo defined by app rabbit 2025-03-20 01:53:17.185 [info] <0.298.0> Running boot step rabbit_queue_location_random defined by app rabbit 2025-03-20 01:53:17.185 [info] <0.298.0> Running boot step rabbit_event defined by app rabbit 2025-03-20 01:53:17.185 [info] <0.298.0> Running boot step rabbit_auth_mechanism_amqplain defined by app rabbit 2025-03-20 01:53:17.185 [info] <0.298.0> Running boot step rabbit_auth_mechanism_plain defined by app rabbit 2025-03-20 01:53:17.185 [info] <0.298.0> Running boot step rabbit_exchange_type_direct defined by app rabbit 2025-03-20 01:53:17.185 [info] <0.298.0> Running boot step rabbit_exchange_type_fanout defined by app rabbit 2025-03-20 01:53:17.185 [info] <0.298.0> Running boot step rabbit_exchange_type_headers defined by app rabbit 2025-03-20 01:53:17.186 [info] <0.298.0> Running boot step rabbit_exchange_type_topic defined by app rabbit 2025-03-20 01:53:17.186 [info] <0.298.0> Running boot step rabbit_mirror_queue_mode_all defined by app rabbit 2025-03-20 01:53:17.186 [info] <0.298.0> Running boot step rabbit_mirror_queue_mode_exactly defined by app rabbit 2025-03-20 01:53:17.186 [info] <0.298.0> Running boot step rabbit_mirror_queue_mode_nodes defined by app rabbit 2025-03-20 01:53:17.186 [info] <0.298.0> Running boot step rabbit_priority_queue defined by app rabbit 2025-03-20 01:53:17.186 [info] <0.298.0> Priority queues enabled, real BQ is rabbit_variable_queue 2025-03-20 01:53:17.186 [info] <0.298.0> Running boot step rabbit_queue_location_client_local defined by app rabbit 2025-03-20 01:53:17.186 [info] <0.298.0> Running boot step rabbit_queue_location_min_masters defined by app rabbit 2025-03-20 01:53:17.186 [info] <0.298.0> Running boot step kernel_ready defined by app rabbit 2025-03-20 01:53:17.186 [info] <0.298.0> Running boot step rabbit_sysmon_minder defined by app rabbit 2025-03-20 01:53:17.186 [info] <0.298.0> Running boot step rabbit_epmd_monitor defined by app rabbit 2025-03-20 01:53:17.200 [info] <0.407.0> epmd monitor knows us, inter-node communication (distribution) port: 25673 2025-03-20 01:53:17.200 [info] <0.298.0> Running boot step guid_generator defined by app rabbit 2025-03-20 01:53:17.304 [info] <0.298.0> Running boot step rabbit_node_monitor defined by app rabbit 2025-03-20 01:53:17.309 [info] <0.411.0> Starting rabbit_node_monitor 2025-03-20 01:53:17.309 [info] <0.298.0> Running boot step delegate_sup defined by app rabbit 2025-03-20 01:53:17.310 [info] <0.298.0> Running boot step rabbit_memory_monitor defined by app rabbit 2025-03-20 01:53:17.310 [info] <0.298.0> Running boot step core_initialized defined by app rabbit 2025-03-20 01:53:17.310 [info] <0.298.0> Running boot step upgrade_queues defined by app rabbit 2025-03-20 01:53:17.382 [info] <0.298.0> Running boot step rabbit_connection_tracking defined by app rabbit 2025-03-20 01:53:17.382 [info] <0.298.0> Running boot step rabbit_connection_tracking_handler defined by app rabbit 2025-03-20 01:53:17.382 [info] <0.298.0> Running boot step rabbit_exchange_parameters defined by app rabbit 2025-03-20 01:53:17.382 [info] <0.298.0> Running boot step rabbit_mirror_queue_misc defined by app rabbit 2025-03-20 01:53:17.383 [info] <0.298.0> Running boot step rabbit_policies defined by app rabbit 2025-03-20 01:53:17.389 [info] <0.298.0> Running boot step rabbit_policy defined by app rabbit 2025-03-20 01:53:17.389 [info] <0.298.0> Running boot step rabbit_queue_location_validator defined by app rabbit 2025-03-20 01:53:17.389 [info] <0.298.0> Running boot step rabbit_vhost_limit defined by app rabbit 2025-03-20 01:53:17.389 [info] <0.298.0> Running boot step rabbit_mgmt_reset_handler defined by app rabbitmq_management 2025-03-20 01:53:17.389 [info] <0.298.0> Running boot step rabbit_mgmt_db_handler defined by app rabbitmq_management_agent 2025-03-20 01:53:17.390 [warning] <0.298.0> force_fine_statistics set to true; ignored. Replaced by {rates_mode, none} in the rabbitmq_management application. 2025-03-20 01:53:17.390 [info] <0.298.0> Management plugin: using rates mode 'basic' 2025-03-20 01:53:17.403 [info] <0.298.0> Running boot step recovery defined by app rabbit 2025-03-20 01:53:17.404 [info] <0.448.0> Making sure data directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2923-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists 2025-03-20 01:53:17.419 [info] <0.448.0> Starting message stores for vhost '/' 2025-03-20 01:53:17.419 [info] <0.452.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index 2025-03-20 01:53:17.420 [info] <0.448.0> Started message store of type transient for vhost '/' 2025-03-20 01:53:17.420 [info] <0.455.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index 2025-03-20 01:53:17.431 [info] <0.448.0> Started message store of type persistent for vhost '/' 2025-03-20 01:53:17.433 [info] <0.448.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.474.0> 2025-03-20 01:53:17.433 [info] <0.448.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2923-1' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.478.0> 2025-03-20 01:53:17.433 [info] <0.448.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-1.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.482.0> 2025-03-20 01:53:17.433 [info] <0.448.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2923-3' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.486.0> 2025-03-20 01:53:17.434 [info] <0.448.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.490.0> 2025-03-20 01:53:17.434 [info] <0.448.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.494.0> 2025-03-20 01:53:17.434 [info] <0.448.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.498.0> 2025-03-20 01:53:17.434 [info] <0.448.0> Mirrored queue 'job_request_consumer' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.502.0> 2025-03-20 01:53:17.435 [info] <0.448.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-2..config_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.506.0> 2025-03-20 01:53:17.435 [info] <0.448.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-3.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.510.0> 2025-03-20 01:53:17.435 [info] <0.448.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2923-3' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.514.0> 2025-03-20 01:53:17.435 [info] <0.448.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-2..tftp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.518.0> 2025-03-20 01:53:17.436 [info] <0.448.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-1.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.522.0> 2025-03-20 01:53:17.436 [info] <0.448.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2923-2' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.526.0> 2025-03-20 01:53:17.436 [info] <0.448.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-2.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.530.0> 2025-03-20 01:53:17.436 [info] <0.448.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-3.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.534.0> 2025-03-20 01:53:17.437 [info] <0.448.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2923-3.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.538.0> 2025-03-20 01:53:17.437 [info] <0.448.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2923-1' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.542.0> 2025-03-20 01:53:17.437 [info] <0.448.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2923-1..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.546.0> 2025-03-20 01:53:17.437 [info] <0.448.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2923-1.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.550.0> 2025-03-20 01:53:17.437 [info] <0.448.0> Mirrored queue 'device_manager_ztp.ztp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.554.0> 2025-03-20 01:53:17.437 [info] <0.448.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2923-2' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.558.0> 2025-03-20 01:53:17.438 [info] <0.448.0> Mirrored queue 'device_manager.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.562.0> 2025-03-20 01:53:17.438 [info] <0.448.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-2.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.566.0> 2025-03-20 01:53:17.438 [info] <0.448.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-1.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.570.0> 2025-03-20 01:53:17.438 [info] <0.448.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2923-2.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.574.0> 2025-03-20 01:53:17.470 [info] <0.298.0> Running boot step load_definitions defined by app rabbitmq_management 2025-03-20 01:53:17.471 [info] <0.298.0> Applying definitions from /etc/rabbitmq/definitions.json 2025-03-20 01:53:17.476 [info] <0.298.0> Asked to import definitions. Acting user: rmq-internal 2025-03-20 01:53:17.476 [info] <0.298.0> Importing users... 2025-03-20 01:53:17.501 [info] <0.298.0> Setting user tags for user 'guest' to [administrator] 2025-03-20 01:53:17.533 [info] <0.298.0> Importing vhosts... 2025-03-20 01:53:17.533 [info] <0.298.0> Importing user permissions... 2025-03-20 01:53:17.533 [info] <0.298.0> Setting permissions for 'guest' in '/' to '.*', '.*', '.*' 2025-03-20 01:53:17.578 [info] <0.298.0> Importing topic permissions... 2025-03-20 01:53:17.578 [info] <0.298.0> Importing parameters... 2025-03-20 01:53:17.578 [info] <0.298.0> Importing global parameters... 2025-03-20 01:53:17.578 [info] <0.298.0> Importing policies... 2025-03-20 01:53:17.609 [info] <0.298.0> Importing queues... 2025-03-20 01:53:17.609 [info] <0.298.0> Importing exchanges... 2025-03-20 01:53:17.609 [info] <0.298.0> Importing bindings... 2025-03-20 01:53:17.609 [info] <0.298.0> Running boot step empty_db_check defined by app rabbit 2025-03-20 01:53:17.609 [info] <0.298.0> Running boot step rabbit_looking_glass defined by app rabbit 2025-03-20 01:53:17.609 [info] <0.298.0> Running boot step rabbit_core_metrics_gc defined by app rabbit 2025-03-20 01:53:17.609 [info] <0.298.0> Running boot step background_gc defined by app rabbit 2025-03-20 01:53:17.609 [info] <0.298.0> Running boot step connection_tracking defined by app rabbit 2025-03-20 01:53:17.616 [info] <0.298.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2923-3' 2025-03-20 01:53:17.624 [info] <0.298.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2923-3' 2025-03-20 01:53:17.629 [info] <0.298.0> Running boot step routing_ready defined by app rabbit 2025-03-20 01:53:17.630 [info] <0.298.0> Running boot step pre_flight defined by app rabbit 2025-03-20 01:53:17.632 [info] <0.298.0> Running boot step notify_cluster defined by app rabbit 2025-03-20 01:53:17.632 [info] <0.298.0> Running boot step networking defined by app rabbit 2025-03-20 01:53:17.632 [info] <0.411.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-1' up 2025-03-20 01:53:17.643 [info] <0.678.0> started TCP listener on [::]:5673 2025-03-20 01:53:17.646 [info] <0.298.0> Running boot step cluster_name defined by app rabbit 2025-03-20 01:53:17.646 [info] <0.298.0> Running boot step direct_client defined by app rabbit 2025-03-20 01:53:17.646 [info] <0.298.0> Running boot step os_signal_handler defined by app rabbit 2025-03-20 01:53:17.646 [info] <0.680.0> Swapping OS signal event handler (erl_signal_server) for our own 2025-03-20 01:53:17.792 [info] <0.731.0> Management plugin: HTTP (non-TLS) listener started on port 15673 2025-03-20 01:53:17.793 [info] <0.863.0> Statistics database started. 2025-03-20 01:53:17.793 [info] <0.862.0> Starting worker pool 'management_worker_pool' with 3 processes in it 2025-03-20 01:53:17.795 [notice] <0.110.0> Changed loghwm of /var/log/contrail/config-database-rabbitmq/rabbitmq.log to 50 2025-03-20 01:53:18.355 [info] <0.8.0> Server startup complete; 3 plugins started. * rabbitmq_management * rabbitmq_web_dispatch * rabbitmq_management_agent 2025-03-20 01:53:20.994 [info] <0.912.0> RabbitMQ is asked to stop... 2025-03-20 01:53:21.135 [info] <0.912.0> Stopping RabbitMQ applications and their dependencies in the following order: rabbitmq_management amqp_client rabbitmq_web_dispatch cowboy cowlib rabbitmq_management_agent rabbit mnesia rabbit_common sysmon_handler os_mon 2025-03-20 01:53:21.135 [info] <0.912.0> Stopping application 'rabbitmq_management' 2025-03-20 01:53:21.139 [warning] <0.727.0> RabbitMQ HTTP listener registry could not find context rabbitmq_management_tls 2025-03-20 01:53:21.140 [info] <0.912.0> Stopping application 'amqp_client' 2025-03-20 01:53:21.140 [info] <0.43.0> Application rabbitmq_management exited with reason: stopped 2025-03-20 01:53:21.142 [info] <0.43.0> Application amqp_client exited with reason: stopped 2025-03-20 01:53:21.142 [info] <0.912.0> Stopping application 'rabbitmq_web_dispatch' 2025-03-20 01:53:21.143 [info] <0.912.0> Stopping application 'cowboy' 2025-03-20 01:53:21.143 [info] <0.43.0> Application rabbitmq_web_dispatch exited with reason: stopped 2025-03-20 01:53:21.148 [info] <0.43.0> Application cowboy exited with reason: stopped 2025-03-20 01:53:21.148 [info] <0.912.0> Stopping application 'cowlib' 2025-03-20 01:53:21.148 [info] <0.43.0> Application cowlib exited with reason: stopped 2025-03-20 01:53:21.148 [info] <0.912.0> Stopping application 'rabbitmq_management_agent' 2025-03-20 01:53:21.150 [info] <0.912.0> Stopping application 'rabbit' 2025-03-20 01:53:21.150 [info] <0.298.0> Will unregister with peer discovery backend rabbit_peer_discovery_classic_config 2025-03-20 01:53:21.150 [info] <0.43.0> Application rabbitmq_management_agent exited with reason: stopped 2025-03-20 01:53:21.151 [info] <0.678.0> stopped TCP listener on [::]:5673 2025-03-20 01:53:21.151 [info] <0.436.0> Closing all connections in vhost '/' on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3' because the vhost is stopping 2025-03-20 01:53:21.152 [info] <0.455.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2923-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' 2025-03-20 01:53:21.168 [info] <0.455.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2923-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' is stopped 2025-03-20 01:53:21.168 [info] <0.452.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2923-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' 2025-03-20 01:53:21.177 [info] <0.452.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2923-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' is stopped 2025-03-20 01:53:21.186 [info] <0.43.0> Application rabbit exited with reason: stopped 2025-03-20 01:53:21.186 [info] <0.912.0> Stopping application 'mnesia' 2025-03-20 01:53:21.195 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-03-20 01:53:21.195 [info] <0.912.0> Stopping application 'rabbit_common' 2025-03-20 01:53:21.195 [info] <0.43.0> Application rabbit_common exited with reason: stopped 2025-03-20 01:53:21.195 [info] <0.912.0> Stopping application 'sysmon_handler' 2025-03-20 01:53:21.198 [info] <0.43.0> Application sysmon_handler exited with reason: stopped 2025-03-20 01:53:21.198 [info] <0.912.0> Stopping application 'os_mon' 2025-03-20 01:53:21.199 [info] <0.43.0> Application os_mon exited with reason: stopped 2025-03-20 01:53:21.199 [info] <0.912.0> Successfully stopped RabbitMQ and its dependencies 2025-03-20 01:53:21.199 [info] <0.912.0> Halting Erlang VM with the following applications: jsx ranch ssl public_key asn1 stdout_formatter xmerl inets tools observer_cli recon credentials_obfuscation crypto lager goldrush compiler syntax_tools sasl stdlib kernel 2025-03-20 01:53:40.951 [info] <0.8.0> Log file opened with Lager 2025-03-20 01:53:49.470 [info] <0.8.0> Feature flags: list of feature flags found: 2025-03-20 01:53:49.471 [info] <0.8.0> Feature flags: feature flag states written to disk: yes 2025-03-20 01:53:49.718 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-03-20 01:53:49.926 [info] <0.311.0> Starting RabbitMQ 3.7.28 on Erlang 21.3.8.21 Copyright (c) 2007-2020 Pivotal Software, Inc. Licensed under the MPL. See https://www.rabbitmq.com/ 2025-03-20 01:53:49.929 [info] <0.311.0> node : contrail@cn-jenkins-deploy-platform-ansible-os-2923-3 home dir : /var/lib/rabbitmq config file(s) : /etc/rabbitmq/rabbitmq.config cookie hash : esZsDxSN6VGbi9JkMSxNZA== log(s) : /var/log/contrail/config-database-rabbitmq/rabbitmq.log : /var/log/rabbitmq/contrail@cn-jenkins-deploy-platform-ansible-os-2923-3_upgrade.log database dir : /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2923-3 2025-03-20 01:53:49.951 [info] <0.311.0> Running boot step pre_boot defined by app rabbit 2025-03-20 01:53:49.951 [info] <0.311.0> Running boot step rabbit_core_metrics defined by app rabbit 2025-03-20 01:53:49.952 [info] <0.311.0> Running boot step rabbit_alarm defined by app rabbit 2025-03-20 01:53:49.962 [info] <0.326.0> Memory high watermark set to 12637 MiB (13251077734 bytes) of 15796 MiB (16563847168 bytes) total 2025-03-20 01:53:49.967 [info] <0.331.0> Enabling free disk space monitoring 2025-03-20 01:53:49.967 [info] <0.331.0> Disk free limit set to 50MB 2025-03-20 01:53:49.976 [info] <0.311.0> Running boot step code_server_cache defined by app rabbit 2025-03-20 01:53:49.976 [info] <0.311.0> Running boot step file_handle_cache defined by app rabbit 2025-03-20 01:53:49.977 [info] <0.349.0> Limiting to approx 65436 file handles (58890 sockets) 2025-03-20 01:53:49.977 [info] <0.352.0> FHC read buffering: OFF 2025-03-20 01:53:49.977 [info] <0.352.0> FHC write buffering: ON 2025-03-20 01:53:49.977 [info] <0.311.0> Running boot step worker_pool defined by app rabbit 2025-03-20 01:53:49.977 [info] <0.312.0> Will use 4 processes for default worker pool 2025-03-20 01:53:49.977 [info] <0.312.0> Starting worker pool 'worker_pool' with 4 processes in it 2025-03-20 01:53:49.977 [info] <0.311.0> Running boot step database defined by app rabbit 2025-03-20 01:53:50.059 [info] <0.311.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-03-20 01:53:50.078 [info] <0.311.0> Successfully synced tables from a peer 2025-03-20 01:53:50.081 [info] <0.311.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-03-20 01:53:50.081 [info] <0.311.0> Successfully synced tables from a peer 2025-03-20 01:53:50.124 [info] <0.311.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-03-20 01:53:50.124 [info] <0.311.0> Successfully synced tables from a peer 2025-03-20 01:53:50.124 [info] <0.311.0> Will register with peer discovery backend rabbit_peer_discovery_classic_config 2025-03-20 01:53:50.124 [info] <0.311.0> Running boot step database_sync defined by app rabbit 2025-03-20 01:53:50.124 [info] <0.311.0> Running boot step feature_flags defined by app rabbit 2025-03-20 01:53:50.125 [info] <0.311.0> Running boot step codec_correctness_check defined by app rabbit 2025-03-20 01:53:50.125 [info] <0.311.0> Running boot step external_infrastructure defined by app rabbit 2025-03-20 01:53:50.125 [info] <0.311.0> Running boot step rabbit_registry defined by app rabbit 2025-03-20 01:53:50.125 [info] <0.311.0> Running boot step rabbit_auth_mechanism_cr_demo defined by app rabbit 2025-03-20 01:53:50.125 [info] <0.311.0> Running boot step rabbit_queue_location_random defined by app rabbit 2025-03-20 01:53:50.125 [info] <0.311.0> Running boot step rabbit_event defined by app rabbit 2025-03-20 01:53:50.125 [info] <0.311.0> Running boot step rabbit_auth_mechanism_amqplain defined by app rabbit 2025-03-20 01:53:50.125 [info] <0.311.0> Running boot step rabbit_auth_mechanism_plain defined by app rabbit 2025-03-20 01:53:50.125 [info] <0.311.0> Running boot step rabbit_exchange_type_direct defined by app rabbit 2025-03-20 01:53:50.125 [info] <0.311.0> Running boot step rabbit_exchange_type_fanout defined by app rabbit 2025-03-20 01:53:50.125 [info] <0.311.0> Running boot step rabbit_exchange_type_headers defined by app rabbit 2025-03-20 01:53:50.125 [info] <0.311.0> Running boot step rabbit_exchange_type_topic defined by app rabbit 2025-03-20 01:53:50.126 [info] <0.311.0> Running boot step rabbit_mirror_queue_mode_all defined by app rabbit 2025-03-20 01:53:50.126 [info] <0.311.0> Running boot step rabbit_mirror_queue_mode_exactly defined by app rabbit 2025-03-20 01:53:50.126 [info] <0.311.0> Running boot step rabbit_mirror_queue_mode_nodes defined by app rabbit 2025-03-20 01:53:50.126 [info] <0.311.0> Running boot step rabbit_priority_queue defined by app rabbit 2025-03-20 01:53:50.126 [info] <0.311.0> Priority queues enabled, real BQ is rabbit_variable_queue 2025-03-20 01:53:50.126 [info] <0.311.0> Running boot step rabbit_queue_location_client_local defined by app rabbit 2025-03-20 01:53:50.126 [info] <0.311.0> Running boot step rabbit_queue_location_min_masters defined by app rabbit 2025-03-20 01:53:50.126 [info] <0.311.0> Running boot step kernel_ready defined by app rabbit 2025-03-20 01:53:50.126 [info] <0.311.0> Running boot step rabbit_sysmon_minder defined by app rabbit 2025-03-20 01:53:50.126 [info] <0.311.0> Running boot step rabbit_epmd_monitor defined by app rabbit 2025-03-20 01:53:50.139 [info] <0.414.0> epmd monitor knows us, inter-node communication (distribution) port: 25673 2025-03-20 01:53:50.140 [info] <0.311.0> Running boot step guid_generator defined by app rabbit 2025-03-20 01:53:50.148 [info] <0.311.0> Running boot step rabbit_node_monitor defined by app rabbit 2025-03-20 01:53:50.151 [info] <0.418.0> Starting rabbit_node_monitor 2025-03-20 01:53:50.151 [info] <0.311.0> Running boot step delegate_sup defined by app rabbit 2025-03-20 01:53:50.151 [info] <0.311.0> Running boot step rabbit_memory_monitor defined by app rabbit 2025-03-20 01:53:50.151 [info] <0.311.0> Running boot step core_initialized defined by app rabbit 2025-03-20 01:53:50.151 [info] <0.311.0> Running boot step upgrade_queues defined by app rabbit 2025-03-20 01:53:50.186 [info] <0.311.0> Running boot step rabbit_connection_tracking defined by app rabbit 2025-03-20 01:53:50.186 [info] <0.311.0> Running boot step rabbit_connection_tracking_handler defined by app rabbit 2025-03-20 01:53:50.186 [info] <0.311.0> Running boot step rabbit_exchange_parameters defined by app rabbit 2025-03-20 01:53:50.186 [info] <0.311.0> Running boot step rabbit_mirror_queue_misc defined by app rabbit 2025-03-20 01:53:50.187 [info] <0.311.0> Running boot step rabbit_policies defined by app rabbit 2025-03-20 01:53:50.187 [info] <0.311.0> Running boot step rabbit_policy defined by app rabbit 2025-03-20 01:53:50.187 [info] <0.311.0> Running boot step rabbit_queue_location_validator defined by app rabbit 2025-03-20 01:53:50.187 [info] <0.311.0> Running boot step rabbit_vhost_limit defined by app rabbit 2025-03-20 01:53:50.187 [info] <0.311.0> Running boot step rabbit_mgmt_reset_handler defined by app rabbitmq_management 2025-03-20 01:53:50.187 [info] <0.311.0> Running boot step rabbit_mgmt_db_handler defined by app rabbitmq_management_agent 2025-03-20 01:53:50.187 [warning] <0.311.0> force_fine_statistics set to true; ignored. Replaced by {rates_mode, none} in the rabbitmq_management application. 2025-03-20 01:53:50.187 [info] <0.311.0> Management plugin: using rates mode 'basic' 2025-03-20 01:53:50.194 [info] <0.311.0> Running boot step recovery defined by app rabbit 2025-03-20 01:53:50.196 [info] <0.455.0> Making sure data directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2923-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists 2025-03-20 01:53:50.218 [info] <0.455.0> Starting message stores for vhost '/' 2025-03-20 01:53:50.219 [info] <0.459.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index 2025-03-20 01:53:50.220 [info] <0.455.0> Started message store of type transient for vhost '/' 2025-03-20 01:53:50.220 [info] <0.462.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index 2025-03-20 01:53:50.222 [info] <0.455.0> Started message store of type persistent for vhost '/' 2025-03-20 01:53:50.224 [info] <0.455.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.481.0> 2025-03-20 01:53:50.224 [info] <0.455.0> Mirrored queue 'job_status_consumer.dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.485.0> 2025-03-20 01:53:50.225 [info] <0.455.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2923-1' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.489.0> 2025-03-20 01:53:50.225 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-1.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.493.0> 2025-03-20 01:53:50.225 [info] <0.455.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2923-3' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.497.0> 2025-03-20 01:53:50.225 [info] <0.455.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.501.0> 2025-03-20 01:53:50.225 [info] <0.455.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.505.0> 2025-03-20 01:53:50.227 [info] <0.455.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.508.0> 2025-03-20 01:53:50.228 [info] <0.455.0> Mirrored queue 'job_request_consumer' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.512.0> 2025-03-20 01:53:50.228 [info] <0.455.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-2..config_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.516.0> 2025-03-20 01:53:50.232 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-3.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.520.0> 2025-03-20 01:53:50.233 [info] <0.455.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2923-3' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.532.0> 2025-03-20 01:53:50.234 [info] <0.455.0> Mirrored queue 'svc_monitor.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.538.0> 2025-03-20 01:53:50.234 [info] <0.455.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-2..tftp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.542.0> 2025-03-20 01:53:50.234 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-1.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.546.0> 2025-03-20 01:53:50.235 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-2.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.550.0> 2025-03-20 01:53:50.235 [info] <0.455.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2923-2' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.554.0> 2025-03-20 01:53:50.236 [info] <0.455.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.560.0> 2025-03-20 01:53:50.236 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-2.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.565.0> 2025-03-20 01:53:50.237 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-3.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.571.0> 2025-03-20 01:53:50.237 [info] <0.455.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2923-3.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.575.0> 2025-03-20 01:53:50.240 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-3.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.579.0> 2025-03-20 01:53:50.241 [info] <0.455.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2923-1' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.587.0> 2025-03-20 01:53:50.242 [info] <0.455.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2923-2..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.592.0> 2025-03-20 01:53:50.242 [info] <0.455.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2923-1..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.596.0> 2025-03-20 01:53:50.243 [info] <0.455.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2923-1.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.602.0> 2025-03-20 01:53:50.243 [info] <0.455.0> Mirrored queue 'device_manager_ztp.ztp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.606.0> 2025-03-20 01:53:50.245 [info] <0.455.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2923-3..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.610.0> 2025-03-20 01:53:50.246 [info] <0.455.0> Mirrored queue 'schema_transformer.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.614.0> 2025-03-20 01:53:50.246 [info] <0.455.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2923-2' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.618.0> 2025-03-20 01:53:50.247 [info] <0.455.0> Mirrored queue 'device_manager.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.622.0> 2025-03-20 01:53:50.248 [info] <0.455.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-1..tftp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.630.0> 2025-03-20 01:53:50.249 [info] <0.455.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-1..config_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.635.0> 2025-03-20 01:53:50.250 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-2.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.639.0> 2025-03-20 01:53:50.250 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-1.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.643.0> 2025-03-20 01:53:50.251 [info] <0.455.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2923-2.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.647.0> 2025-03-20 01:53:50.251 [info] <0.311.0> Running boot step load_definitions defined by app rabbitmq_management 2025-03-20 01:53:50.253 [info] <0.311.0> Applying definitions from /etc/rabbitmq/definitions.json 2025-03-20 01:53:50.253 [info] <0.311.0> Asked to import definitions. Acting user: rmq-internal 2025-03-20 01:53:50.253 [info] <0.311.0> Importing users... 2025-03-20 01:53:50.312 [info] <0.311.0> Setting user tags for user 'guest' to [administrator] 2025-03-20 01:53:50.395 [info] <0.311.0> Importing vhosts... 2025-03-20 01:53:50.396 [info] <0.311.0> Importing user permissions... 2025-03-20 01:53:50.397 [info] <0.311.0> Setting permissions for 'guest' in '/' to '.*', '.*', '.*' 2025-03-20 01:53:50.429 [info] <0.311.0> Importing topic permissions... 2025-03-20 01:53:50.430 [info] <0.311.0> Importing parameters... 2025-03-20 01:53:50.430 [info] <0.311.0> Importing global parameters... 2025-03-20 01:53:50.430 [info] <0.311.0> Importing policies... 2025-03-20 01:53:50.443 [info] <0.311.0> Importing queues... 2025-03-20 01:53:50.443 [info] <0.311.0> Importing exchanges... 2025-03-20 01:53:50.443 [info] <0.311.0> Importing bindings... 2025-03-20 01:53:50.443 [info] <0.311.0> Running boot step empty_db_check defined by app rabbit 2025-03-20 01:53:50.443 [info] <0.311.0> Running boot step rabbit_looking_glass defined by app rabbit 2025-03-20 01:53:50.443 [info] <0.311.0> Running boot step rabbit_core_metrics_gc defined by app rabbit 2025-03-20 01:53:50.443 [info] <0.311.0> Running boot step background_gc defined by app rabbit 2025-03-20 01:53:50.444 [info] <0.311.0> Running boot step connection_tracking defined by app rabbit 2025-03-20 01:53:50.450 [info] <0.311.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2923-3' 2025-03-20 01:53:50.471 [info] <0.311.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2923-3' 2025-03-20 01:53:50.471 [info] <0.311.0> Running boot step routing_ready defined by app rabbit 2025-03-20 01:53:50.471 [info] <0.311.0> Running boot step pre_flight defined by app rabbit 2025-03-20 01:53:50.471 [info] <0.311.0> Running boot step notify_cluster defined by app rabbit 2025-03-20 01:53:50.471 [info] <0.311.0> Running boot step networking defined by app rabbit 2025-03-20 01:53:50.472 [info] <0.418.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-1' up 2025-03-20 01:53:50.482 [info] <0.755.0> started TCP listener on [::]:5673 2025-03-20 01:53:50.482 [info] <0.311.0> Running boot step cluster_name defined by app rabbit 2025-03-20 01:53:50.488 [info] <0.311.0> Running boot step direct_client defined by app rabbit 2025-03-20 01:53:50.490 [info] <0.311.0> Running boot step os_signal_handler defined by app rabbit 2025-03-20 01:53:50.490 [info] <0.757.0> Swapping OS signal event handler (erl_signal_server) for our own 2025-03-20 01:53:50.715 [info] <0.808.0> Management plugin: HTTP (non-TLS) listener started on port 15673 2025-03-20 01:53:50.715 [info] <0.950.0> Statistics database started. 2025-03-20 01:53:50.715 [info] <0.949.0> Starting worker pool 'management_worker_pool' with 3 processes in it 2025-03-20 01:53:50.716 [notice] <0.106.0> Changed loghwm of /var/log/contrail/config-database-rabbitmq/rabbitmq.log to 50 2025-03-20 01:53:51.186 [info] <0.8.0> Server startup complete; 3 plugins started. * rabbitmq_management * rabbitmq_web_dispatch * rabbitmq_management_agent 2025-03-20 01:54:10.711 [info] <0.418.0> node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-2' up 2025-03-20 01:54:11.685 [info] <0.418.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-2' up 2025-03-20 02:00:41.287 [info] <0.418.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-1' down 2025-03-20 02:00:41.298 [info] <0.418.0> Keeping contrail@cn-jenkins-deploy-platform-ansible-os-2923-1 listeners: the node is already back 2025-03-20 02:00:41.309 [info] <0.1541.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-3.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Slave saw deaths of mirrors 2025-03-20 02:00:41.309 [info] <0.560.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Slave saw deaths of mirrors 2025-03-20 02:00:41.309 [info] <0.1518.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-1.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Slave saw deaths of mirrors 2025-03-20 02:00:41.309 [info] <0.560.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Promoting slave to master 2025-03-20 02:00:41.309 [info] <0.560.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:00:41.310 [info] <0.560.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:00:41.311 [info] <0.1741.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Synchronising: all slaves already synced 2025-03-20 02:00:41.331 [info] <0.418.0> node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-1' down: connection_closed 2025-03-20 02:00:41.368 [info] <0.1402.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-3..config_queue' in vhost '/': Slave saw deaths of mirrors 2025-03-20 02:00:41.373 [info] <0.1693.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2923-3' in vhost '/': Slave saw deaths of mirrors 2025-03-20 02:00:41.374 [info] <0.1565.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-3.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Slave saw deaths of mirrors 2025-03-20 02:00:41.374 [info] <0.542.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-2..tftp_queue' in vhost '/': Slave saw deaths of mirrors 2025-03-20 02:00:41.374 [info] <0.542.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-2..tftp_queue' in vhost '/': Promoting slave to master 2025-03-20 02:00:41.375 [info] <0.542.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-2..tftp_queue' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:00:41.375 [info] <0.542.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-2..tftp_queue' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:00:41.375 [info] <0.1717.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2923-3' in vhost '/': Slave saw deaths of mirrors 2025-03-20 02:00:41.376 [info] <0.1614.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2923-1' in vhost '/': Slave saw deaths of mirrors 2025-03-20 02:00:41.376 [info] <0.1298.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Slave saw deaths of mirrors 2025-03-20 02:00:41.376 [info] <0.635.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-1..config_queue' in vhost '/': Slave saw deaths of mirrors 2025-03-20 02:00:41.376 [info] <0.635.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-1..config_queue' in vhost '/': Promoting slave to master 2025-03-20 02:00:41.376 [info] <0.635.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-1..config_queue' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:00:41.376 [info] <0.635.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-1..config_queue' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:00:41.384 [info] <0.1484.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Slave saw deaths of mirrors 2025-03-20 02:00:41.385 [info] <0.512.0> Mirrored queue 'job_request_consumer' in vhost '/': Slave saw deaths of mirrors 2025-03-20 02:00:41.385 [info] <0.512.0> Mirrored queue 'job_request_consumer' in vhost '/': Promoting slave to master 2025-03-20 02:00:41.385 [info] <0.512.0> Mirrored queue 'job_request_consumer' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:00:41.385 [info] <0.512.0> Mirrored queue 'job_request_consumer' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:00:41.386 [info] <0.1577.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-2.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Slave saw deaths of mirrors 2025-03-20 02:00:41.386 [info] <0.1553.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-2.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Slave saw deaths of mirrors 2025-03-20 02:00:41.388 [info] <0.1601.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-1.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Slave saw deaths of mirrors 2025-03-20 02:00:41.388 [info] <0.508.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Promoting slave to master 2025-03-20 02:00:41.389 [info] <0.508.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:00:41.389 [info] <0.508.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:00:41.389 [info] <0.1529.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-3.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Slave saw deaths of mirrors 2025-03-20 02:00:41.389 [info] <0.1903.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Synchronising: all slaves already synced 2025-03-20 02:00:41.396 [info] <0.1870.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-1..config_queue' in vhost '/': Synchronising: all slaves already synced 2025-03-20 02:00:41.398 [info] <0.516.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-2..config_queue' in vhost '/': Promoting slave to master 2025-03-20 02:00:41.398 [info] <0.516.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-2..config_queue' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:00:41.399 [info] <0.516.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-2..config_queue' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:00:41.400 [info] <0.1864.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-2..tftp_queue' in vhost '/': Synchronising: all slaves already synced 2025-03-20 02:00:41.400 [info] <0.1913.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-2..config_queue' in vhost '/': Synchronising: all slaves already synced 2025-03-20 02:00:41.401 [info] <0.1649.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2923-2' in vhost '/': Slave saw deaths of mirrors 2025-03-20 02:00:41.403 [info] <0.1891.0> Mirrored queue 'job_request_consumer' in vhost '/': Synchronising: all slaves already synced 2025-03-20 02:00:41.407 [info] <0.1413.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-3..tftp_queue' in vhost '/': Slave saw deaths of mirrors 2025-03-20 02:00:41.423 [info] <0.622.0> Mirrored queue 'device_manager.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Promoting slave to master 2025-03-20 02:00:41.423 [info] <0.606.0> Mirrored queue 'device_manager_ztp.ztp_queue' in vhost '/': Promoting slave to master 2025-03-20 02:00:41.424 [info] <0.606.0> Mirrored queue 'device_manager_ztp.ztp_queue' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:00:41.424 [info] <0.606.0> Mirrored queue 'device_manager_ztp.ztp_queue' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:00:42.137 [warning] lager_file_backend dropped 8 messages in the last second that exceeded the limit of 50 messages/sec 2025-03-20 02:01:11.454 [info] <0.418.0> node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-1' up 2025-03-20 02:01:40.767 [info] <0.508.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:01:40.767 [info] <0.508.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:01:40.769 [info] <0.2122.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Synchronising: mirrors ['contrail@cn-jenkins-deploy-platform-ansible-os-2923-1'] to sync 2025-03-20 02:01:40.769 [info] <0.508.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Synchronising: complete 2025-03-20 02:01:40.771 [info] <0.516.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-2..config_queue' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:01:40.772 [info] <0.516.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-2..config_queue' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:01:40.785 [info] <0.542.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-2..tftp_queue' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:01:40.785 [info] <0.560.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:01:40.785 [info] <0.542.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-2..tftp_queue' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:01:40.785 [info] <0.560.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:01:40.786 [info] <0.512.0> Mirrored queue 'job_request_consumer' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:01:40.786 [info] <0.512.0> Mirrored queue 'job_request_consumer' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:01:40.791 [info] <0.606.0> Mirrored queue 'device_manager_ztp.ztp_queue' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:01:40.791 [info] <0.606.0> Mirrored queue 'device_manager_ztp.ztp_queue' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:01:40.835 [info] <0.622.0> Mirrored queue 'device_manager.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Synchronising: 584 messages to synchronise 2025-03-20 02:01:40.835 [info] <0.622.0> Mirrored queue 'device_manager.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:01:40.846 [info] <0.2128.0> Mirrored queue 'device_manager.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Synchronising: mirrors ['contrail@cn-jenkins-deploy-platform-ansible-os-2923-1'] to sync 2025-03-20 02:01:40.849 [info] <0.622.0> Mirrored queue 'device_manager.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Synchronising: complete 2025-03-20 02:01:40.864 [info] <0.2123.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-2..config_queue' in vhost '/': Synchronising: all slaves already synced 2025-03-20 02:01:40.874 [info] <0.2126.0> Mirrored queue 'job_request_consumer' in vhost '/': Synchronising: all slaves already synced 2025-03-20 02:01:40.874 [info] <0.630.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-1..tftp_queue' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:01:40.874 [info] <0.630.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-1..tftp_queue' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:01:40.875 [info] <0.2125.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Synchronising: all slaves already synced 2025-03-20 02:01:40.875 [info] <0.2124.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-2..tftp_queue' in vhost '/': Synchronising: all slaves already synced 2025-03-20 02:01:40.897 [info] <0.2127.0> Mirrored queue 'device_manager_ztp.ztp_queue' in vhost '/': Synchronising: all slaves already synced 2025-03-20 02:01:40.898 [info] <0.635.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-1..config_queue' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:01:40.898 [info] <0.635.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-1..config_queue' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:01:40.909 [info] <0.2129.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-1..tftp_queue' in vhost '/': Synchronising: all slaves already synced 2025-03-20 02:01:40.914 [info] <0.2130.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-1..config_queue' in vhost '/': Synchronising: all slaves already synced 2025-03-20 02:01:40.952 [info] <0.418.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-1' up 2025-03-20 02:02:17.219 [info] <0.2168.0> accepting AMQP connection <0.2168.0> (10.0.0.12:57814 -> 10.0.0.47:5673) 2025-03-20 02:02:17.219 [info] <0.2171.0> accepting AMQP connection <0.2171.0> (10.0.0.47:40332 -> 10.0.0.47:5673) 2025-03-20 02:02:17.222 [info] <0.2174.0> accepting AMQP connection <0.2174.0> (10.0.0.47:40342 -> 10.0.0.47:5673) 2025-03-20 02:02:17.222 [info] <0.2177.0> accepting AMQP connection <0.2177.0> (10.0.0.47:40358 -> 10.0.0.47:5673) 2025-03-20 02:02:17.224 [info] <0.2171.0> connection <0.2171.0> (10.0.0.47:40332 -> 10.0.0.47:5673): user 'guest' authenticated and granted access to vhost '/' 2025-03-20 02:02:17.228 [info] <0.2189.0> accepting AMQP connection <0.2189.0> (10.0.0.12:57828 -> 10.0.0.47:5673) 2025-03-20 02:02:17.229 [info] <0.2180.0> accepting AMQP connection <0.2180.0> (10.0.0.47:40374 -> 10.0.0.47:5673) 2025-03-20 02:02:17.234 [info] <0.2192.0> accepting AMQP connection <0.2192.0> (10.0.0.42:49264 -> 10.0.0.47:5673) 2025-03-20 02:02:17.235 [info] <0.2195.0> accepting AMQP connection <0.2195.0> (10.0.0.42:49268 -> 10.0.0.47:5673) 2025-03-20 02:02:17.235 [info] <0.2198.0> accepting AMQP connection <0.2198.0> (10.0.0.12:57860 -> 10.0.0.47:5673) 2025-03-20 02:02:17.237 [info] <0.2202.0> accepting AMQP connection <0.2202.0> (10.0.0.47:40384 -> 10.0.0.47:5673) 2025-03-20 02:02:17.238 [info] <0.2205.0> accepting AMQP connection <0.2205.0> (10.0.0.12:57866 -> 10.0.0.47:5673) 2025-03-20 02:02:17.238 [info] <0.2168.0> connection <0.2168.0> (10.0.0.12:57814 -> 10.0.0.47:5673): user 'guest' authenticated and granted access to vhost '/' 2025-03-20 02:02:17.238 [info] <0.2209.0> accepting AMQP connection <0.2209.0> (10.0.0.12:57844 -> 10.0.0.47:5673) 2025-03-20 02:02:17.240 [info] <0.2215.0> accepting AMQP connection <0.2215.0> (10.0.0.47:40398 -> 10.0.0.47:5673) 2025-03-20 02:02:17.241 [info] <0.2220.0> accepting AMQP connection <0.2220.0> (10.0.0.42:49246 -> 10.0.0.47:5673) 2025-03-20 02:02:17.241 [info] <0.2223.0> accepting AMQP connection <0.2223.0> (10.0.0.42:49260 -> 10.0.0.47:5673) 2025-03-20 02:02:17.241 [info] <0.2226.0> accepting AMQP connection <0.2226.0> (10.0.0.12:57856 -> 10.0.0.47:5673) 2025-03-20 02:02:17.241 [info] <0.2229.0> accepting AMQP connection <0.2229.0> (10.0.0.47:40400 -> 10.0.0.47:5673) 2025-03-20 02:02:17.241 [info] <0.2232.0> accepting AMQP connection <0.2232.0> (10.0.0.42:49276 -> 10.0.0.47:5673) 2025-03-20 02:02:17.241 [info] <0.2236.0> accepting AMQP connection <0.2236.0> (10.0.0.42:49278 -> 10.0.0.47:5673) 2025-03-20 02:02:17.243 [info] <0.2174.0> connection <0.2174.0> (10.0.0.47:40342 -> 10.0.0.47:5673): user 'guest' authenticated and granted access to vhost '/' 2025-03-20 02:02:17.243 [info] <0.2240.0> accepting AMQP connection <0.2240.0> (10.0.0.47:40408 -> 10.0.0.47:5673) 2025-03-20 02:02:17.243 [info] <0.2243.0> accepting AMQP connection <0.2243.0> (10.0.0.12:57868 -> 10.0.0.47:5673) 2025-03-20 02:02:17.244 [info] <0.2246.0> accepting AMQP connection <0.2246.0> (10.0.0.47:40412 -> 10.0.0.47:5673) 2025-03-20 02:02:17.245 [info] <0.2195.0> connection <0.2195.0> (10.0.0.42:49268 -> 10.0.0.47:5673): user 'guest' authenticated and granted access to vhost '/' 2025-03-20 02:02:17.245 [info] <0.2253.0> accepting AMQP connection <0.2253.0> (10.0.0.47:40416 -> 10.0.0.47:5673) 2025-03-20 02:02:17.245 [info] <0.2258.0> accepting AMQP connection <0.2258.0> (10.0.0.42:49286 -> 10.0.0.47:5673) 2025-03-20 02:02:17.246 [info] <0.2189.0> connection <0.2189.0> (10.0.0.12:57828 -> 10.0.0.47:5673): user 'guest' authenticated and granted access to vhost '/' 2025-03-20 02:02:17.246 [info] <0.2192.0> connection <0.2192.0> (10.0.0.42:49264 -> 10.0.0.47:5673): user 'guest' authenticated and granted access to vhost '/' 2025-03-20 02:02:17.247 [info] <0.2180.0> connection <0.2180.0> (10.0.0.47:40374 -> 10.0.0.47:5673): user 'guest' authenticated and granted access to vhost '/' 2025-03-20 02:02:17.249 [info] <0.2209.0> connection <0.2209.0> (10.0.0.12:57844 -> 10.0.0.47:5673): user 'guest' authenticated and granted access to vhost '/' 2025-03-20 02:02:17.249 [info] <0.2205.0> connection <0.2205.0> (10.0.0.12:57866 -> 10.0.0.47:5673): user 'guest' authenticated and granted access to vhost '/' 2025-03-20 02:02:17.250 [info] <0.2198.0> connection <0.2198.0> (10.0.0.12:57860 -> 10.0.0.47:5673): user 'guest' authenticated and granted access to vhost '/' 2025-03-20 02:02:17.250 [info] <0.2177.0> connection <0.2177.0> (10.0.0.47:40358 -> 10.0.0.47:5673): user 'guest' authenticated and granted access to vhost '/' 2025-03-20 02:02:17.255 [info] <0.2246.0> connection <0.2246.0> (10.0.0.47:40412 -> 10.0.0.47:5673): user 'guest' authenticated and granted access to vhost '/' 2025-03-20 02:02:17.255 [info] <0.2220.0> connection <0.2220.0> (10.0.0.42:49246 -> 10.0.0.47:5673): user 'guest' authenticated and granted access to vhost '/' 2025-03-20 02:02:17.256 [info] <0.2236.0> connection <0.2236.0> (10.0.0.42:49278 -> 10.0.0.47:5673): user 'guest' authenticated and granted access to vhost '/' 2025-03-20 02:02:17.257 [info] <0.2240.0> connection <0.2240.0> (10.0.0.47:40408 -> 10.0.0.47:5673): user 'guest' authenticated and granted access to vhost '/' 2025-03-20 02:02:17.260 [info] <0.2243.0> connection <0.2243.0> (10.0.0.12:57868 -> 10.0.0.47:5673): user 'guest' authenticated and granted access to vhost '/' 2025-03-20 02:02:17.260 [info] <0.2229.0> connection <0.2229.0> (10.0.0.47:40400 -> 10.0.0.47:5673): user 'guest' authenticated and granted access to vhost '/' 2025-03-20 02:02:17.261 [info] <0.2202.0> connection <0.2202.0> (10.0.0.47:40384 -> 10.0.0.47:5673): user 'guest' authenticated and granted access to vhost '/' 2025-03-20 02:02:17.262 [info] <0.2223.0> connection <0.2223.0> (10.0.0.42:49260 -> 10.0.0.47:5673): user 'guest' authenticated and granted access to vhost '/' 2025-03-20 02:02:17.262 [info] <0.2232.0> connection <0.2232.0> (10.0.0.42:49276 -> 10.0.0.47:5673): user 'guest' authenticated and granted access to vhost '/' 2025-03-20 02:02:17.263 [info] <0.2226.0> connection <0.2226.0> (10.0.0.12:57856 -> 10.0.0.47:5673): user 'guest' authenticated and granted access to vhost '/' 2025-03-20 02:02:17.264 [info] <0.2215.0> connection <0.2215.0> (10.0.0.47:40398 -> 10.0.0.47:5673): user 'guest' authenticated and granted access to vhost '/' 2025-03-20 02:02:17.265 [info] <0.2258.0> connection <0.2258.0> (10.0.0.42:49286 -> 10.0.0.47:5673): user 'guest' authenticated and granted access to vhost '/' 2025-03-20 02:02:17.266 [info] <0.2253.0> connection <0.2253.0> (10.0.0.47:40416 -> 10.0.0.47:5673): user 'guest' authenticated and granted access to vhost '/' 2025-03-20 02:02:17.407 [info] <0.418.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-2' down 2025-03-20 02:02:17.418 [info] <0.418.0> Keeping contrail@cn-jenkins-deploy-platform-ansible-os-2923-2 listeners: the node is already back 2025-03-20 02:02:18.202 [warning] lager_file_backend dropped 185 messages in the last second that exceeded the limit of 50 messages/sec 2025-03-20 02:02:52.542 [info] <0.418.0> node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-2' up 2025-03-20 02:03:36.210 [info] <0.1413.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-3..tftp_queue' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:03:36.210 [info] <0.1495.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:03:36.210 [info] <0.1413.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-3..tftp_queue' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:03:36.210 [info] <0.1495.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:03:36.211 [info] <0.1589.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-1.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:03:36.211 [info] <0.1589.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-1.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:03:36.215 [info] <0.1484.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:03:36.215 [info] <0.1484.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:03:36.223 [info] <0.2476.0> Mirrored queue 'job_status_consumer.dummy' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:03:36.223 [info] <0.2476.0> Mirrored queue 'job_status_consumer.dummy' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:03:36.224 [info] <0.508.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:03:36.224 [info] <0.508.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:03:36.232 [info] <0.516.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-2..config_queue' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:03:36.232 [info] <0.516.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-2..config_queue' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:03:36.232 [info] <0.1471.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:03:36.233 [info] <0.1471.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:03:36.253 [info] <0.1693.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2923-3' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:03:36.253 [info] <0.1693.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2923-3' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:03:36.267 [info] <0.1649.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2923-2' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:03:36.267 [info] <0.1649.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2923-2' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:03:36.267 [info] <0.1507.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-2.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:03:36.268 [info] <0.1507.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-2.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:03:36.268 [info] <0.2916.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-3..tftp_queue' in vhost '/': Synchronising: all slaves already synced 2025-03-20 02:03:36.269 [info] <0.1298.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:03:36.269 [info] <0.2917.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Synchronising: all slaves already synced 2025-03-20 02:03:36.269 [info] <0.1637.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2923-1' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:03:36.269 [info] <0.1637.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2923-1' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:03:36.271 [info] <0.1298.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:03:36.276 [info] <0.2934.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Synchronising: all slaves already synced 2025-03-20 02:03:36.276 [info] <0.1192.0> Mirrored queue 'svc_monitor.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:03:36.276 [info] <0.1192.0> Mirrored queue 'svc_monitor.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:03:36.276 [info] <0.1541.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-3.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:03:36.276 [info] <0.1541.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-3.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:03:36.291 [info] <0.542.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-2..tftp_queue' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:03:36.291 [info] <0.512.0> Mirrored queue 'job_request_consumer' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:03:36.291 [info] <0.1717.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2923-3' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:03:36.291 [info] <0.542.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-2..tftp_queue' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:03:36.291 [info] <0.512.0> Mirrored queue 'job_request_consumer' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:03:36.291 [info] <0.1717.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2923-3' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:03:36.311 [info] <0.2918.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-1.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Synchronising: all slaves already synced 2025-03-20 02:03:36.314 [info] <0.1577.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-2.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:03:36.314 [info] <0.1577.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-2.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:03:36.329 [info] <0.2923.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Synchronising: all slaves already synced 2025-03-20 02:03:36.333 [info] <0.2933.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2923-1' in vhost '/': Synchronising: all slaves already synced 2025-03-20 02:03:36.333 [info] <0.1391.0> Mirrored queue 'schema_transformer.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:03:36.333 [info] <0.1391.0> Mirrored queue 'schema_transformer.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:03:36.334 [info] <0.1614.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2923-1' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:03:36.334 [info] <0.1614.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2923-1' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:03:36.335 [info] <0.2941.0> Mirrored queue 'schema_transformer.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Synchronising: all slaves already synced 2025-03-20 02:03:36.335 [info] <0.1601.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-1.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:03:37.124 [warning] lager_file_backend dropped 76 messages in the last second that exceeded the limit of 50 messages/sec 2025-03-20 02:03:38.516 [info] <0.418.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-2' down 2025-03-20 02:03:38.527 [info] <0.418.0> Keeping contrail@cn-jenkins-deploy-platform-ansible-os-2923-2 listeners: the node is already back 2025-03-20 02:03:38.562 [info] <0.1673.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2923-2' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:03:38.562 [info] <0.1673.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2923-2' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:03:38.563 [info] <0.3017.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2923-2' in vhost '/': Synchronising: all slaves already synced 2025-03-20 02:03:38.563 [info] <0.1368.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2923-2.-8082-0' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:03:38.564 [info] <0.1368.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2923-2.-8082-0' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:03:38.564 [info] <0.3020.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2923-2.-8082-0' in vhost '/': Synchronising: all slaves already synced 2025-03-20 02:03:38.565 [info] <0.1495.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:03:38.566 [info] <0.1495.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:03:38.567 [info] <0.3027.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Synchronising: all slaves already synced 2025-03-20 02:03:38.568 [info] <0.1298.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:03:38.568 [info] <0.1298.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:03:38.569 [info] <0.1553.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-2.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:03:38.569 [info] <0.1553.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-2.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:03:38.570 [info] <0.2702.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Master saw deaths of mirrors 2025-03-20 02:03:38.570 [info] <0.1601.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-1.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:03:38.570 [info] <0.1413.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-3..tftp_queue' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:03:38.570 [info] <0.1601.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-1.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:03:38.570 [info] <0.1413.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-3..tftp_queue' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:03:38.571 [info] <0.2479.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2923-2..dummy' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:03:38.571 [info] <0.2622.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-2.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Master saw deaths of mirrors 2025-03-20 02:03:38.571 [info] <0.2479.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2923-2..dummy' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:03:38.571 [info] <0.1192.0> Mirrored queue 'svc_monitor.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:03:38.571 [info] <0.2635.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-3..tftp_queue' in vhost '/': Master saw deaths of mirrors 2025-03-20 02:03:38.572 [info] <0.1192.0> Mirrored queue 'svc_monitor.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:03:38.572 [info] <0.2655.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-1.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Master saw deaths of mirrors 2025-03-20 02:03:38.572 [info] <0.516.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-2..config_queue' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:03:38.572 [info] <0.516.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-2..config_queue' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:03:38.572 [info] <0.3047.0> Mirrored queue 'svc_monitor.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Synchronising: all slaves already synced 2025-03-20 02:03:38.572 [info] <0.2525.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2923-2..dummy' in vhost '/': Master saw deaths of mirrors 2025-03-20 02:03:38.573 [info] <0.1518.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-1.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:03:38.573 [info] <0.1518.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-1.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:03:38.575 [info] <0.1228.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2923-3.-8082-0' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:03:38.575 [info] <0.1912.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-2..config_queue' in vhost '/': Master saw deaths of mirrors 2025-03-20 02:03:38.576 [info] <0.606.0> Mirrored queue 'device_manager_ztp.ztp_queue' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:03:38.576 [info] <0.1228.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2923-3.-8082-0' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:03:38.576 [info] <0.606.0> Mirrored queue 'device_manager_ztp.ztp_queue' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:03:38.576 [info] <0.2630.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-1.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Master saw deaths of mirrors 2025-03-20 02:03:38.576 [info] <0.3032.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Synchronising: all slaves already synced 2025-03-20 02:03:38.577 [info] <0.560.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:03:38.577 [info] <0.560.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Synchronising: batch size: 5 2025-03-20 02:03:38.577 [info] <0.1926.0> Mirrored queue 'device_manager_ztp.ztp_queue' in vhost '/': Master saw deaths of mirrors 2025-03-20 02:03:38.577 [info] <0.3059.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2923-3.-8082-0' in vhost '/': Synchronising: all slaves already synced 2025-03-20 02:03:38.577 [info] <0.1739.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Master saw deaths of mirrors 2025-03-20 02:03:38.578 [info] <0.3037.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-2.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Synchronising: all slaves already synced 2025-03-20 02:03:38.578 [info] <0.3041.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-1.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Synchronising: all slaves already synced 2025-03-20 02:03:38.578 [info] <0.1565.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-3.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:03:38.578 [info] <0.3042.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-3..tftp_queue' in vhost '/': Synchronising: all slaves already synced 2025-03-20 02:03:38.578 [info] <0.1614.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2923-1' in vhost '/': Synchronising: 0 messages to synchronise 2025-03-20 02:03:39.072 [warning] lager_file_backend dropped 98 messages in the last second that exceeded the limit of 50 messages/sec 2025-03-20 02:03:49.220 [info] <0.418.0> node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-2' down: connection_closed 2025-03-20 02:03:50.183 [info] <0.418.0> node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-2' up 2025-03-20 02:03:53.920 [info] <0.418.0> node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-2' down: connection_closed 2025-03-20 02:03:56.149 [info] <0.3294.0> RabbitMQ is asked to stop... 2025-03-20 02:03:56.248 [info] <0.3294.0> Stopping RabbitMQ applications and their dependencies in the following order: rabbitmq_management amqp_client rabbitmq_web_dispatch cowboy cowlib rabbitmq_management_agent rabbit mnesia rabbit_common sysmon_handler os_mon 2025-03-20 02:03:56.248 [info] <0.3294.0> Stopping application 'rabbitmq_management' 2025-03-20 02:03:56.253 [warning] <0.804.0> RabbitMQ HTTP listener registry could not find context rabbitmq_management_tls 2025-03-20 02:03:56.254 [info] <0.3294.0> Stopping application 'amqp_client' 2025-03-20 02:03:56.254 [info] <0.43.0> Application rabbitmq_management exited with reason: stopped 2025-03-20 02:03:56.256 [info] <0.43.0> Application amqp_client exited with reason: stopped 2025-03-20 02:03:56.256 [info] <0.3294.0> Stopping application 'rabbitmq_web_dispatch' 2025-03-20 02:03:56.258 [info] <0.3294.0> Stopping application 'cowboy' 2025-03-20 02:03:56.258 [info] <0.43.0> Application rabbitmq_web_dispatch exited with reason: stopped 2025-03-20 02:03:56.259 [info] <0.43.0> Application cowboy exited with reason: stopped 2025-03-20 02:03:56.259 [info] <0.3294.0> Stopping application 'cowlib' 2025-03-20 02:03:56.259 [info] <0.43.0> Application cowlib exited with reason: stopped 2025-03-20 02:03:56.259 [info] <0.3294.0> Stopping application 'rabbitmq_management_agent' 2025-03-20 02:03:56.267 [info] <0.43.0> Application rabbitmq_management_agent exited with reason: stopped 2025-03-20 02:03:56.267 [info] <0.3294.0> Stopping application 'rabbit' 2025-03-20 02:03:56.267 [info] <0.311.0> Will unregister with peer discovery backend rabbit_peer_discovery_classic_config 2025-03-20 02:03:56.267 [info] <0.755.0> stopped TCP listener on [::]:5673 2025-03-20 02:03:56.268 [error] <0.2436.0> Error on AMQP connection <0.2436.0> (10.0.0.47:40436 -> 10.0.0.47:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-03-20 02:03:56.268 [error] <0.2243.0> Error on AMQP connection <0.2243.0> (10.0.0.12:57868 -> 10.0.0.47:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-03-20 02:03:56.268 [error] <0.2202.0> Error on AMQP connection <0.2202.0> (10.0.0.47:40384 -> 10.0.0.47:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-03-20 02:03:56.268 [error] <0.2195.0> Error on AMQP connection <0.2195.0> (10.0.0.42:49268 -> 10.0.0.47:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-03-20 02:03:56.268 [error] <0.2240.0> Error on AMQP connection <0.2240.0> (10.0.0.47:40408 -> 10.0.0.47:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-03-20 02:03:56.268 [error] <0.2198.0> Error on AMQP connection <0.2198.0> (10.0.0.12:57860 -> 10.0.0.47:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-03-20 02:03:56.268 [error] <0.2177.0> Error on AMQP connection <0.2177.0> (10.0.0.47:40358 -> 10.0.0.47:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-03-20 02:03:56.268 [error] <0.2419.0> Error on AMQP connection <0.2419.0> (10.0.0.42:49296 -> 10.0.0.47:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-03-20 02:03:56.268 [error] <0.2425.0> Error on AMQP connection <0.2425.0> (10.0.0.47:40428 -> 10.0.0.47:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-03-20 02:03:56.268 [error] <0.2422.0> Error on AMQP connection <0.2422.0> (10.0.0.12:57878 -> 10.0.0.47:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-03-20 02:03:56.268 [error] <0.2220.0> Error on AMQP connection <0.2220.0> (10.0.0.42:49246 -> 10.0.0.47:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-03-20 02:03:56.269 [error] <0.2223.0> Error on AMQP connection <0.2223.0> (10.0.0.42:49260 -> 10.0.0.47:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-03-20 02:03:56.269 [error] <0.2171.0> Error on AMQP connection <0.2171.0> (10.0.0.47:40332 -> 10.0.0.47:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-03-20 02:03:56.269 [error] <0.2189.0> Error on AMQP connection <0.2189.0> (10.0.0.12:57828 -> 10.0.0.47:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-03-20 02:03:56.269 [error] <0.2209.0> Error on AMQP connection <0.2209.0> (10.0.0.12:57844 -> 10.0.0.47:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-03-20 02:03:56.268 [error] <0.2168.0> Error on AMQP connection <0.2168.0> (10.0.0.12:57814 -> 10.0.0.47:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-03-20 02:03:56.269 [error] <0.2258.0> Error on AMQP connection <0.2258.0> (10.0.0.42:49286 -> 10.0.0.47:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-03-20 02:03:56.268 [error] <0.2180.0> Error on AMQP connection <0.2180.0> (10.0.0.47:40374 -> 10.0.0.47:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-03-20 02:03:56.268 [error] <0.2174.0> Error on AMQP connection <0.2174.0> (10.0.0.47:40342 -> 10.0.0.47:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-03-20 02:03:56.268 [error] <0.2226.0> Error on AMQP connection <0.2226.0> (10.0.0.12:57856 -> 10.0.0.47:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-03-20 02:03:56.268 [error] <0.2205.0> Error on AMQP connection <0.2205.0> (10.0.0.12:57866 -> 10.0.0.47:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-03-20 02:03:56.268 [error] <0.2253.0> Error on AMQP connection <0.2253.0> (10.0.0.47:40416 -> 10.0.0.47:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-03-20 02:03:56.268 [error] <0.2215.0> Error on AMQP connection <0.2215.0> (10.0.0.47:40398 -> 10.0.0.47:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-03-20 02:03:56.269 [error] <0.2236.0> Error on AMQP connection <0.2236.0> (10.0.0.42:49278 -> 10.0.0.47:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-03-20 02:03:56.269 [error] <0.2232.0> Error on AMQP connection <0.2232.0> (10.0.0.42:49276 -> 10.0.0.47:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-03-20 02:03:56.269 [error] <0.2192.0> Error on AMQP connection <0.2192.0> (10.0.0.42:49264 -> 10.0.0.47:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-03-20 02:03:56.269 [error] <0.2229.0> Error on AMQP connection <0.2229.0> (10.0.0.47:40400 -> 10.0.0.47:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-03-20 02:03:56.269 [error] <0.2246.0> Error on AMQP connection <0.2246.0> (10.0.0.47:40412 -> 10.0.0.47:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-03-20 02:03:56.349 [error] <0.2449.0> Supervisor {<0.2449.0>,rabbit_channel_sup_sup} had child channel_sup started with rabbit_channel_sup:start_link() at undefined exit with reason shutdown in context shutdown_error 2025-03-20 02:03:56.352 [info] <0.443.0> Closing all connections in vhost '/' on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3' because the vhost is stopping 2025-03-20 02:03:56.414 [info] <0.462.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2923-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' 2025-03-20 02:03:56.422 [info] <0.462.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2923-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' is stopped 2025-03-20 02:04:30.157 [info] <0.8.0> Log file opened with Lager 2025-03-20 02:04:33.755 [info] <0.8.0> Feature flags: list of feature flags found: 2025-03-20 02:04:33.756 [info] <0.8.0> Feature flags: feature flag states written to disk: yes 2025-03-20 02:04:33.786 [info] <0.251.0> Starting RabbitMQ 3.7.28 on Erlang 21.3.8.21 Copyright (c) 2007-2020 Pivotal Software, Inc. Licensed under the MPL. See https://www.rabbitmq.com/ 2025-03-20 02:04:33.786 [info] <0.251.0> node : contrail@cn-jenkins-deploy-platform-ansible-os-2923-3 home dir : /var/lib/rabbitmq config file(s) : /etc/rabbitmq/rabbitmq.config cookie hash : esZsDxSN6VGbi9JkMSxNZA== log(s) : /var/log/contrail/config-database-rabbitmq/rabbitmq.log : /var/log/rabbitmq/contrail@cn-jenkins-deploy-platform-ansible-os-2923-3_upgrade.log database dir : /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2923-3 2025-03-20 02:04:33.803 [info] <0.251.0> Running boot step pre_boot defined by app rabbit 2025-03-20 02:04:33.803 [info] <0.251.0> Running boot step rabbit_core_metrics defined by app rabbit 2025-03-20 02:04:33.804 [info] <0.251.0> Running boot step rabbit_alarm defined by app rabbit 2025-03-20 02:04:33.819 [info] <0.257.0> Memory high watermark set to 12637 MiB (13251077734 bytes) of 15796 MiB (16563847168 bytes) total 2025-03-20 02:04:33.826 [info] <0.259.0> Enabling free disk space monitoring 2025-03-20 02:04:33.826 [info] <0.259.0> Disk free limit set to 50MB 2025-03-20 02:04:33.830 [info] <0.251.0> Running boot step code_server_cache defined by app rabbit 2025-03-20 02:04:33.830 [info] <0.251.0> Running boot step file_handle_cache defined by app rabbit 2025-03-20 02:04:33.830 [info] <0.262.0> Limiting to approx 65436 file handles (58890 sockets) 2025-03-20 02:04:33.831 [info] <0.263.0> FHC read buffering: OFF 2025-03-20 02:04:33.831 [info] <0.263.0> FHC write buffering: ON 2025-03-20 02:04:33.831 [info] <0.251.0> Running boot step worker_pool defined by app rabbit 2025-03-20 02:04:33.831 [info] <0.252.0> Will use 4 processes for default worker pool 2025-03-20 02:04:33.831 [info] <0.252.0> Starting worker pool 'worker_pool' with 4 processes in it 2025-03-20 02:04:33.832 [info] <0.251.0> Running boot step database defined by app rabbit 2025-03-20 02:04:33.832 [info] <0.251.0> Node database directory at /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2923-3 is empty. Assuming we need to join an existing cluster or initialise from scratch... 2025-03-20 02:04:33.832 [info] <0.251.0> Configured peer discovery backend: rabbit_peer_discovery_classic_config 2025-03-20 02:04:33.832 [info] <0.251.0> Will try to lock with peer discovery backend rabbit_peer_discovery_classic_config 2025-03-20 02:04:33.832 [info] <0.251.0> Peer discovery backend does not support locking, falling back to randomized delay 2025-03-20 02:04:33.832 [info] <0.251.0> Peer discovery backend rabbit_peer_discovery_classic_config supports registration. 2025-03-20 02:04:33.832 [info] <0.251.0> Will wait for 12813 milliseconds before proceeding with registration... 2025-03-20 02:04:46.645 [info] <0.251.0> All discovered existing cluster peers: contrail@cn-jenkins-deploy-platform-ansible-os-2923-1, contrail@cn-jenkins-deploy-platform-ansible-os-2923-2, contrail@cn-jenkins-deploy-platform-ansible-os-2923-3 2025-03-20 02:04:46.646 [info] <0.251.0> Peer nodes we can cluster with: contrail@cn-jenkins-deploy-platform-ansible-os-2923-1, contrail@cn-jenkins-deploy-platform-ansible-os-2923-2 2025-03-20 02:04:46.648 [info] <0.251.0> Node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-1' selected for auto-clustering 2025-03-20 02:04:46.690 [info] <0.251.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-03-20 02:04:46.782 [info] <0.251.0> Successfully synced tables from a peer 2025-03-20 02:04:46.783 [info] <0.251.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-03-20 02:04:46.783 [info] <0.251.0> Successfully synced tables from a peer 2025-03-20 02:04:46.826 [info] <0.251.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-03-20 02:04:46.832 [info] <0.251.0> Successfully synced tables from a peer 2025-03-20 02:04:46.834 [info] <0.251.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2923-3' 2025-03-20 02:04:46.835 [info] <0.251.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2923-3' 2025-03-20 02:04:46.836 [info] <0.251.0> Will register with peer discovery backend rabbit_peer_discovery_classic_config 2025-03-20 02:04:46.837 [info] <0.251.0> Running boot step database_sync defined by app rabbit 2025-03-20 02:04:46.837 [info] <0.251.0> Running boot step feature_flags defined by app rabbit 2025-03-20 02:04:46.839 [info] <0.251.0> Running boot step codec_correctness_check defined by app rabbit 2025-03-20 02:04:46.843 [info] <0.251.0> Running boot step external_infrastructure defined by app rabbit 2025-03-20 02:04:46.844 [info] <0.251.0> Running boot step rabbit_registry defined by app rabbit 2025-03-20 02:04:46.844 [info] <0.251.0> Running boot step rabbit_auth_mechanism_cr_demo defined by app rabbit 2025-03-20 02:04:46.844 [info] <0.251.0> Running boot step rabbit_queue_location_random defined by app rabbit 2025-03-20 02:04:46.845 [info] <0.251.0> Running boot step rabbit_event defined by app rabbit 2025-03-20 02:04:46.845 [info] <0.251.0> Running boot step rabbit_auth_mechanism_amqplain defined by app rabbit 2025-03-20 02:04:46.845 [info] <0.251.0> Running boot step rabbit_auth_mechanism_plain defined by app rabbit 2025-03-20 02:04:46.846 [info] <0.251.0> Running boot step rabbit_exchange_type_direct defined by app rabbit 2025-03-20 02:04:46.846 [info] <0.251.0> Running boot step rabbit_exchange_type_fanout defined by app rabbit 2025-03-20 02:04:46.846 [info] <0.251.0> Running boot step rabbit_exchange_type_headers defined by app rabbit 2025-03-20 02:04:46.847 [info] <0.251.0> Running boot step rabbit_exchange_type_topic defined by app rabbit 2025-03-20 02:04:46.847 [info] <0.251.0> Running boot step rabbit_mirror_queue_mode_all defined by app rabbit 2025-03-20 02:04:46.847 [info] <0.251.0> Running boot step rabbit_mirror_queue_mode_exactly defined by app rabbit 2025-03-20 02:04:46.848 [info] <0.251.0> Running boot step rabbit_mirror_queue_mode_nodes defined by app rabbit 2025-03-20 02:04:46.848 [info] <0.251.0> Running boot step rabbit_priority_queue defined by app rabbit 2025-03-20 02:04:46.848 [info] <0.251.0> Priority queues enabled, real BQ is rabbit_variable_queue 2025-03-20 02:04:46.848 [info] <0.251.0> Running boot step rabbit_queue_location_client_local defined by app rabbit 2025-03-20 02:04:46.848 [info] <0.251.0> Running boot step rabbit_queue_location_min_masters defined by app rabbit 2025-03-20 02:04:46.848 [info] <0.251.0> Running boot step kernel_ready defined by app rabbit 2025-03-20 02:04:46.848 [info] <0.251.0> Running boot step rabbit_sysmon_minder defined by app rabbit 2025-03-20 02:04:46.849 [info] <0.251.0> Running boot step rabbit_epmd_monitor defined by app rabbit 2025-03-20 02:04:46.850 [info] <0.397.0> epmd monitor knows us, inter-node communication (distribution) port: 25673 2025-03-20 02:04:46.850 [info] <0.251.0> Running boot step guid_generator defined by app rabbit 2025-03-20 02:04:46.854 [info] <0.251.0> Running boot step rabbit_node_monitor defined by app rabbit 2025-03-20 02:04:46.855 [info] <0.401.0> Starting rabbit_node_monitor 2025-03-20 02:04:46.855 [info] <0.251.0> Running boot step delegate_sup defined by app rabbit 2025-03-20 02:04:46.855 [info] <0.251.0> Running boot step rabbit_memory_monitor defined by app rabbit 2025-03-20 02:04:46.856 [info] <0.251.0> Running boot step core_initialized defined by app rabbit 2025-03-20 02:04:46.856 [info] <0.251.0> Running boot step upgrade_queues defined by app rabbit 2025-03-20 02:04:46.884 [info] <0.251.0> message_store upgrades: 1 to apply 2025-03-20 02:04:46.884 [info] <0.251.0> message_store upgrades: Applying rabbit_variable_queue:move_messages_to_vhost_store 2025-03-20 02:04:46.884 [info] <0.251.0> message_store upgrades: No durable queues found. Skipping message store migration 2025-03-20 02:04:46.884 [info] <0.251.0> message_store upgrades: Removing the old message store data 2025-03-20 02:04:46.886 [info] <0.251.0> message_store upgrades: All upgrades applied successfully 2025-03-20 02:04:46.919 [info] <0.251.0> Running boot step rabbit_connection_tracking defined by app rabbit 2025-03-20 02:04:46.919 [info] <0.251.0> Running boot step rabbit_connection_tracking_handler defined by app rabbit 2025-03-20 02:04:46.919 [info] <0.251.0> Running boot step rabbit_exchange_parameters defined by app rabbit 2025-03-20 02:04:46.919 [info] <0.251.0> Running boot step rabbit_mirror_queue_misc defined by app rabbit 2025-03-20 02:04:46.920 [info] <0.251.0> Running boot step rabbit_policies defined by app rabbit 2025-03-20 02:04:46.920 [info] <0.251.0> Running boot step rabbit_policy defined by app rabbit 2025-03-20 02:04:46.920 [info] <0.251.0> Running boot step rabbit_queue_location_validator defined by app rabbit 2025-03-20 02:04:46.920 [info] <0.251.0> Running boot step rabbit_vhost_limit defined by app rabbit 2025-03-20 02:04:46.920 [info] <0.251.0> Running boot step rabbit_mgmt_reset_handler defined by app rabbitmq_management 2025-03-20 02:04:46.920 [info] <0.251.0> Running boot step rabbit_mgmt_db_handler defined by app rabbitmq_management_agent 2025-03-20 02:04:46.920 [warning] <0.251.0> force_fine_statistics set to true; ignored. Replaced by {rates_mode, none} in the rabbitmq_management application. 2025-03-20 02:04:46.920 [info] <0.251.0> Management plugin: using rates mode 'basic' 2025-03-20 02:04:46.940 [info] <0.251.0> Running boot step recovery defined by app rabbit 2025-03-20 02:04:46.944 [info] <0.438.0> Making sure data directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2923-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists 2025-03-20 02:04:46.951 [info] <0.438.0> Starting message stores for vhost '/' 2025-03-20 02:04:46.951 [info] <0.442.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index 2025-03-20 02:04:46.952 [info] <0.438.0> Started message store of type transient for vhost '/' 2025-03-20 02:04:46.952 [info] <0.445.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index 2025-03-20 02:04:46.953 [warning] <0.445.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": rebuilding indices from scratch 2025-03-20 02:04:46.954 [info] <0.438.0> Started message store of type persistent for vhost '/' 2025-03-20 02:04:46.955 [info] <0.438.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-3..tftp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.464.0> 2025-03-20 02:04:46.955 [info] <0.438.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.468.0> 2025-03-20 02:04:46.956 [info] <0.438.0> Mirrored queue 'job_status_consumer.dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.472.0> 2025-03-20 02:04:46.956 [info] <0.438.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2923-1' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.476.0> 2025-03-20 02:04:46.956 [info] <0.438.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-1.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.480.0> 2025-03-20 02:04:46.957 [info] <0.438.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.484.0> 2025-03-20 02:04:46.957 [info] <0.438.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2923-3' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.488.0> 2025-03-20 02:04:46.957 [info] <0.438.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.492.0> 2025-03-20 02:04:46.957 [info] <0.438.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.496.0> 2025-03-20 02:04:46.957 [info] <0.438.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.500.0> 2025-03-20 02:04:46.958 [info] <0.438.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-2..config_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.504.0> 2025-03-20 02:04:46.958 [info] <0.438.0> Mirrored queue 'job_request_consumer' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.508.0> 2025-03-20 02:04:46.958 [info] <0.438.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-3.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.512.0> 2025-03-20 02:04:46.959 [info] <0.438.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2923-3' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.516.0> 2025-03-20 02:04:46.959 [info] <0.438.0> Mirrored queue 'svc_monitor.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.520.0> 2025-03-20 02:04:46.959 [info] <0.438.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-2..tftp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.524.0> 2025-03-20 02:04:46.959 [info] <0.438.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-1.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.528.0> 2025-03-20 02:04:46.959 [info] <0.438.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-2.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.532.0> 2025-03-20 02:04:46.960 [info] <0.438.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2923-2' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.536.0> 2025-03-20 02:04:46.960 [info] <0.438.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.540.0> 2025-03-20 02:04:46.960 [info] <0.438.0> Mirrored queue 'schema_transformer.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.544.0> 2025-03-20 02:04:46.960 [info] <0.438.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-2.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.548.0> 2025-03-20 02:04:46.961 [info] <0.438.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-3.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.552.0> 2025-03-20 02:04:46.961 [info] <0.438.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2923-3.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.556.0> 2025-03-20 02:04:46.961 [info] <0.438.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-3.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.560.0> 2025-03-20 02:04:46.962 [info] <0.438.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2923-1' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.564.0> 2025-03-20 02:04:46.962 [info] <0.438.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2923-2..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.568.0> 2025-03-20 02:04:46.962 [info] <0.438.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2923-1..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.572.0> 2025-03-20 02:04:46.962 [info] <0.438.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2923-1.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.576.0> 2025-03-20 02:04:46.963 [info] <0.438.0> Mirrored queue 'device_manager_ztp.ztp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.580.0> 2025-03-20 02:04:46.963 [info] <0.438.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2923-3..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.584.0> 2025-03-20 02:04:46.963 [info] <0.438.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2923-2' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.588.0> 2025-03-20 02:04:46.963 [info] <0.438.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-3..config_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.592.0> 2025-03-20 02:04:46.964 [info] <0.438.0> Mirrored queue 'device_manager.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.596.0> 2025-03-20 02:04:46.964 [info] <0.438.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-1..tftp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.600.0> 2025-03-20 02:04:46.964 [info] <0.438.0> Mirrored queue 'device_manager.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.604.0> 2025-03-20 02:04:46.964 [info] <0.438.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-1..config_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.608.0> 2025-03-20 02:04:46.965 [info] <0.438.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-2.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.612.0> 2025-03-20 02:04:46.965 [info] <0.438.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-1.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.616.0> 2025-03-20 02:04:46.965 [info] <0.438.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2923-2.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.620.0> 2025-03-20 02:04:46.965 [info] <0.438.0> Mirrored queue 'device_manager.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.624.0> 2025-03-20 02:04:46.969 [info] <0.251.0> Running boot step load_definitions defined by app rabbitmq_management 2025-03-20 02:04:46.969 [info] <0.251.0> Applying definitions from /etc/rabbitmq/definitions.json 2025-03-20 02:04:46.969 [info] <0.251.0> Asked to import definitions. Acting user: rmq-internal 2025-03-20 02:04:46.969 [info] <0.251.0> Importing users... 2025-03-20 02:04:46.979 [info] <0.251.0> Setting user tags for user 'guest' to [administrator] 2025-03-20 02:04:46.999 [info] <0.251.0> Importing vhosts... 2025-03-20 02:04:46.999 [info] <0.251.0> Importing user permissions... 2025-03-20 02:04:46.999 [info] <0.251.0> Setting permissions for 'guest' in '/' to '.*', '.*', '.*' 2025-03-20 02:04:47.066 [info] <0.251.0> Importing topic permissions... 2025-03-20 02:04:47.066 [info] <0.251.0> Importing parameters... 2025-03-20 02:04:47.067 [info] <0.251.0> Importing global parameters... 2025-03-20 02:04:47.067 [info] <0.251.0> Importing policies... 2025-03-20 02:04:47.167 [info] <0.251.0> Importing queues... 2025-03-20 02:04:47.168 [info] <0.251.0> Importing exchanges... 2025-03-20 02:04:47.168 [info] <0.251.0> Importing bindings... 2025-03-20 02:04:47.168 [info] <0.251.0> Running boot step empty_db_check defined by app rabbit 2025-03-20 02:04:47.168 [info] <0.251.0> Running boot step rabbit_looking_glass defined by app rabbit 2025-03-20 02:04:47.168 [info] <0.251.0> Running boot step rabbit_core_metrics_gc defined by app rabbit 2025-03-20 02:04:47.168 [info] <0.251.0> Running boot step background_gc defined by app rabbit 2025-03-20 02:04:47.168 [info] <0.251.0> Running boot step connection_tracking defined by app rabbit 2025-03-20 02:04:47.170 [info] <0.251.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2923-3' 2025-03-20 02:04:47.173 [info] <0.251.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2923-3' 2025-03-20 02:04:47.173 [info] <0.251.0> Running boot step routing_ready defined by app rabbit 2025-03-20 02:04:47.173 [info] <0.251.0> Running boot step pre_flight defined by app rabbit 2025-03-20 02:04:47.173 [info] <0.251.0> Running boot step notify_cluster defined by app rabbit 2025-03-20 02:04:47.173 [info] <0.251.0> Running boot step networking defined by app rabbit 2025-03-20 02:04:47.175 [info] <0.401.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-2' up 2025-03-20 02:04:47.176 [info] <0.773.0> started TCP listener on [::]:5673 2025-03-20 02:04:47.176 [info] <0.251.0> Running boot step cluster_name defined by app rabbit 2025-03-20 02:04:47.176 [info] <0.251.0> Running boot step direct_client defined by app rabbit 2025-03-20 02:04:47.177 [info] <0.251.0> Running boot step os_signal_handler defined by app rabbit 2025-03-20 02:04:47.177 [info] <0.775.0> Swapping OS signal event handler (erl_signal_server) for our own 2025-03-20 02:04:47.302 [info] <0.401.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-1' up 2025-03-20 02:04:47.351 [info] <0.909.0> Management plugin: HTTP (non-TLS) listener started on port 15673 2025-03-20 02:04:47.352 [info] <0.1015.0> Statistics database started. 2025-03-20 02:04:47.352 [info] <0.1014.0> Starting worker pool 'management_worker_pool' with 3 processes in it 2025-03-20 02:04:47.352 [notice] <0.106.0> Changed loghwm of /var/log/contrail/config-database-rabbitmq/rabbitmq.log to 50 2025-03-20 02:04:47.594 [info] <0.8.0> Server startup complete; 3 plugins started. * rabbitmq_management * rabbitmq_web_dispatch * rabbitmq_management_agent 2025-03-20 02:04:57.446 [info] <0.1040.0> RabbitMQ is asked to stop... 2025-03-20 02:04:57.501 [info] <0.1040.0> Stopping RabbitMQ applications and their dependencies in the following order: rabbitmq_management amqp_client rabbitmq_web_dispatch cowboy cowlib rabbitmq_management_agent rabbit mnesia rabbit_common sysmon_handler os_mon 2025-03-20 02:04:57.501 [info] <0.1040.0> Stopping application 'rabbitmq_management' 2025-03-20 02:04:57.504 [warning] <0.905.0> RabbitMQ HTTP listener registry could not find context rabbitmq_management_tls 2025-03-20 02:04:57.506 [info] <0.1040.0> Stopping application 'amqp_client' 2025-03-20 02:04:57.506 [info] <0.43.0> Application rabbitmq_management exited with reason: stopped 2025-03-20 02:04:57.507 [info] <0.1040.0> Stopping application 'rabbitmq_web_dispatch' 2025-03-20 02:04:57.507 [info] <0.43.0> Application amqp_client exited with reason: stopped 2025-03-20 02:04:57.508 [info] <0.1040.0> Stopping application 'cowboy' 2025-03-20 02:04:57.509 [info] <0.43.0> Application rabbitmq_web_dispatch exited with reason: stopped 2025-03-20 02:04:57.511 [info] <0.1040.0> Stopping application 'cowlib' 2025-03-20 02:04:57.511 [info] <0.43.0> Application cowboy exited with reason: stopped 2025-03-20 02:04:57.511 [info] <0.43.0> Application cowlib exited with reason: stopped 2025-03-20 02:04:57.511 [info] <0.1040.0> Stopping application 'rabbitmq_management_agent' 2025-03-20 02:04:57.513 [info] <0.43.0> Application rabbitmq_management_agent exited with reason: stopped 2025-03-20 02:04:57.513 [info] <0.1040.0> Stopping application 'rabbit' 2025-03-20 02:04:57.514 [info] <0.251.0> Will unregister with peer discovery backend rabbit_peer_discovery_classic_config 2025-03-20 02:04:57.514 [info] <0.773.0> stopped TCP listener on [::]:5673 2025-03-20 02:04:57.521 [info] <0.426.0> Closing all connections in vhost '/' on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3' because the vhost is stopping 2025-03-20 02:04:57.523 [info] <0.445.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2923-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' 2025-03-20 02:04:57.529 [info] <0.445.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2923-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' is stopped 2025-03-20 02:04:57.529 [info] <0.442.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2923-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' 2025-03-20 02:04:57.533 [info] <0.442.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2923-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' is stopped 2025-03-20 02:04:57.538 [info] <0.1040.0> Stopping application 'mnesia' 2025-03-20 02:04:57.538 [info] <0.43.0> Application rabbit exited with reason: stopped 2025-03-20 02:04:57.541 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-03-20 02:04:57.541 [info] <0.1040.0> Stopping application 'rabbit_common' 2025-03-20 02:04:57.541 [info] <0.43.0> Application rabbit_common exited with reason: stopped 2025-03-20 02:04:57.541 [info] <0.1040.0> Stopping application 'sysmon_handler' 2025-03-20 02:04:57.542 [info] <0.43.0> Application sysmon_handler exited with reason: stopped 2025-03-20 02:04:57.542 [info] <0.1040.0> Stopping application 'os_mon' 2025-03-20 02:04:57.543 [info] <0.43.0> Application os_mon exited with reason: stopped 2025-03-20 02:04:57.543 [info] <0.1040.0> Successfully stopped RabbitMQ and its dependencies 2025-03-20 02:04:57.544 [info] <0.1040.0> Halting Erlang VM with the following applications: jsx ranch ssl public_key asn1 observer_cli tools credentials_obfuscation crypto stdout_formatter xmerl recon inets lager goldrush compiler syntax_tools sasl stdlib kernel 2025-03-20 02:05:03.867 [info] <0.8.0> Log file opened with Lager 2025-03-20 02:05:05.941 [info] <0.8.0> Feature flags: list of feature flags found: 2025-03-20 02:05:05.941 [info] <0.8.0> Feature flags: feature flag states written to disk: yes 2025-03-20 02:05:06.036 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-03-20 02:05:06.107 [info] <0.302.0> Starting RabbitMQ 3.7.28 on Erlang 21.3.8.21 Copyright (c) 2007-2020 Pivotal Software, Inc. Licensed under the MPL. See https://www.rabbitmq.com/ 2025-03-20 02:05:06.108 [info] <0.302.0> node : contrail@cn-jenkins-deploy-platform-ansible-os-2923-3 home dir : /var/lib/rabbitmq config file(s) : /etc/rabbitmq/rabbitmq.config cookie hash : esZsDxSN6VGbi9JkMSxNZA== log(s) : /var/log/contrail/config-database-rabbitmq/rabbitmq.log : /var/log/rabbitmq/contrail@cn-jenkins-deploy-platform-ansible-os-2923-3_upgrade.log database dir : /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2923-3 2025-03-20 02:05:06.123 [info] <0.302.0> Running boot step pre_boot defined by app rabbit 2025-03-20 02:05:06.123 [info] <0.302.0> Running boot step rabbit_core_metrics defined by app rabbit 2025-03-20 02:05:06.124 [info] <0.302.0> Running boot step rabbit_alarm defined by app rabbit 2025-03-20 02:05:06.129 [info] <0.316.0> Memory high watermark set to 12637 MiB (13251077734 bytes) of 15796 MiB (16563847168 bytes) total 2025-03-20 02:05:06.135 [info] <0.318.0> Enabling free disk space monitoring 2025-03-20 02:05:06.135 [info] <0.318.0> Disk free limit set to 50MB 2025-03-20 02:05:06.139 [info] <0.302.0> Running boot step code_server_cache defined by app rabbit 2025-03-20 02:05:06.139 [info] <0.302.0> Running boot step file_handle_cache defined by app rabbit 2025-03-20 02:05:06.139 [info] <0.321.0> Limiting to approx 65436 file handles (58890 sockets) 2025-03-20 02:05:06.140 [info] <0.322.0> FHC read buffering: OFF 2025-03-20 02:05:06.140 [info] <0.322.0> FHC write buffering: ON 2025-03-20 02:05:06.140 [info] <0.302.0> Running boot step worker_pool defined by app rabbit 2025-03-20 02:05:06.140 [info] <0.303.0> Will use 4 processes for default worker pool 2025-03-20 02:05:06.140 [info] <0.303.0> Starting worker pool 'worker_pool' with 4 processes in it 2025-03-20 02:05:06.140 [info] <0.302.0> Running boot step database defined by app rabbit 2025-03-20 02:05:06.199 [info] <0.302.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-03-20 02:05:06.249 [info] <0.302.0> Successfully synced tables from a peer 2025-03-20 02:05:06.250 [info] <0.302.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-03-20 02:05:06.250 [info] <0.302.0> Successfully synced tables from a peer 2025-03-20 02:05:06.279 [info] <0.302.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-03-20 02:05:06.279 [info] <0.302.0> Successfully synced tables from a peer 2025-03-20 02:05:06.279 [info] <0.302.0> Will register with peer discovery backend rabbit_peer_discovery_classic_config 2025-03-20 02:05:06.279 [info] <0.302.0> Running boot step database_sync defined by app rabbit 2025-03-20 02:05:06.279 [info] <0.302.0> Running boot step feature_flags defined by app rabbit 2025-03-20 02:05:06.279 [info] <0.302.0> Running boot step codec_correctness_check defined by app rabbit 2025-03-20 02:05:06.279 [info] <0.302.0> Running boot step external_infrastructure defined by app rabbit 2025-03-20 02:05:06.280 [info] <0.302.0> Running boot step rabbit_registry defined by app rabbit 2025-03-20 02:05:06.280 [info] <0.302.0> Running boot step rabbit_auth_mechanism_cr_demo defined by app rabbit 2025-03-20 02:05:06.280 [info] <0.302.0> Running boot step rabbit_queue_location_random defined by app rabbit 2025-03-20 02:05:06.280 [info] <0.302.0> Running boot step rabbit_event defined by app rabbit 2025-03-20 02:05:06.280 [info] <0.302.0> Running boot step rabbit_auth_mechanism_amqplain defined by app rabbit 2025-03-20 02:05:06.280 [info] <0.302.0> Running boot step rabbit_auth_mechanism_plain defined by app rabbit 2025-03-20 02:05:06.280 [info] <0.302.0> Running boot step rabbit_exchange_type_direct defined by app rabbit 2025-03-20 02:05:06.281 [info] <0.302.0> Running boot step rabbit_exchange_type_fanout defined by app rabbit 2025-03-20 02:05:06.281 [info] <0.302.0> Running boot step rabbit_exchange_type_headers defined by app rabbit 2025-03-20 02:05:06.281 [info] <0.302.0> Running boot step rabbit_exchange_type_topic defined by app rabbit 2025-03-20 02:05:06.281 [info] <0.302.0> Running boot step rabbit_mirror_queue_mode_all defined by app rabbit 2025-03-20 02:05:06.281 [info] <0.302.0> Running boot step rabbit_mirror_queue_mode_exactly defined by app rabbit 2025-03-20 02:05:06.281 [info] <0.302.0> Running boot step rabbit_mirror_queue_mode_nodes defined by app rabbit 2025-03-20 02:05:06.281 [info] <0.302.0> Running boot step rabbit_priority_queue defined by app rabbit 2025-03-20 02:05:06.281 [info] <0.302.0> Priority queues enabled, real BQ is rabbit_variable_queue 2025-03-20 02:05:06.281 [info] <0.302.0> Running boot step rabbit_queue_location_client_local defined by app rabbit 2025-03-20 02:05:06.281 [info] <0.302.0> Running boot step rabbit_queue_location_min_masters defined by app rabbit 2025-03-20 02:05:06.281 [info] <0.302.0> Running boot step kernel_ready defined by app rabbit 2025-03-20 02:05:06.281 [info] <0.302.0> Running boot step rabbit_sysmon_minder defined by app rabbit 2025-03-20 02:05:06.281 [info] <0.302.0> Running boot step rabbit_epmd_monitor defined by app rabbit 2025-03-20 02:05:06.282 [info] <0.414.0> epmd monitor knows us, inter-node communication (distribution) port: 25673 2025-03-20 02:05:06.282 [info] <0.302.0> Running boot step guid_generator defined by app rabbit 2025-03-20 02:05:06.287 [info] <0.302.0> Running boot step rabbit_node_monitor defined by app rabbit 2025-03-20 02:05:06.289 [info] <0.418.0> Starting rabbit_node_monitor 2025-03-20 02:05:06.289 [info] <0.302.0> Running boot step delegate_sup defined by app rabbit 2025-03-20 02:05:06.290 [info] <0.302.0> Running boot step rabbit_memory_monitor defined by app rabbit 2025-03-20 02:05:06.290 [info] <0.302.0> Running boot step core_initialized defined by app rabbit 2025-03-20 02:05:06.290 [info] <0.302.0> Running boot step upgrade_queues defined by app rabbit 2025-03-20 02:05:06.312 [info] <0.302.0> Running boot step rabbit_connection_tracking defined by app rabbit 2025-03-20 02:05:06.312 [info] <0.302.0> Running boot step rabbit_connection_tracking_handler defined by app rabbit 2025-03-20 02:05:06.312 [info] <0.302.0> Running boot step rabbit_exchange_parameters defined by app rabbit 2025-03-20 02:05:06.312 [info] <0.302.0> Running boot step rabbit_mirror_queue_misc defined by app rabbit 2025-03-20 02:05:06.313 [info] <0.302.0> Running boot step rabbit_policies defined by app rabbit 2025-03-20 02:05:06.313 [info] <0.302.0> Running boot step rabbit_policy defined by app rabbit 2025-03-20 02:05:06.313 [info] <0.302.0> Running boot step rabbit_queue_location_validator defined by app rabbit 2025-03-20 02:05:06.313 [info] <0.302.0> Running boot step rabbit_vhost_limit defined by app rabbit 2025-03-20 02:05:06.313 [info] <0.302.0> Running boot step rabbit_mgmt_reset_handler defined by app rabbitmq_management 2025-03-20 02:05:06.314 [info] <0.302.0> Running boot step rabbit_mgmt_db_handler defined by app rabbitmq_management_agent 2025-03-20 02:05:06.314 [warning] <0.302.0> force_fine_statistics set to true; ignored. Replaced by {rates_mode, none} in the rabbitmq_management application. 2025-03-20 02:05:06.314 [info] <0.302.0> Management plugin: using rates mode 'basic' 2025-03-20 02:05:06.316 [info] <0.302.0> Running boot step recovery defined by app rabbit 2025-03-20 02:05:06.316 [info] <0.455.0> Making sure data directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2923-3/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists 2025-03-20 02:05:06.321 [info] <0.455.0> Starting message stores for vhost '/' 2025-03-20 02:05:06.321 [info] <0.459.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index 2025-03-20 02:05:06.322 [info] <0.455.0> Started message store of type transient for vhost '/' 2025-03-20 02:05:06.323 [info] <0.462.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index 2025-03-20 02:05:06.324 [info] <0.455.0> Started message store of type persistent for vhost '/' 2025-03-20 02:05:06.325 [info] <0.455.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-3..tftp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.481.0> 2025-03-20 02:05:06.326 [info] <0.455.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.485.0> 2025-03-20 02:05:06.326 [info] <0.455.0> Mirrored queue 'job_status_consumer.dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.489.0> 2025-03-20 02:05:06.326 [info] <0.455.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2923-1' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.493.0> 2025-03-20 02:05:06.327 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-1.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.497.0> 2025-03-20 02:05:06.327 [info] <0.455.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.501.0> 2025-03-20 02:05:06.327 [info] <0.455.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2923-3' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.505.0> 2025-03-20 02:05:06.327 [info] <0.455.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.509.0> 2025-03-20 02:05:06.328 [info] <0.455.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.513.0> 2025-03-20 02:05:06.328 [info] <0.455.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.517.0> 2025-03-20 02:05:06.328 [info] <0.455.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-2..config_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.521.0> 2025-03-20 02:05:06.328 [info] <0.455.0> Mirrored queue 'job_request_consumer' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.525.0> 2025-03-20 02:05:06.328 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-3.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.529.0> 2025-03-20 02:05:06.329 [info] <0.455.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2923-3' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.533.0> 2025-03-20 02:05:06.330 [info] <0.455.0> Mirrored queue 'svc_monitor.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.539.0> 2025-03-20 02:05:06.330 [info] <0.455.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-2..tftp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.545.0> 2025-03-20 02:05:06.331 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-1.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.549.0> 2025-03-20 02:05:06.331 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-2.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.553.0> 2025-03-20 02:05:06.331 [info] <0.455.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2923-2' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.557.0> 2025-03-20 02:05:06.333 [info] <0.455.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.563.0> 2025-03-20 02:05:06.333 [info] <0.455.0> Mirrored queue 'schema_transformer.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.570.0> 2025-03-20 02:05:06.337 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-2.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.574.0> 2025-03-20 02:05:06.338 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-3.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.581.0> 2025-03-20 02:05:06.338 [info] <0.455.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2923-3.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.585.0> 2025-03-20 02:05:06.339 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-3.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.589.0> 2025-03-20 02:05:06.339 [info] <0.455.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2923-1' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.593.0> 2025-03-20 02:05:06.339 [info] <0.455.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2923-2..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.597.0> 2025-03-20 02:05:06.340 [info] <0.455.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2923-1..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.601.0> 2025-03-20 02:05:06.340 [info] <0.455.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2923-1.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.605.0> 2025-03-20 02:05:06.340 [info] <0.455.0> Mirrored queue 'device_manager_ztp.ztp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.609.0> 2025-03-20 02:05:06.340 [info] <0.455.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2923-3..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.613.0> 2025-03-20 02:05:06.341 [info] <0.455.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2923-2' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.617.0> 2025-03-20 02:05:06.341 [info] <0.455.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-3..config_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.621.0> 2025-03-20 02:05:06.341 [info] <0.455.0> Mirrored queue 'device_manager.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.625.0> 2025-03-20 02:05:06.341 [info] <0.455.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-1..tftp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.629.0> 2025-03-20 02:05:06.342 [info] <0.455.0> Mirrored queue 'device_manager.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.633.0> 2025-03-20 02:05:06.342 [info] <0.455.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2923-1..config_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.637.0> 2025-03-20 02:05:06.342 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-2.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2923-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.641.0> 2025-03-20 02:05:06.342 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2923-1.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2923-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.645.0> 2025-03-20 02:05:06.343 [info] <0.455.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2923-2.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.649.0> 2025-03-20 02:05:06.343 [info] <0.455.0> Mirrored queue 'device_manager.cn-jenkins-deploy-platform-ansible-os-2923-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-3': <0.653.0> 2025-03-20 02:05:06.343 [info] <0.302.0> Running boot step load_definitions defined by app rabbitmq_management 2025-03-20 02:05:06.343 [info] <0.302.0> Applying definitions from /etc/rabbitmq/definitions.json 2025-03-20 02:05:06.343 [info] <0.302.0> Asked to import definitions. Acting user: rmq-internal 2025-03-20 02:05:06.343 [info] <0.302.0> Importing users... 2025-03-20 02:05:06.472 [info] <0.302.0> Setting user tags for user 'guest' to [administrator] 2025-03-20 02:05:06.513 [info] <0.302.0> Importing vhosts... 2025-03-20 02:05:06.513 [info] <0.302.0> Importing user permissions... 2025-03-20 02:05:06.513 [info] <0.302.0> Setting permissions for 'guest' in '/' to '.*', '.*', '.*' 2025-03-20 02:05:06.548 [info] <0.302.0> Importing topic permissions... 2025-03-20 02:05:06.548 [info] <0.302.0> Importing parameters... 2025-03-20 02:05:06.548 [info] <0.302.0> Importing global parameters... 2025-03-20 02:05:06.548 [info] <0.302.0> Importing policies... 2025-03-20 02:05:06.557 [info] <0.302.0> Importing queues... 2025-03-20 02:05:06.557 [info] <0.302.0> Importing exchanges... 2025-03-20 02:05:06.557 [info] <0.302.0> Importing bindings... 2025-03-20 02:05:06.557 [info] <0.302.0> Running boot step empty_db_check defined by app rabbit 2025-03-20 02:05:06.557 [info] <0.302.0> Running boot step rabbit_looking_glass defined by app rabbit 2025-03-20 02:05:06.557 [info] <0.302.0> Running boot step rabbit_core_metrics_gc defined by app rabbit 2025-03-20 02:05:06.557 [info] <0.302.0> Running boot step background_gc defined by app rabbit 2025-03-20 02:05:06.557 [info] <0.302.0> Running boot step connection_tracking defined by app rabbit 2025-03-20 02:05:06.558 [info] <0.302.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2923-3' 2025-03-20 02:05:06.559 [info] <0.302.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2923-3' 2025-03-20 02:05:06.559 [info] <0.302.0> Running boot step routing_ready defined by app rabbit 2025-03-20 02:05:06.559 [info] <0.302.0> Running boot step pre_flight defined by app rabbit 2025-03-20 02:05:06.559 [info] <0.302.0> Running boot step notify_cluster defined by app rabbit 2025-03-20 02:05:06.559 [info] <0.302.0> Running boot step networking defined by app rabbit 2025-03-20 02:05:06.560 [info] <0.418.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-1' up 2025-03-20 02:05:06.561 [info] <0.790.0> started TCP listener on [::]:5673 2025-03-20 02:05:06.562 [info] <0.302.0> Running boot step cluster_name defined by app rabbit 2025-03-20 02:05:06.562 [info] <0.302.0> Running boot step direct_client defined by app rabbit 2025-03-20 02:05:06.562 [info] <0.302.0> Running boot step os_signal_handler defined by app rabbit 2025-03-20 02:05:06.562 [info] <0.792.0> Swapping OS signal event handler (erl_signal_server) for our own 2025-03-20 02:05:06.681 [info] <0.418.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2923-2' up 2025-03-20 02:05:06.821 [info] <0.926.0> Management plugin: HTTP (non-TLS) listener started on port 15673 2025-03-20 02:05:06.821 [info] <0.1032.0> Statistics database started. 2025-03-20 02:05:06.821 [info] <0.1031.0> Starting worker pool 'management_worker_pool' with 3 processes in it 2025-03-20 02:05:06.823 [notice] <0.106.0> Changed loghwm of /var/log/contrail/config-database-rabbitmq/rabbitmq.log to 50 2025-03-20 02:05:06.960 [info] <0.8.0> Server startup complete; 3 plugins started. * rabbitmq_management * rabbitmq_management_agent * rabbitmq_web_dispatch