2025-01-27 00:46:41.469 [info] <0.8.0> Log file opened with Lager 2025-01-27 00:46:43.838 [info] <0.8.0> Feature flags: list of feature flags found: 2025-01-27 00:46:43.838 [info] <0.8.0> Feature flags: feature flag states written to disk: yes 2025-01-27 00:46:43.859 [info] <0.240.0> Starting RabbitMQ 3.7.28 on Erlang 21.3.8.21 Copyright (c) 2007-2020 Pivotal Software, Inc. Licensed under the MPL. See https://www.rabbitmq.com/ 2025-01-27 00:46:43.860 [info] <0.240.0> node : contrail@cn-jenkins-deploy-platform-ansible-os-2660-2 home dir : /var/lib/rabbitmq config file(s) : /etc/rabbitmq/rabbitmq.config cookie hash : esZsDxSN6VGbi9JkMSxNZA== log(s) : /var/log/contrail/config-database-rabbitmq/rabbitmq.log : /var/log/rabbitmq/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2_upgrade.log database dir : /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2 2025-01-27 00:46:43.874 [info] <0.240.0> Running boot step pre_boot defined by app rabbit 2025-01-27 00:46:43.874 [info] <0.240.0> Running boot step rabbit_core_metrics defined by app rabbit 2025-01-27 00:46:43.874 [info] <0.240.0> Running boot step rabbit_alarm defined by app rabbit 2025-01-27 00:46:43.880 [info] <0.246.0> Memory high watermark set to 12637 MiB (13251084288 bytes) of 15796 MiB (16563855360 bytes) total 2025-01-27 00:46:43.885 [info] <0.248.0> Enabling free disk space monitoring 2025-01-27 00:46:43.885 [info] <0.248.0> Disk free limit set to 50MB 2025-01-27 00:46:43.890 [info] <0.240.0> Running boot step code_server_cache defined by app rabbit 2025-01-27 00:46:43.890 [info] <0.240.0> Running boot step file_handle_cache defined by app rabbit 2025-01-27 00:46:43.891 [info] <0.251.0> Limiting to approx 65436 file handles (58890 sockets) 2025-01-27 00:46:43.891 [info] <0.252.0> FHC read buffering: OFF 2025-01-27 00:46:43.891 [info] <0.252.0> FHC write buffering: ON 2025-01-27 00:46:43.891 [info] <0.240.0> Running boot step worker_pool defined by app rabbit 2025-01-27 00:46:43.891 [info] <0.241.0> Will use 4 processes for default worker pool 2025-01-27 00:46:43.891 [info] <0.241.0> Starting worker pool 'worker_pool' with 4 processes in it 2025-01-27 00:46:43.892 [info] <0.240.0> Running boot step database defined by app rabbit 2025-01-27 00:46:43.892 [info] <0.240.0> Node database directory at /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2 is empty. Assuming we need to join an existing cluster or initialise from scratch... 2025-01-27 00:46:43.892 [info] <0.240.0> Configured peer discovery backend: rabbit_peer_discovery_classic_config 2025-01-27 00:46:43.892 [info] <0.240.0> Will try to lock with peer discovery backend rabbit_peer_discovery_classic_config 2025-01-27 00:46:43.892 [info] <0.240.0> Peer discovery backend does not support locking, falling back to randomized delay 2025-01-27 00:46:43.892 [info] <0.240.0> Peer discovery backend rabbit_peer_discovery_classic_config supports registration. 2025-01-27 00:46:43.892 [info] <0.240.0> Will wait for 11519 milliseconds before proceeding with registration... 2025-01-27 00:46:55.411 [info] <0.240.0> All discovered existing cluster peers: contrail@cn-jenkins-deploy-platform-ansible-os-2660-1, contrail@cn-jenkins-deploy-platform-ansible-os-2660-2, contrail@cn-jenkins-deploy-platform-ansible-os-2660-3 2025-01-27 00:46:55.412 [info] <0.240.0> Peer nodes we can cluster with: contrail@cn-jenkins-deploy-platform-ansible-os-2660-1, contrail@cn-jenkins-deploy-platform-ansible-os-2660-3 2025-01-27 00:46:55.421 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-2660-1: {error,tables_not_present} 2025-01-27 00:46:55.425 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-2660-3: {error,tables_not_present} 2025-01-27 00:46:55.425 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 9 retries left... 2025-01-27 00:46:55.927 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-2660-1: {error,tables_not_present} 2025-01-27 00:46:55.928 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-2660-3: {error,tables_not_present} 2025-01-27 00:46:55.928 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 8 retries left... 2025-01-27 00:46:56.431 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-2660-1: {error,tables_not_present} 2025-01-27 00:46:56.432 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-2660-3: {error,tables_not_present} 2025-01-27 00:46:56.432 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 7 retries left... 2025-01-27 00:46:56.933 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-2660-1: {error,tables_not_present} 2025-01-27 00:46:56.934 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-2660-3: {error,tables_not_present} 2025-01-27 00:46:56.935 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 6 retries left... 2025-01-27 00:46:57.436 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-2660-1: {error,tables_not_present} 2025-01-27 00:46:57.437 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-2660-3: {error,tables_not_present} 2025-01-27 00:46:57.437 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 5 retries left... 2025-01-27 00:46:57.938 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-2660-1: {error,tables_not_present} 2025-01-27 00:46:57.939 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-2660-3: {error,tables_not_present} 2025-01-27 00:46:57.939 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 4 retries left... 2025-01-27 00:46:58.440 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-2660-1: {error,tables_not_present} 2025-01-27 00:46:58.441 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-2660-3: {error,tables_not_present} 2025-01-27 00:46:58.441 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 3 retries left... 2025-01-27 00:46:58.942 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-2660-1: {error,tables_not_present} 2025-01-27 00:46:58.943 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-2660-3: {error,tables_not_present} 2025-01-27 00:46:58.943 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 2 retries left... 2025-01-27 00:46:59.444 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-2660-1: {error,tables_not_present} 2025-01-27 00:46:59.445 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-2660-3: {error,tables_not_present} 2025-01-27 00:46:59.445 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 1 retries left... 2025-01-27 00:46:59.946 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-2660-1: {error,tables_not_present} 2025-01-27 00:46:59.947 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-2660-3: {error,tables_not_present} 2025-01-27 00:46:59.947 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 0 retries left... 2025-01-27 00:47:00.447 [warning] <0.240.0> Could not successfully contact any node of: contrail@cn-jenkins-deploy-platform-ansible-os-2660-1,contrail@cn-jenkins-deploy-platform-ansible-os-2660-3 (as in Erlang distribution). Starting as a blank standalone node... 2025-01-27 00:47:00.449 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-01-27 00:47:00.590 [info] <0.240.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-01-27 00:47:00.590 [info] <0.240.0> Successfully synced tables from a peer 2025-01-27 00:47:00.616 [info] <0.240.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-01-27 00:47:00.616 [info] <0.240.0> Successfully synced tables from a peer 2025-01-27 00:47:00.644 [info] <0.240.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-01-27 00:47:00.644 [info] <0.240.0> Successfully synced tables from a peer 2025-01-27 00:47:00.644 [info] <0.240.0> Will register with peer discovery backend rabbit_peer_discovery_classic_config 2025-01-27 00:47:00.644 [info] <0.240.0> Running boot step database_sync defined by app rabbit 2025-01-27 00:47:00.644 [info] <0.240.0> Running boot step feature_flags defined by app rabbit 2025-01-27 00:47:00.644 [info] <0.240.0> Running boot step codec_correctness_check defined by app rabbit 2025-01-27 00:47:00.644 [info] <0.240.0> Running boot step external_infrastructure defined by app rabbit 2025-01-27 00:47:00.644 [info] <0.240.0> Running boot step rabbit_registry defined by app rabbit 2025-01-27 00:47:00.644 [info] <0.240.0> Running boot step rabbit_auth_mechanism_cr_demo defined by app rabbit 2025-01-27 00:47:00.644 [info] <0.240.0> Running boot step rabbit_queue_location_random defined by app rabbit 2025-01-27 00:47:00.644 [info] <0.240.0> Running boot step rabbit_event defined by app rabbit 2025-01-27 00:47:00.645 [info] <0.240.0> Running boot step rabbit_auth_mechanism_amqplain defined by app rabbit 2025-01-27 00:47:00.645 [info] <0.240.0> Running boot step rabbit_auth_mechanism_plain defined by app rabbit 2025-01-27 00:47:00.645 [info] <0.240.0> Running boot step rabbit_exchange_type_direct defined by app rabbit 2025-01-27 00:47:00.645 [info] <0.240.0> Running boot step rabbit_exchange_type_fanout defined by app rabbit 2025-01-27 00:47:00.645 [info] <0.240.0> Running boot step rabbit_exchange_type_headers defined by app rabbit 2025-01-27 00:47:00.645 [info] <0.240.0> Running boot step rabbit_exchange_type_topic defined by app rabbit 2025-01-27 00:47:00.645 [info] <0.240.0> Running boot step rabbit_mirror_queue_mode_all defined by app rabbit 2025-01-27 00:47:00.645 [info] <0.240.0> Running boot step rabbit_mirror_queue_mode_exactly defined by app rabbit 2025-01-27 00:47:00.645 [info] <0.240.0> Running boot step rabbit_mirror_queue_mode_nodes defined by app rabbit 2025-01-27 00:47:00.645 [info] <0.240.0> Running boot step rabbit_priority_queue defined by app rabbit 2025-01-27 00:47:00.645 [info] <0.240.0> Priority queues enabled, real BQ is rabbit_variable_queue 2025-01-27 00:47:00.645 [info] <0.240.0> Running boot step rabbit_queue_location_client_local defined by app rabbit 2025-01-27 00:47:00.645 [info] <0.240.0> Running boot step rabbit_queue_location_min_masters defined by app rabbit 2025-01-27 00:47:00.645 [info] <0.240.0> Running boot step kernel_ready defined by app rabbit 2025-01-27 00:47:00.645 [info] <0.240.0> Running boot step rabbit_sysmon_minder defined by app rabbit 2025-01-27 00:47:00.646 [info] <0.240.0> Running boot step rabbit_epmd_monitor defined by app rabbit 2025-01-27 00:47:00.646 [info] <0.440.0> epmd monitor knows us, inter-node communication (distribution) port: 25673 2025-01-27 00:47:00.646 [info] <0.240.0> Running boot step guid_generator defined by app rabbit 2025-01-27 00:47:00.652 [info] <0.240.0> Running boot step rabbit_node_monitor defined by app rabbit 2025-01-27 00:47:00.652 [info] <0.444.0> Starting rabbit_node_monitor 2025-01-27 00:47:00.652 [info] <0.240.0> Running boot step delegate_sup defined by app rabbit 2025-01-27 00:47:00.653 [info] <0.240.0> Running boot step rabbit_memory_monitor defined by app rabbit 2025-01-27 00:47:00.653 [info] <0.240.0> Running boot step core_initialized defined by app rabbit 2025-01-27 00:47:00.653 [info] <0.240.0> Running boot step upgrade_queues defined by app rabbit 2025-01-27 00:47:00.673 [info] <0.240.0> message_store upgrades: 1 to apply 2025-01-27 00:47:00.673 [info] <0.240.0> message_store upgrades: Applying rabbit_variable_queue:move_messages_to_vhost_store 2025-01-27 00:47:00.673 [info] <0.240.0> message_store upgrades: No durable queues found. Skipping message store migration 2025-01-27 00:47:00.673 [info] <0.240.0> message_store upgrades: Removing the old message store data 2025-01-27 00:47:00.674 [info] <0.240.0> message_store upgrades: All upgrades applied successfully 2025-01-27 00:47:00.704 [info] <0.240.0> Running boot step rabbit_connection_tracking defined by app rabbit 2025-01-27 00:47:00.704 [info] <0.240.0> Running boot step rabbit_connection_tracking_handler defined by app rabbit 2025-01-27 00:47:00.704 [info] <0.240.0> Running boot step rabbit_exchange_parameters defined by app rabbit 2025-01-27 00:47:00.704 [info] <0.240.0> Running boot step rabbit_mirror_queue_misc defined by app rabbit 2025-01-27 00:47:00.705 [info] <0.240.0> Running boot step rabbit_policies defined by app rabbit 2025-01-27 00:47:00.705 [info] <0.240.0> Running boot step rabbit_policy defined by app rabbit 2025-01-27 00:47:00.705 [info] <0.240.0> Running boot step rabbit_queue_location_validator defined by app rabbit 2025-01-27 00:47:00.705 [info] <0.240.0> Running boot step rabbit_vhost_limit defined by app rabbit 2025-01-27 00:47:00.705 [info] <0.240.0> Running boot step rabbit_mgmt_reset_handler defined by app rabbitmq_management 2025-01-27 00:47:00.706 [info] <0.240.0> Running boot step rabbit_mgmt_db_handler defined by app rabbitmq_management_agent 2025-01-27 00:47:00.706 [warning] <0.240.0> force_fine_statistics set to true; ignored. Replaced by {rates_mode, none} in the rabbitmq_management application. 2025-01-27 00:47:00.706 [info] <0.240.0> Management plugin: using rates mode 'basic' 2025-01-27 00:47:00.706 [info] <0.240.0> Running boot step recovery defined by app rabbit 2025-01-27 00:47:00.706 [info] <0.240.0> Running boot step load_definitions defined by app rabbitmq_management 2025-01-27 00:47:00.707 [info] <0.240.0> Applying definitions from /etc/rabbitmq/definitions.json 2025-01-27 00:47:00.707 [info] <0.240.0> Asked to import definitions. Acting user: rmq-internal 2025-01-27 00:47:00.707 [info] <0.240.0> Importing users... 2025-01-27 00:47:00.707 [info] <0.240.0> Creating user 'guest' 2025-01-27 00:47:00.713 [info] <0.240.0> Setting user tags for user 'guest' to [administrator] 2025-01-27 00:47:00.716 [info] <0.240.0> Importing vhosts... 2025-01-27 00:47:00.716 [info] <0.240.0> Adding vhost '/' 2025-01-27 00:47:00.735 [info] <0.492.0> Making sure data directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists 2025-01-27 00:47:00.741 [info] <0.492.0> Starting message stores for vhost '/' 2025-01-27 00:47:00.741 [info] <0.496.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index 2025-01-27 00:47:00.742 [info] <0.492.0> Started message store of type transient for vhost '/' 2025-01-27 00:47:00.742 [info] <0.499.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index 2025-01-27 00:47:00.742 [warning] <0.499.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": rebuilding indices from scratch 2025-01-27 00:47:00.743 [info] <0.492.0> Started message store of type persistent for vhost '/' 2025-01-27 00:47:00.744 [info] <0.240.0> Importing user permissions... 2025-01-27 00:47:00.744 [info] <0.240.0> Setting permissions for 'guest' in '/' to '.*', '.*', '.*' 2025-01-27 00:47:00.747 [info] <0.240.0> Importing topic permissions... 2025-01-27 00:47:00.747 [info] <0.240.0> Importing parameters... 2025-01-27 00:47:00.747 [info] <0.240.0> Importing global parameters... 2025-01-27 00:47:00.747 [info] <0.240.0> Importing policies... 2025-01-27 00:47:00.752 [info] <0.240.0> Importing queues... 2025-01-27 00:47:00.752 [info] <0.240.0> Importing exchanges... 2025-01-27 00:47:00.752 [info] <0.240.0> Importing bindings... 2025-01-27 00:47:00.753 [info] <0.240.0> Running boot step empty_db_check defined by app rabbit 2025-01-27 00:47:00.753 [info] <0.240.0> Running boot step rabbit_looking_glass defined by app rabbit 2025-01-27 00:47:00.753 [info] <0.240.0> Running boot step rabbit_core_metrics_gc defined by app rabbit 2025-01-27 00:47:00.753 [info] <0.240.0> Running boot step background_gc defined by app rabbit 2025-01-27 00:47:00.753 [info] <0.240.0> Running boot step connection_tracking defined by app rabbit 2025-01-27 00:47:00.756 [info] <0.240.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2660-2' 2025-01-27 00:47:00.760 [info] <0.240.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2660-2' 2025-01-27 00:47:00.761 [info] <0.240.0> Running boot step routing_ready defined by app rabbit 2025-01-27 00:47:00.761 [info] <0.240.0> Running boot step pre_flight defined by app rabbit 2025-01-27 00:47:00.761 [info] <0.240.0> Running boot step notify_cluster defined by app rabbit 2025-01-27 00:47:00.761 [info] <0.240.0> Running boot step networking defined by app rabbit 2025-01-27 00:47:00.762 [info] <0.545.0> started TCP listener on [::]:5673 2025-01-27 00:47:00.763 [info] <0.240.0> Running boot step cluster_name defined by app rabbit 2025-01-27 00:47:00.763 [info] <0.240.0> Running boot step direct_client defined by app rabbit 2025-01-27 00:47:00.763 [info] <0.240.0> Running boot step os_signal_handler defined by app rabbit 2025-01-27 00:47:00.763 [info] <0.547.0> Swapping OS signal event handler (erl_signal_server) for our own 2025-01-27 00:47:00.789 [info] <0.597.0> Management plugin: HTTP (non-TLS) listener started on port 15673 2025-01-27 00:47:00.789 [info] <0.703.0> Statistics database started. 2025-01-27 00:47:00.789 [info] <0.702.0> Starting worker pool 'management_worker_pool' with 3 processes in it 2025-01-27 00:47:00.790 [notice] <0.106.0> Changed loghwm of /var/log/contrail/config-database-rabbitmq/rabbitmq.log to 50 2025-01-27 00:47:00.993 [info] <0.8.0> Server startup complete; 3 plugins started. * rabbitmq_management * rabbitmq_management_agent * rabbitmq_web_dispatch 2025-01-27 00:47:06.720 [info] <0.727.0> RabbitMQ is asked to stop... 2025-01-27 00:47:06.789 [info] <0.727.0> Stopping RabbitMQ applications and their dependencies in the following order: rabbitmq_management amqp_client rabbitmq_web_dispatch cowboy cowlib rabbitmq_management_agent rabbit mnesia rabbit_common sysmon_handler os_mon 2025-01-27 00:47:06.789 [info] <0.727.0> Stopping application 'rabbitmq_management' 2025-01-27 00:47:06.791 [warning] <0.553.0> RabbitMQ HTTP listener registry could not find context rabbitmq_management_tls 2025-01-27 00:47:06.793 [info] <0.43.0> Application rabbitmq_management exited with reason: stopped 2025-01-27 00:47:06.793 [info] <0.727.0> Stopping application 'amqp_client' 2025-01-27 00:47:06.794 [info] <0.43.0> Application amqp_client exited with reason: stopped 2025-01-27 00:47:06.794 [info] <0.727.0> Stopping application 'rabbitmq_web_dispatch' 2025-01-27 00:47:06.795 [info] <0.727.0> Stopping application 'cowboy' 2025-01-27 00:47:06.795 [info] <0.43.0> Application rabbitmq_web_dispatch exited with reason: stopped 2025-01-27 00:47:06.796 [info] <0.43.0> Application cowboy exited with reason: stopped 2025-01-27 00:47:06.796 [info] <0.727.0> Stopping application 'cowlib' 2025-01-27 00:47:06.796 [info] <0.43.0> Application cowlib exited with reason: stopped 2025-01-27 00:47:06.796 [info] <0.727.0> Stopping application 'rabbitmq_management_agent' 2025-01-27 00:47:06.798 [info] <0.43.0> Application rabbitmq_management_agent exited with reason: stopped 2025-01-27 00:47:06.798 [info] <0.727.0> Stopping application 'rabbit' 2025-01-27 00:47:06.798 [info] <0.240.0> Will unregister with peer discovery backend rabbit_peer_discovery_classic_config 2025-01-27 00:47:06.798 [info] <0.545.0> stopped TCP listener on [::]:5673 2025-01-27 00:47:06.798 [info] <0.469.0> Closing all connections in vhost '/' on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2' because the vhost is stopping 2025-01-27 00:47:06.798 [info] <0.499.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' 2025-01-27 00:47:06.803 [info] <0.499.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' is stopped 2025-01-27 00:47:06.803 [info] <0.496.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' 2025-01-27 00:47:06.809 [info] <0.496.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' is stopped 2025-01-27 00:47:06.812 [info] <0.43.0> Application rabbit exited with reason: stopped 2025-01-27 00:47:06.812 [info] <0.727.0> Stopping application 'mnesia' 2025-01-27 00:47:06.815 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-01-27 00:47:06.815 [info] <0.727.0> Stopping application 'rabbit_common' 2025-01-27 00:47:06.815 [info] <0.43.0> Application rabbit_common exited with reason: stopped 2025-01-27 00:47:06.815 [info] <0.727.0> Stopping application 'sysmon_handler' 2025-01-27 00:47:06.816 [info] <0.43.0> Application sysmon_handler exited with reason: stopped 2025-01-27 00:47:06.816 [info] <0.727.0> Stopping application 'os_mon' 2025-01-27 00:47:06.817 [info] <0.43.0> Application os_mon exited with reason: stopped 2025-01-27 00:47:06.817 [info] <0.727.0> Successfully stopped RabbitMQ and its dependencies 2025-01-27 00:47:16.543 [info] <0.742.0> Resetting Rabbit forcefully 2025-01-27 00:47:18.345 [info] <0.758.0> RabbitMQ is asked to stop... 2025-01-27 00:47:18.399 [info] <0.758.0> Stopping RabbitMQ applications and their dependencies in the following order: rabbitmq_management amqp_client rabbitmq_management_agent rabbitmq_web_dispatch rabbit mnesia rabbit_common sysmon_handler os_mon 2025-01-27 00:47:18.399 [info] <0.758.0> Stopping application 'rabbitmq_management' 2025-01-27 00:47:18.399 [info] <0.758.0> Stopping application 'amqp_client' 2025-01-27 00:47:18.399 [info] <0.758.0> Stopping application 'rabbitmq_management_agent' 2025-01-27 00:47:18.399 [info] <0.758.0> Stopping application 'rabbitmq_web_dispatch' 2025-01-27 00:47:18.399 [info] <0.758.0> Stopping application 'rabbit' 2025-01-27 00:47:18.399 [info] <0.758.0> Stopping application 'mnesia' 2025-01-27 00:47:18.399 [info] <0.758.0> Stopping application 'rabbit_common' 2025-01-27 00:47:18.399 [info] <0.758.0> Stopping application 'sysmon_handler' 2025-01-27 00:47:18.399 [info] <0.758.0> Stopping application 'os_mon' 2025-01-27 00:47:18.399 [info] <0.758.0> Successfully stopped RabbitMQ and its dependencies 2025-01-27 00:47:18.399 [info] <0.758.0> Halting Erlang VM with the following applications: ranch ssl public_key asn1 observer_cli tools credentials_obfuscation crypto stdout_formatter recon inets jsx xmerl lager goldrush compiler syntax_tools sasl stdlib kernel 2025-01-27 00:47:23.722 [info] <0.8.0> Log file opened with Lager 2025-01-27 00:47:25.243 [info] <0.8.0> Feature flags: list of feature flags found: 2025-01-27 00:47:25.243 [info] <0.8.0> Feature flags: feature flag states written to disk: yes 2025-01-27 00:47:25.262 [info] <0.240.0> Starting RabbitMQ 3.7.28 on Erlang 21.3.8.21 Copyright (c) 2007-2020 Pivotal Software, Inc. Licensed under the MPL. See https://www.rabbitmq.com/ 2025-01-27 00:47:25.263 [info] <0.240.0> node : contrail@cn-jenkins-deploy-platform-ansible-os-2660-2 home dir : /var/lib/rabbitmq config file(s) : /etc/rabbitmq/rabbitmq.config cookie hash : esZsDxSN6VGbi9JkMSxNZA== log(s) : /var/log/contrail/config-database-rabbitmq/rabbitmq.log : /var/log/rabbitmq/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2_upgrade.log database dir : /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2 2025-01-27 00:47:25.275 [info] <0.240.0> Running boot step pre_boot defined by app rabbit 2025-01-27 00:47:25.275 [info] <0.240.0> Running boot step rabbit_core_metrics defined by app rabbit 2025-01-27 00:47:25.276 [info] <0.240.0> Running boot step rabbit_alarm defined by app rabbit 2025-01-27 00:47:25.280 [info] <0.246.0> Memory high watermark set to 12637 MiB (13251084288 bytes) of 15796 MiB (16563855360 bytes) total 2025-01-27 00:47:25.284 [info] <0.248.0> Enabling free disk space monitoring 2025-01-27 00:47:25.284 [info] <0.248.0> Disk free limit set to 50MB 2025-01-27 00:47:25.288 [info] <0.240.0> Running boot step code_server_cache defined by app rabbit 2025-01-27 00:47:25.288 [info] <0.240.0> Running boot step file_handle_cache defined by app rabbit 2025-01-27 00:47:25.288 [info] <0.251.0> Limiting to approx 65436 file handles (58890 sockets) 2025-01-27 00:47:25.289 [info] <0.252.0> FHC read buffering: OFF 2025-01-27 00:47:25.289 [info] <0.252.0> FHC write buffering: ON 2025-01-27 00:47:25.289 [info] <0.240.0> Running boot step worker_pool defined by app rabbit 2025-01-27 00:47:25.289 [info] <0.241.0> Will use 4 processes for default worker pool 2025-01-27 00:47:25.289 [info] <0.241.0> Starting worker pool 'worker_pool' with 4 processes in it 2025-01-27 00:47:25.289 [info] <0.240.0> Running boot step database defined by app rabbit 2025-01-27 00:47:25.289 [info] <0.240.0> Node database directory at /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2 is empty. Assuming we need to join an existing cluster or initialise from scratch... 2025-01-27 00:47:25.289 [info] <0.240.0> Configured peer discovery backend: rabbit_peer_discovery_classic_config 2025-01-27 00:47:25.289 [info] <0.240.0> Will try to lock with peer discovery backend rabbit_peer_discovery_classic_config 2025-01-27 00:47:25.289 [info] <0.240.0> Peer discovery backend does not support locking, falling back to randomized delay 2025-01-27 00:47:25.289 [info] <0.240.0> Peer discovery backend rabbit_peer_discovery_classic_config supports registration. 2025-01-27 00:47:25.289 [info] <0.240.0> Will wait for 58088 milliseconds before proceeding with registration... 2025-01-27 00:48:11.627 [info] <0.294.0> RabbitMQ hasn't finished starting yet. Waiting for startup to finish before stopping... 2025-01-27 00:48:23.378 [info] <0.240.0> All discovered existing cluster peers: contrail@cn-jenkins-deploy-platform-ansible-os-2660-1, contrail@cn-jenkins-deploy-platform-ansible-os-2660-2, contrail@cn-jenkins-deploy-platform-ansible-os-2660-3 2025-01-27 00:48:23.378 [info] <0.240.0> Peer nodes we can cluster with: contrail@cn-jenkins-deploy-platform-ansible-os-2660-1, contrail@cn-jenkins-deploy-platform-ansible-os-2660-3 2025-01-27 00:48:23.383 [info] <0.240.0> Node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-1' selected for auto-clustering 2025-01-27 00:48:23.604 [info] <0.240.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-01-27 00:48:23.604 [info] <0.240.0> Successfully synced tables from a peer 2025-01-27 00:48:23.886 [info] <0.240.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-01-27 00:48:23.886 [info] <0.240.0> Successfully synced tables from a peer 2025-01-27 00:48:23.917 [info] <0.240.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-01-27 00:48:23.918 [info] <0.240.0> Successfully synced tables from a peer 2025-01-27 00:48:23.924 [info] <0.240.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2660-2' 2025-01-27 00:48:23.929 [info] <0.240.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2660-2' 2025-01-27 00:48:23.930 [info] <0.240.0> Will register with peer discovery backend rabbit_peer_discovery_classic_config 2025-01-27 00:48:23.930 [info] <0.240.0> Running boot step database_sync defined by app rabbit 2025-01-27 00:48:23.930 [info] <0.240.0> Running boot step feature_flags defined by app rabbit 2025-01-27 00:48:23.930 [info] <0.240.0> Running boot step codec_correctness_check defined by app rabbit 2025-01-27 00:48:23.930 [info] <0.240.0> Running boot step external_infrastructure defined by app rabbit 2025-01-27 00:48:23.930 [info] <0.240.0> Running boot step rabbit_registry defined by app rabbit 2025-01-27 00:48:23.930 [info] <0.240.0> Running boot step rabbit_auth_mechanism_cr_demo defined by app rabbit 2025-01-27 00:48:23.930 [info] <0.240.0> Running boot step rabbit_queue_location_random defined by app rabbit 2025-01-27 00:48:23.930 [info] <0.240.0> Running boot step rabbit_event defined by app rabbit 2025-01-27 00:48:23.931 [info] <0.240.0> Running boot step rabbit_auth_mechanism_amqplain defined by app rabbit 2025-01-27 00:48:23.931 [info] <0.240.0> Running boot step rabbit_auth_mechanism_plain defined by app rabbit 2025-01-27 00:48:23.931 [info] <0.240.0> Running boot step rabbit_exchange_type_direct defined by app rabbit 2025-01-27 00:48:23.931 [info] <0.240.0> Running boot step rabbit_exchange_type_fanout defined by app rabbit 2025-01-27 00:48:23.931 [info] <0.240.0> Running boot step rabbit_exchange_type_headers defined by app rabbit 2025-01-27 00:48:23.931 [info] <0.240.0> Running boot step rabbit_exchange_type_topic defined by app rabbit 2025-01-27 00:48:23.931 [info] <0.240.0> Running boot step rabbit_mirror_queue_mode_all defined by app rabbit 2025-01-27 00:48:23.931 [info] <0.240.0> Running boot step rabbit_mirror_queue_mode_exactly defined by app rabbit 2025-01-27 00:48:23.931 [info] <0.240.0> Running boot step rabbit_mirror_queue_mode_nodes defined by app rabbit 2025-01-27 00:48:23.931 [info] <0.240.0> Running boot step rabbit_priority_queue defined by app rabbit 2025-01-27 00:48:23.931 [info] <0.240.0> Priority queues enabled, real BQ is rabbit_variable_queue 2025-01-27 00:48:23.932 [info] <0.240.0> Running boot step rabbit_queue_location_client_local defined by app rabbit 2025-01-27 00:48:23.932 [info] <0.240.0> Running boot step rabbit_queue_location_min_masters defined by app rabbit 2025-01-27 00:48:23.932 [info] <0.240.0> Running boot step kernel_ready defined by app rabbit 2025-01-27 00:48:23.932 [info] <0.240.0> Running boot step rabbit_sysmon_minder defined by app rabbit 2025-01-27 00:48:23.932 [info] <0.240.0> Running boot step rabbit_epmd_monitor defined by app rabbit 2025-01-27 00:48:23.933 [info] <0.467.0> epmd monitor knows us, inter-node communication (distribution) port: 25673 2025-01-27 00:48:23.933 [info] <0.240.0> Running boot step guid_generator defined by app rabbit 2025-01-27 00:48:23.937 [info] <0.240.0> Running boot step rabbit_node_monitor defined by app rabbit 2025-01-27 00:48:23.938 [info] <0.471.0> Starting rabbit_node_monitor 2025-01-27 00:48:23.938 [info] <0.240.0> Running boot step delegate_sup defined by app rabbit 2025-01-27 00:48:23.938 [info] <0.240.0> Running boot step rabbit_memory_monitor defined by app rabbit 2025-01-27 00:48:23.939 [info] <0.240.0> Running boot step core_initialized defined by app rabbit 2025-01-27 00:48:23.939 [info] <0.240.0> Running boot step upgrade_queues defined by app rabbit 2025-01-27 00:48:23.962 [info] <0.240.0> message_store upgrades: 1 to apply 2025-01-27 00:48:23.963 [info] <0.240.0> message_store upgrades: Applying rabbit_variable_queue:move_messages_to_vhost_store 2025-01-27 00:48:23.963 [info] <0.240.0> message_store upgrades: No durable queues found. Skipping message store migration 2025-01-27 00:48:23.963 [info] <0.240.0> message_store upgrades: Removing the old message store data 2025-01-27 00:48:23.963 [info] <0.240.0> message_store upgrades: All upgrades applied successfully 2025-01-27 00:48:23.991 [info] <0.240.0> Running boot step rabbit_connection_tracking defined by app rabbit 2025-01-27 00:48:23.991 [info] <0.240.0> Running boot step rabbit_connection_tracking_handler defined by app rabbit 2025-01-27 00:48:23.991 [info] <0.240.0> Running boot step rabbit_exchange_parameters defined by app rabbit 2025-01-27 00:48:23.992 [info] <0.240.0> Running boot step rabbit_mirror_queue_misc defined by app rabbit 2025-01-27 00:48:23.992 [info] <0.240.0> Running boot step rabbit_policies defined by app rabbit 2025-01-27 00:48:23.992 [info] <0.240.0> Running boot step rabbit_policy defined by app rabbit 2025-01-27 00:48:23.992 [info] <0.240.0> Running boot step rabbit_queue_location_validator defined by app rabbit 2025-01-27 00:48:23.992 [info] <0.240.0> Running boot step rabbit_vhost_limit defined by app rabbit 2025-01-27 00:48:23.993 [info] <0.240.0> Running boot step rabbit_mgmt_reset_handler defined by app rabbitmq_management 2025-01-27 00:48:23.993 [info] <0.240.0> Running boot step rabbit_mgmt_db_handler defined by app rabbitmq_management_agent 2025-01-27 00:48:23.993 [warning] <0.240.0> force_fine_statistics set to true; ignored. Replaced by {rates_mode, none} in the rabbitmq_management application. 2025-01-27 00:48:23.993 [info] <0.240.0> Management plugin: using rates mode 'basic' 2025-01-27 00:48:23.994 [info] <0.240.0> Running boot step recovery defined by app rabbit 2025-01-27 00:48:23.995 [info] <0.508.0> Making sure data directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists 2025-01-27 00:48:24.002 [info] <0.508.0> Starting message stores for vhost '/' 2025-01-27 00:48:24.002 [info] <0.512.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index 2025-01-27 00:48:24.003 [info] <0.508.0> Started message store of type transient for vhost '/' 2025-01-27 00:48:24.003 [info] <0.515.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index 2025-01-27 00:48:24.004 [warning] <0.515.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": rebuilding indices from scratch 2025-01-27 00:48:24.005 [info] <0.508.0> Started message store of type persistent for vhost '/' 2025-01-27 00:48:24.006 [info] <0.240.0> Running boot step load_definitions defined by app rabbitmq_management 2025-01-27 00:48:24.006 [info] <0.240.0> Applying definitions from /etc/rabbitmq/definitions.json 2025-01-27 00:48:24.007 [info] <0.240.0> Asked to import definitions. Acting user: rmq-internal 2025-01-27 00:48:24.007 [info] <0.240.0> Importing users... 2025-01-27 00:48:24.013 [info] <0.240.0> Setting user tags for user 'guest' to [administrator] 2025-01-27 00:48:24.018 [info] <0.240.0> Importing vhosts... 2025-01-27 00:48:24.018 [info] <0.240.0> Importing user permissions... 2025-01-27 00:48:24.019 [info] <0.240.0> Setting permissions for 'guest' in '/' to '.*', '.*', '.*' 2025-01-27 00:48:24.024 [info] <0.240.0> Importing topic permissions... 2025-01-27 00:48:24.024 [info] <0.240.0> Importing parameters... 2025-01-27 00:48:24.024 [info] <0.240.0> Importing global parameters... 2025-01-27 00:48:24.024 [info] <0.240.0> Importing policies... 2025-01-27 00:48:24.029 [info] <0.240.0> Importing queues... 2025-01-27 00:48:24.029 [info] <0.240.0> Importing exchanges... 2025-01-27 00:48:24.029 [info] <0.240.0> Importing bindings... 2025-01-27 00:48:24.029 [info] <0.240.0> Running boot step empty_db_check defined by app rabbit 2025-01-27 00:48:24.029 [info] <0.240.0> Running boot step rabbit_looking_glass defined by app rabbit 2025-01-27 00:48:24.029 [info] <0.240.0> Running boot step rabbit_core_metrics_gc defined by app rabbit 2025-01-27 00:48:24.029 [info] <0.240.0> Running boot step background_gc defined by app rabbit 2025-01-27 00:48:24.029 [info] <0.240.0> Running boot step connection_tracking defined by app rabbit 2025-01-27 00:48:24.030 [info] <0.240.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2660-2' 2025-01-27 00:48:24.031 [info] <0.240.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2660-2' 2025-01-27 00:48:24.031 [info] <0.240.0> Running boot step routing_ready defined by app rabbit 2025-01-27 00:48:24.031 [info] <0.240.0> Running boot step pre_flight defined by app rabbit 2025-01-27 00:48:24.031 [info] <0.240.0> Running boot step notify_cluster defined by app rabbit 2025-01-27 00:48:24.031 [info] <0.240.0> Running boot step networking defined by app rabbit 2025-01-27 00:48:24.032 [info] <0.471.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-3' up 2025-01-27 00:48:24.034 [info] <0.556.0> started TCP listener on [::]:5673 2025-01-27 00:48:24.034 [info] <0.240.0> Running boot step cluster_name defined by app rabbit 2025-01-27 00:48:24.034 [info] <0.240.0> Running boot step direct_client defined by app rabbit 2025-01-27 00:48:24.034 [info] <0.240.0> Running boot step os_signal_handler defined by app rabbit 2025-01-27 00:48:24.034 [info] <0.558.0> Swapping OS signal event handler (erl_signal_server) for our own 2025-01-27 00:48:24.046 [info] <0.471.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-1' up 2025-01-27 00:48:24.081 [info] <0.610.0> Management plugin: HTTP (non-TLS) listener started on port 15673 2025-01-27 00:48:24.081 [info] <0.716.0> Statistics database started. 2025-01-27 00:48:24.081 [info] <0.715.0> Starting worker pool 'management_worker_pool' with 3 processes in it 2025-01-27 00:48:24.082 [notice] <0.106.0> Changed loghwm of /var/log/contrail/config-database-rabbitmq/rabbitmq.log to 50 2025-01-27 00:48:24.196 [info] <0.8.0> Server startup complete; 3 plugins started. * rabbitmq_management * rabbitmq_management_agent * rabbitmq_web_dispatch 2025-01-27 00:48:24.257 [info] <0.294.0> RabbitMQ is asked to stop... 2025-01-27 00:48:24.382 [info] <0.294.0> Stopping RabbitMQ applications and their dependencies in the following order: rabbitmq_management amqp_client rabbitmq_web_dispatch cowboy cowlib rabbitmq_management_agent rabbit mnesia rabbit_common sysmon_handler os_mon 2025-01-27 00:48:24.382 [info] <0.294.0> Stopping application 'rabbitmq_management' 2025-01-27 00:48:24.385 [warning] <0.564.0> RabbitMQ HTTP listener registry could not find context rabbitmq_management_tls 2025-01-27 00:48:24.387 [info] <0.43.0> Application rabbitmq_management exited with reason: stopped 2025-01-27 00:48:24.387 [info] <0.294.0> Stopping application 'amqp_client' 2025-01-27 00:48:24.390 [info] <0.43.0> Application amqp_client exited with reason: stopped 2025-01-27 00:48:24.390 [info] <0.294.0> Stopping application 'rabbitmq_web_dispatch' 2025-01-27 00:48:24.393 [info] <0.43.0> Application rabbitmq_web_dispatch exited with reason: stopped 2025-01-27 00:48:24.393 [info] <0.294.0> Stopping application 'cowboy' 2025-01-27 00:48:24.394 [info] <0.294.0> Stopping application 'cowlib' 2025-01-27 00:48:24.394 [info] <0.43.0> Application cowboy exited with reason: stopped 2025-01-27 00:48:24.394 [info] <0.43.0> Application cowlib exited with reason: stopped 2025-01-27 00:48:24.394 [info] <0.294.0> Stopping application 'rabbitmq_management_agent' 2025-01-27 00:48:24.396 [info] <0.43.0> Application rabbitmq_management_agent exited with reason: stopped 2025-01-27 00:48:24.396 [info] <0.294.0> Stopping application 'rabbit' 2025-01-27 00:48:24.396 [info] <0.240.0> Will unregister with peer discovery backend rabbit_peer_discovery_classic_config 2025-01-27 00:48:24.396 [info] <0.556.0> stopped TCP listener on [::]:5673 2025-01-27 00:48:24.397 [info] <0.515.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' 2025-01-27 00:48:24.397 [info] <0.496.0> Closing all connections in vhost '/' on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2' because the vhost is stopping 2025-01-27 00:48:24.409 [info] <0.515.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' is stopped 2025-01-27 00:48:24.410 [info] <0.512.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' 2025-01-27 00:48:24.416 [info] <0.512.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' is stopped 2025-01-27 00:48:24.419 [info] <0.43.0> Application rabbit exited with reason: stopped 2025-01-27 00:48:24.419 [info] <0.294.0> Stopping application 'mnesia' 2025-01-27 00:48:24.422 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-01-27 00:48:24.422 [info] <0.294.0> Stopping application 'rabbit_common' 2025-01-27 00:48:24.422 [info] <0.43.0> Application rabbit_common exited with reason: stopped 2025-01-27 00:48:24.422 [info] <0.294.0> Stopping application 'sysmon_handler' 2025-01-27 00:48:24.423 [info] <0.43.0> Application sysmon_handler exited with reason: stopped 2025-01-27 00:48:24.423 [info] <0.294.0> Stopping application 'os_mon' 2025-01-27 00:48:24.425 [info] <0.43.0> Application os_mon exited with reason: stopped 2025-01-27 00:48:24.425 [info] <0.294.0> Successfully stopped RabbitMQ and its dependencies 2025-01-27 00:48:36.308 [info] <0.740.0> Resetting Rabbit forcefully 2025-01-27 00:48:37.199 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-01-27 00:48:37.204 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-01-27 00:48:37.296 [info] <0.746.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-01-27 00:48:37.296 [info] <0.746.0> Successfully synced tables from a peer 2025-01-27 00:48:37.322 [info] <0.746.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-01-27 00:48:37.322 [info] <0.746.0> Successfully synced tables from a peer 2025-01-27 00:48:37.347 [info] <0.746.0> Waiting for Mnesia tables for 30000 ms, 0 retries left 2025-01-27 00:48:37.347 [info] <0.746.0> Successfully synced tables from a peer 2025-01-27 00:48:37.352 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-01-27 00:48:37.360 [info] <0.746.0> Clustering with ['contrail@cn-jenkins-deploy-platform-ansible-os-2660-3','contrail@cn-jenkins-deploy-platform-ansible-os-2660-1'] as disc node 2025-01-27 00:48:37.422 [info] <0.746.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-01-27 00:48:37.422 [info] <0.746.0> Successfully synced tables from a peer 2025-01-27 00:48:37.873 [info] <0.746.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-01-27 00:48:37.873 [info] <0.746.0> Successfully synced tables from a peer 2025-01-27 00:48:37.903 [info] <0.746.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-01-27 00:48:37.903 [info] <0.746.0> Successfully synced tables from a peer 2025-01-27 00:48:37.909 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-01-27 00:48:37.909 [error] <0.746.0> Failed to create a tracked connection table for node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': {node_not_running,'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2'} 2025-01-27 00:48:37.909 [info] <0.746.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2660-2' 2025-01-27 00:48:37.909 [error] <0.746.0> Failed to create a per-vhost tracked connection table for node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': {node_not_running,'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2'} 2025-01-27 00:48:37.909 [info] <0.746.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2660-2' 2025-01-27 00:48:39.190 [info] <0.1107.0> RabbitMQ is asked to stop... 2025-01-27 00:48:39.293 [info] <0.1107.0> Stopping RabbitMQ applications and their dependencies in the following order: rabbitmq_management amqp_client rabbitmq_management_agent rabbitmq_web_dispatch rabbit mnesia rabbit_common sysmon_handler os_mon 2025-01-27 00:48:39.293 [info] <0.1107.0> Stopping application 'rabbitmq_management' 2025-01-27 00:48:39.293 [info] <0.1107.0> Stopping application 'amqp_client' 2025-01-27 00:48:39.293 [info] <0.1107.0> Stopping application 'rabbitmq_management_agent' 2025-01-27 00:48:39.293 [info] <0.1107.0> Stopping application 'rabbitmq_web_dispatch' 2025-01-27 00:48:39.293 [info] <0.1107.0> Stopping application 'rabbit' 2025-01-27 00:48:39.293 [info] <0.1107.0> Stopping application 'mnesia' 2025-01-27 00:48:39.293 [info] <0.1107.0> Stopping application 'rabbit_common' 2025-01-27 00:48:39.293 [info] <0.1107.0> Stopping application 'sysmon_handler' 2025-01-27 00:48:39.293 [info] <0.1107.0> Stopping application 'os_mon' 2025-01-27 00:48:39.293 [info] <0.1107.0> Successfully stopped RabbitMQ and its dependencies 2025-01-27 00:48:39.293 [info] <0.1107.0> Halting Erlang VM with the following applications: ranch ssl public_key asn1 observer_cli tools credentials_obfuscation crypto stdout_formatter recon inets jsx xmerl lager goldrush compiler syntax_tools sasl stdlib kernel 2025-01-27 00:48:45.831 [info] <0.8.0> Log file opened with Lager 2025-01-27 00:48:47.361 [info] <0.8.0> Feature flags: list of feature flags found: 2025-01-27 00:48:47.361 [info] <0.8.0> Feature flags: feature flag states written to disk: yes 2025-01-27 00:48:47.438 [info] <0.8.0> upgrades: Backing up mnesia dir to "/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2-upgrade-backup" 2025-01-27 00:48:47.440 [info] <0.8.0> upgrades: Mnesia dir backed up to "/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2-upgrade-backup" 2025-01-27 00:48:47.454 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-01-27 00:48:47.544 [info] <0.301.0> Starting RabbitMQ 3.7.28 on Erlang 21.3.8.21 Copyright (c) 2007-2020 Pivotal Software, Inc. Licensed under the MPL. See https://www.rabbitmq.com/ 2025-01-27 00:48:47.545 [info] <0.301.0> node : contrail@cn-jenkins-deploy-platform-ansible-os-2660-2 home dir : /var/lib/rabbitmq config file(s) : /etc/rabbitmq/rabbitmq.config cookie hash : esZsDxSN6VGbi9JkMSxNZA== log(s) : /var/log/contrail/config-database-rabbitmq/rabbitmq.log : /var/log/rabbitmq/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2_upgrade.log database dir : /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2 2025-01-27 00:48:47.567 [info] <0.301.0> Running boot step pre_boot defined by app rabbit 2025-01-27 00:48:47.567 [info] <0.301.0> Running boot step rabbit_core_metrics defined by app rabbit 2025-01-27 00:48:47.568 [info] <0.301.0> Running boot step rabbit_alarm defined by app rabbit 2025-01-27 00:48:47.580 [info] <0.314.0> Memory high watermark set to 12637 MiB (13251084288 bytes) of 15796 MiB (16563855360 bytes) total 2025-01-27 00:48:47.595 [info] <0.326.0> Enabling free disk space monitoring 2025-01-27 00:48:47.595 [info] <0.326.0> Disk free limit set to 50MB 2025-01-27 00:48:47.604 [info] <0.301.0> Running boot step code_server_cache defined by app rabbit 2025-01-27 00:48:47.605 [info] <0.301.0> Running boot step file_handle_cache defined by app rabbit 2025-01-27 00:48:47.605 [info] <0.337.0> Limiting to approx 65436 file handles (58890 sockets) 2025-01-27 00:48:47.605 [info] <0.338.0> FHC read buffering: OFF 2025-01-27 00:48:47.605 [info] <0.338.0> FHC write buffering: ON 2025-01-27 00:48:47.605 [info] <0.301.0> Running boot step worker_pool defined by app rabbit 2025-01-27 00:48:47.605 [info] <0.302.0> Will use 4 processes for default worker pool 2025-01-27 00:48:47.606 [info] <0.302.0> Starting worker pool 'worker_pool' with 4 processes in it 2025-01-27 00:48:47.606 [info] <0.301.0> Running boot step database defined by app rabbit 2025-01-27 00:48:47.623 [info] <0.301.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-01-27 00:48:47.676 [info] <0.301.0> Successfully synced tables from a peer 2025-01-27 00:48:47.678 [info] <0.301.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-01-27 00:48:47.678 [info] <0.301.0> Successfully synced tables from a peer 2025-01-27 00:48:47.714 [info] <0.301.0> upgrades: Mnesia backup removed 2025-01-27 00:48:47.714 [info] <0.301.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-01-27 00:48:47.714 [info] <0.301.0> Successfully synced tables from a peer 2025-01-27 00:48:47.714 [info] <0.301.0> Will register with peer discovery backend rabbit_peer_discovery_classic_config 2025-01-27 00:48:47.714 [info] <0.301.0> Running boot step database_sync defined by app rabbit 2025-01-27 00:48:47.714 [info] <0.301.0> Running boot step feature_flags defined by app rabbit 2025-01-27 00:48:47.715 [info] <0.301.0> Running boot step codec_correctness_check defined by app rabbit 2025-01-27 00:48:47.715 [info] <0.301.0> Running boot step external_infrastructure defined by app rabbit 2025-01-27 00:48:47.715 [info] <0.301.0> Running boot step rabbit_registry defined by app rabbit 2025-01-27 00:48:47.715 [info] <0.301.0> Running boot step rabbit_auth_mechanism_cr_demo defined by app rabbit 2025-01-27 00:48:47.715 [info] <0.301.0> Running boot step rabbit_queue_location_random defined by app rabbit 2025-01-27 00:48:47.715 [info] <0.301.0> Running boot step rabbit_event defined by app rabbit 2025-01-27 00:48:47.715 [info] <0.301.0> Running boot step rabbit_auth_mechanism_amqplain defined by app rabbit 2025-01-27 00:48:47.715 [info] <0.301.0> Running boot step rabbit_auth_mechanism_plain defined by app rabbit 2025-01-27 00:48:47.715 [info] <0.301.0> Running boot step rabbit_exchange_type_direct defined by app rabbit 2025-01-27 00:48:47.715 [info] <0.301.0> Running boot step rabbit_exchange_type_fanout defined by app rabbit 2025-01-27 00:48:47.715 [info] <0.301.0> Running boot step rabbit_exchange_type_headers defined by app rabbit 2025-01-27 00:48:47.715 [info] <0.301.0> Running boot step rabbit_exchange_type_topic defined by app rabbit 2025-01-27 00:48:47.715 [info] <0.301.0> Running boot step rabbit_mirror_queue_mode_all defined by app rabbit 2025-01-27 00:48:47.715 [info] <0.301.0> Running boot step rabbit_mirror_queue_mode_exactly defined by app rabbit 2025-01-27 00:48:47.715 [info] <0.301.0> Running boot step rabbit_mirror_queue_mode_nodes defined by app rabbit 2025-01-27 00:48:47.716 [info] <0.301.0> Running boot step rabbit_priority_queue defined by app rabbit 2025-01-27 00:48:47.716 [info] <0.301.0> Priority queues enabled, real BQ is rabbit_variable_queue 2025-01-27 00:48:47.716 [info] <0.301.0> Running boot step rabbit_queue_location_client_local defined by app rabbit 2025-01-27 00:48:47.716 [info] <0.301.0> Running boot step rabbit_queue_location_min_masters defined by app rabbit 2025-01-27 00:48:47.716 [info] <0.301.0> Running boot step kernel_ready defined by app rabbit 2025-01-27 00:48:47.716 [info] <0.301.0> Running boot step rabbit_sysmon_minder defined by app rabbit 2025-01-27 00:48:47.716 [info] <0.301.0> Running boot step rabbit_epmd_monitor defined by app rabbit 2025-01-27 00:48:47.717 [info] <0.410.0> epmd monitor knows us, inter-node communication (distribution) port: 25673 2025-01-27 00:48:47.717 [info] <0.301.0> Running boot step guid_generator defined by app rabbit 2025-01-27 00:48:47.720 [info] <0.301.0> Running boot step rabbit_node_monitor defined by app rabbit 2025-01-27 00:48:47.721 [info] <0.414.0> Starting rabbit_node_monitor 2025-01-27 00:48:47.722 [info] <0.301.0> Running boot step delegate_sup defined by app rabbit 2025-01-27 00:48:47.722 [info] <0.301.0> Running boot step rabbit_memory_monitor defined by app rabbit 2025-01-27 00:48:47.724 [info] <0.301.0> Running boot step core_initialized defined by app rabbit 2025-01-27 00:48:47.724 [info] <0.301.0> Running boot step upgrade_queues defined by app rabbit 2025-01-27 00:48:47.745 [info] <0.301.0> message_store upgrades: 1 to apply 2025-01-27 00:48:47.745 [info] <0.301.0> message_store upgrades: Applying rabbit_variable_queue:move_messages_to_vhost_store 2025-01-27 00:48:47.745 [info] <0.301.0> message_store upgrades: No durable queues found. Skipping message store migration 2025-01-27 00:48:47.745 [info] <0.301.0> message_store upgrades: Removing the old message store data 2025-01-27 00:48:47.745 [info] <0.301.0> message_store upgrades: All upgrades applied successfully 2025-01-27 00:48:47.769 [info] <0.301.0> Running boot step rabbit_connection_tracking defined by app rabbit 2025-01-27 00:48:47.769 [info] <0.301.0> Running boot step rabbit_connection_tracking_handler defined by app rabbit 2025-01-27 00:48:47.769 [info] <0.301.0> Running boot step rabbit_exchange_parameters defined by app rabbit 2025-01-27 00:48:47.769 [info] <0.301.0> Running boot step rabbit_mirror_queue_misc defined by app rabbit 2025-01-27 00:48:47.770 [info] <0.301.0> Running boot step rabbit_policies defined by app rabbit 2025-01-27 00:48:47.770 [info] <0.301.0> Running boot step rabbit_policy defined by app rabbit 2025-01-27 00:48:47.770 [info] <0.301.0> Running boot step rabbit_queue_location_validator defined by app rabbit 2025-01-27 00:48:47.770 [info] <0.301.0> Running boot step rabbit_vhost_limit defined by app rabbit 2025-01-27 00:48:47.770 [info] <0.301.0> Running boot step rabbit_mgmt_reset_handler defined by app rabbitmq_management 2025-01-27 00:48:47.770 [info] <0.301.0> Running boot step rabbit_mgmt_db_handler defined by app rabbitmq_management_agent 2025-01-27 00:48:47.771 [warning] <0.301.0> force_fine_statistics set to true; ignored. Replaced by {rates_mode, none} in the rabbitmq_management application. 2025-01-27 00:48:47.771 [info] <0.301.0> Management plugin: using rates mode 'basic' 2025-01-27 00:48:47.772 [info] <0.301.0> Running boot step recovery defined by app rabbit 2025-01-27 00:48:47.773 [info] <0.451.0> Making sure data directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists 2025-01-27 00:48:47.778 [info] <0.451.0> Starting message stores for vhost '/' 2025-01-27 00:48:47.778 [info] <0.455.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index 2025-01-27 00:48:47.779 [info] <0.451.0> Started message store of type transient for vhost '/' 2025-01-27 00:48:47.779 [info] <0.458.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index 2025-01-27 00:48:47.779 [warning] <0.458.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": rebuilding indices from scratch 2025-01-27 00:48:47.780 [info] <0.451.0> Started message store of type persistent for vhost '/' 2025-01-27 00:48:47.781 [info] <0.451.0> Mirrored queue 'job_status_consumer.dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.477.0> 2025-01-27 00:48:47.781 [info] <0.451.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2660-2..tftp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.481.0> 2025-01-27 00:48:47.782 [info] <0.451.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2660-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.485.0> 2025-01-27 00:48:47.782 [info] <0.451.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2660-1.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.489.0> 2025-01-27 00:48:47.782 [info] <0.451.0> Mirrored queue 'job_request_consumer' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.493.0> 2025-01-27 00:48:47.782 [info] <0.451.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2660-2.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.497.0> 2025-01-27 00:48:47.783 [info] <0.451.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2660-3.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.500.0> 2025-01-27 00:48:47.783 [info] <0.451.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2660-3..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.505.0> 2025-01-27 00:48:47.783 [info] <0.451.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2660-2..config_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.509.0> 2025-01-27 00:48:47.783 [info] <0.451.0> Mirrored queue 'schema_transformer.cn-jenkins-deploy-platform-ansible-os-2660-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.513.0> 2025-01-27 00:48:47.783 [info] <0.451.0> Mirrored queue 'svc_monitor.cn-jenkins-deploy-platform-ansible-os-2660-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.516.0> 2025-01-27 00:48:47.784 [info] <0.451.0> Mirrored queue 'device_manager_ztp.ztp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.521.0> 2025-01-27 00:48:47.784 [info] <0.451.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2660-1..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.525.0> 2025-01-27 00:48:47.784 [info] <0.451.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2660-2..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.529.0> 2025-01-27 00:48:47.785 [info] <0.301.0> Running boot step load_definitions defined by app rabbitmq_management 2025-01-27 00:48:47.786 [info] <0.301.0> Applying definitions from /etc/rabbitmq/definitions.json 2025-01-27 00:48:47.786 [info] <0.301.0> Asked to import definitions. Acting user: rmq-internal 2025-01-27 00:48:47.786 [info] <0.301.0> Importing users... 2025-01-27 00:48:47.799 [info] <0.301.0> Setting user tags for user 'guest' to [administrator] 2025-01-27 00:48:47.816 [info] <0.301.0> Importing vhosts... 2025-01-27 00:48:47.816 [info] <0.301.0> Importing user permissions... 2025-01-27 00:48:47.816 [info] <0.301.0> Setting permissions for 'guest' in '/' to '.*', '.*', '.*' 2025-01-27 00:48:47.833 [info] <0.301.0> Importing topic permissions... 2025-01-27 00:48:47.833 [info] <0.301.0> Importing parameters... 2025-01-27 00:48:47.833 [info] <0.301.0> Importing global parameters... 2025-01-27 00:48:47.833 [info] <0.301.0> Importing policies... 2025-01-27 00:48:47.845 [info] <0.301.0> Importing queues... 2025-01-27 00:48:47.845 [info] <0.301.0> Importing exchanges... 2025-01-27 00:48:47.845 [info] <0.301.0> Importing bindings... 2025-01-27 00:48:47.845 [info] <0.301.0> Running boot step empty_db_check defined by app rabbit 2025-01-27 00:48:47.845 [info] <0.301.0> Running boot step rabbit_looking_glass defined by app rabbit 2025-01-27 00:48:47.845 [info] <0.301.0> Running boot step rabbit_core_metrics_gc defined by app rabbit 2025-01-27 00:48:47.846 [info] <0.301.0> Running boot step background_gc defined by app rabbit 2025-01-27 00:48:47.846 [info] <0.301.0> Running boot step connection_tracking defined by app rabbit 2025-01-27 00:48:47.873 [info] <0.301.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2660-2' 2025-01-27 00:48:47.893 [info] <0.301.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2660-2' 2025-01-27 00:48:47.893 [info] <0.301.0> Running boot step routing_ready defined by app rabbit 2025-01-27 00:48:47.893 [info] <0.301.0> Running boot step pre_flight defined by app rabbit 2025-01-27 00:48:47.893 [info] <0.301.0> Running boot step notify_cluster defined by app rabbit 2025-01-27 00:48:47.893 [info] <0.301.0> Running boot step networking defined by app rabbit 2025-01-27 00:48:47.894 [info] <0.414.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-1' up 2025-01-27 00:48:47.896 [info] <0.603.0> started TCP listener on [::]:5673 2025-01-27 00:48:47.896 [info] <0.301.0> Running boot step cluster_name defined by app rabbit 2025-01-27 00:48:47.896 [info] <0.301.0> Running boot step direct_client defined by app rabbit 2025-01-27 00:48:47.896 [info] <0.301.0> Running boot step os_signal_handler defined by app rabbit 2025-01-27 00:48:47.896 [info] <0.605.0> Swapping OS signal event handler (erl_signal_server) for our own 2025-01-27 00:48:47.942 [info] <0.414.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-3' up 2025-01-27 00:48:47.989 [info] <0.685.0> Management plugin: HTTP (non-TLS) listener started on port 15673 2025-01-27 00:48:47.989 [info] <0.791.0> Statistics database started. 2025-01-27 00:48:47.989 [info] <0.790.0> Starting worker pool 'management_worker_pool' with 3 processes in it 2025-01-27 00:48:47.990 [notice] <0.106.0> Changed loghwm of /var/log/contrail/config-database-rabbitmq/rabbitmq.log to 50 2025-01-27 00:48:48.142 [info] <0.8.0> Server startup complete; 3 plugins started. * rabbitmq_management * rabbitmq_management_agent * rabbitmq_web_dispatch 2025-01-27 02:06:39.035 [info] <0.1142.0> RabbitMQ is asked to stop... 2025-01-27 02:06:39.089 [info] <0.1142.0> Stopping RabbitMQ applications and their dependencies in the following order: rabbitmq_management amqp_client rabbitmq_web_dispatch cowboy cowlib rabbitmq_management_agent rabbit mnesia rabbit_common sysmon_handler os_mon 2025-01-27 02:06:39.089 [info] <0.1142.0> Stopping application 'rabbitmq_management' 2025-01-27 02:06:39.090 [warning] <0.611.0> RabbitMQ HTTP listener registry could not find context rabbitmq_management_tls 2025-01-27 02:06:39.092 [info] <0.43.0> Application rabbitmq_management exited with reason: stopped 2025-01-27 02:06:39.092 [info] <0.1142.0> Stopping application 'amqp_client' 2025-01-27 02:06:39.094 [info] <0.43.0> Application amqp_client exited with reason: stopped 2025-01-27 02:06:39.094 [info] <0.1142.0> Stopping application 'rabbitmq_web_dispatch' 2025-01-27 02:06:39.095 [info] <0.1142.0> Stopping application 'cowboy' 2025-01-27 02:06:39.095 [info] <0.43.0> Application rabbitmq_web_dispatch exited with reason: stopped 2025-01-27 02:06:39.096 [info] <0.43.0> Application cowboy exited with reason: stopped 2025-01-27 02:06:39.096 [info] <0.1142.0> Stopping application 'cowlib' 2025-01-27 02:06:39.096 [info] <0.43.0> Application cowlib exited with reason: stopped 2025-01-27 02:06:39.096 [info] <0.1142.0> Stopping application 'rabbitmq_management_agent' 2025-01-27 02:06:39.098 [info] <0.43.0> Application rabbitmq_management_agent exited with reason: stopped 2025-01-27 02:06:39.098 [info] <0.1142.0> Stopping application 'rabbit' 2025-01-27 02:06:39.098 [info] <0.301.0> Will unregister with peer discovery backend rabbit_peer_discovery_classic_config 2025-01-27 02:06:39.098 [info] <0.603.0> stopped TCP listener on [::]:5673 2025-01-27 02:06:39.098 [info] <0.439.0> Closing all connections in vhost '/' on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2' because the vhost is stopping 2025-01-27 02:06:39.099 [info] <0.458.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' 2025-01-27 02:06:39.105 [info] <0.458.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' is stopped 2025-01-27 02:06:39.105 [info] <0.455.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' 2025-01-27 02:06:39.108 [info] <0.455.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' is stopped 2025-01-27 02:06:39.115 [info] <0.1142.0> Stopping application 'mnesia' 2025-01-27 02:06:39.115 [info] <0.43.0> Application rabbit exited with reason: stopped 2025-01-27 02:06:39.120 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-01-27 02:06:39.120 [info] <0.1142.0> Stopping application 'rabbit_common' 2025-01-27 02:06:39.121 [info] <0.43.0> Application rabbit_common exited with reason: stopped 2025-01-27 02:06:39.121 [info] <0.1142.0> Stopping application 'sysmon_handler' 2025-01-27 02:06:39.122 [info] <0.43.0> Application sysmon_handler exited with reason: stopped 2025-01-27 02:06:39.122 [info] <0.1142.0> Stopping application 'os_mon' 2025-01-27 02:06:39.123 [info] <0.43.0> Application os_mon exited with reason: stopped 2025-01-27 02:06:39.123 [info] <0.1142.0> Successfully stopped RabbitMQ and its dependencies 2025-01-27 02:06:39.123 [info] <0.1142.0> Halting Erlang VM with the following applications: ranch ssl public_key asn1 observer_cli tools credentials_obfuscation crypto stdout_formatter recon inets jsx xmerl lager goldrush compiler syntax_tools sasl stdlib kernel 2025-01-27 02:07:07.460 [info] <0.8.0> Log file opened with Lager 2025-01-27 02:07:13.515 [info] <0.8.0> Feature flags: list of feature flags found: 2025-01-27 02:07:13.516 [info] <0.8.0> Feature flags: feature flag states written to disk: yes 2025-01-27 02:07:13.646 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-01-27 02:07:13.710 [info] <0.299.0> Starting RabbitMQ 3.7.28 on Erlang 21.3.8.21 Copyright (c) 2007-2020 Pivotal Software, Inc. Licensed under the MPL. See https://www.rabbitmq.com/ 2025-01-27 02:07:13.711 [info] <0.299.0> node : contrail@cn-jenkins-deploy-platform-ansible-os-2660-2 home dir : /var/lib/rabbitmq config file(s) : /etc/rabbitmq/rabbitmq.config cookie hash : esZsDxSN6VGbi9JkMSxNZA== log(s) : /var/log/contrail/config-database-rabbitmq/rabbitmq.log : /var/log/rabbitmq/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2_upgrade.log database dir : /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2 2025-01-27 02:07:13.771 [info] <0.299.0> Running boot step pre_boot defined by app rabbit 2025-01-27 02:07:13.771 [info] <0.299.0> Running boot step rabbit_core_metrics defined by app rabbit 2025-01-27 02:07:13.772 [info] <0.299.0> Running boot step rabbit_alarm defined by app rabbit 2025-01-27 02:07:13.798 [info] <0.335.0> Memory high watermark set to 12637 MiB (13251084288 bytes) of 15796 MiB (16563855360 bytes) total 2025-01-27 02:07:13.831 [info] <0.348.0> Enabling free disk space monitoring 2025-01-27 02:07:13.831 [info] <0.348.0> Disk free limit set to 50MB 2025-01-27 02:07:13.843 [info] <0.299.0> Running boot step code_server_cache defined by app rabbit 2025-01-27 02:07:13.843 [info] <0.299.0> Running boot step file_handle_cache defined by app rabbit 2025-01-27 02:07:13.850 [info] <0.377.0> FHC read buffering: OFF 2025-01-27 02:07:13.850 [info] <0.377.0> FHC write buffering: ON 2025-01-27 02:07:13.850 [info] <0.373.0> Limiting to approx 65436 file handles (58890 sockets) 2025-01-27 02:07:13.850 [info] <0.299.0> Running boot step worker_pool defined by app rabbit 2025-01-27 02:07:13.850 [info] <0.300.0> Will use 4 processes for default worker pool 2025-01-27 02:07:13.850 [info] <0.300.0> Starting worker pool 'worker_pool' with 4 processes in it 2025-01-27 02:07:13.851 [info] <0.299.0> Running boot step database defined by app rabbit 2025-01-27 02:07:13.900 [info] <0.299.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-01-27 02:07:13.901 [info] <0.299.0> Successfully synced tables from a peer 2025-01-27 02:07:13.919 [info] <0.299.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-01-27 02:07:13.920 [info] <0.299.0> Successfully synced tables from a peer 2025-01-27 02:07:14.280 [info] <0.299.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-01-27 02:07:14.281 [info] <0.299.0> Successfully synced tables from a peer 2025-01-27 02:07:14.281 [info] <0.299.0> Will register with peer discovery backend rabbit_peer_discovery_classic_config 2025-01-27 02:07:14.283 [info] <0.299.0> Running boot step database_sync defined by app rabbit 2025-01-27 02:07:14.283 [info] <0.299.0> Running boot step feature_flags defined by app rabbit 2025-01-27 02:07:14.283 [info] <0.299.0> Running boot step codec_correctness_check defined by app rabbit 2025-01-27 02:07:14.283 [info] <0.299.0> Running boot step external_infrastructure defined by app rabbit 2025-01-27 02:07:14.283 [info] <0.299.0> Running boot step rabbit_registry defined by app rabbit 2025-01-27 02:07:14.284 [info] <0.299.0> Running boot step rabbit_auth_mechanism_cr_demo defined by app rabbit 2025-01-27 02:07:14.284 [info] <0.299.0> Running boot step rabbit_queue_location_random defined by app rabbit 2025-01-27 02:07:14.284 [info] <0.299.0> Running boot step rabbit_event defined by app rabbit 2025-01-27 02:07:14.284 [info] <0.299.0> Running boot step rabbit_auth_mechanism_amqplain defined by app rabbit 2025-01-27 02:07:14.284 [info] <0.299.0> Running boot step rabbit_auth_mechanism_plain defined by app rabbit 2025-01-27 02:07:14.284 [info] <0.299.0> Running boot step rabbit_exchange_type_direct defined by app rabbit 2025-01-27 02:07:14.284 [info] <0.299.0> Running boot step rabbit_exchange_type_fanout defined by app rabbit 2025-01-27 02:07:14.284 [info] <0.299.0> Running boot step rabbit_exchange_type_headers defined by app rabbit 2025-01-27 02:07:14.284 [info] <0.299.0> Running boot step rabbit_exchange_type_topic defined by app rabbit 2025-01-27 02:07:14.284 [info] <0.299.0> Running boot step rabbit_mirror_queue_mode_all defined by app rabbit 2025-01-27 02:07:14.284 [info] <0.299.0> Running boot step rabbit_mirror_queue_mode_exactly defined by app rabbit 2025-01-27 02:07:14.284 [info] <0.299.0> Running boot step rabbit_mirror_queue_mode_nodes defined by app rabbit 2025-01-27 02:07:14.285 [info] <0.299.0> Running boot step rabbit_priority_queue defined by app rabbit 2025-01-27 02:07:14.285 [info] <0.299.0> Priority queues enabled, real BQ is rabbit_variable_queue 2025-01-27 02:07:14.285 [info] <0.299.0> Running boot step rabbit_queue_location_client_local defined by app rabbit 2025-01-27 02:07:14.285 [info] <0.299.0> Running boot step rabbit_queue_location_min_masters defined by app rabbit 2025-01-27 02:07:14.285 [info] <0.299.0> Running boot step kernel_ready defined by app rabbit 2025-01-27 02:07:14.285 [info] <0.299.0> Running boot step rabbit_sysmon_minder defined by app rabbit 2025-01-27 02:07:14.285 [info] <0.299.0> Running boot step rabbit_epmd_monitor defined by app rabbit 2025-01-27 02:07:14.317 [info] <0.411.0> epmd monitor knows us, inter-node communication (distribution) port: 25673 2025-01-27 02:07:14.317 [info] <0.299.0> Running boot step guid_generator defined by app rabbit 2025-01-27 02:07:14.357 [info] <0.299.0> Running boot step rabbit_node_monitor defined by app rabbit 2025-01-27 02:07:14.366 [info] <0.415.0> Starting rabbit_node_monitor 2025-01-27 02:07:14.366 [info] <0.299.0> Running boot step delegate_sup defined by app rabbit 2025-01-27 02:07:14.366 [info] <0.299.0> Running boot step rabbit_memory_monitor defined by app rabbit 2025-01-27 02:07:14.367 [info] <0.299.0> Running boot step core_initialized defined by app rabbit 2025-01-27 02:07:14.367 [info] <0.299.0> Running boot step upgrade_queues defined by app rabbit 2025-01-27 02:07:14.436 [info] <0.299.0> Running boot step rabbit_connection_tracking defined by app rabbit 2025-01-27 02:07:14.437 [info] <0.299.0> Running boot step rabbit_connection_tracking_handler defined by app rabbit 2025-01-27 02:07:14.437 [info] <0.299.0> Running boot step rabbit_exchange_parameters defined by app rabbit 2025-01-27 02:07:14.437 [info] <0.299.0> Running boot step rabbit_mirror_queue_misc defined by app rabbit 2025-01-27 02:07:14.437 [info] <0.299.0> Running boot step rabbit_policies defined by app rabbit 2025-01-27 02:07:14.439 [info] <0.299.0> Running boot step rabbit_policy defined by app rabbit 2025-01-27 02:07:14.439 [info] <0.299.0> Running boot step rabbit_queue_location_validator defined by app rabbit 2025-01-27 02:07:14.439 [info] <0.299.0> Running boot step rabbit_vhost_limit defined by app rabbit 2025-01-27 02:07:14.439 [info] <0.299.0> Running boot step rabbit_mgmt_reset_handler defined by app rabbitmq_management 2025-01-27 02:07:14.439 [info] <0.299.0> Running boot step rabbit_mgmt_db_handler defined by app rabbitmq_management_agent 2025-01-27 02:07:14.439 [warning] <0.299.0> force_fine_statistics set to true; ignored. Replaced by {rates_mode, none} in the rabbitmq_management application. 2025-01-27 02:07:14.439 [info] <0.299.0> Management plugin: using rates mode 'basic' 2025-01-27 02:07:14.444 [info] <0.299.0> Running boot step recovery defined by app rabbit 2025-01-27 02:07:14.465 [info] <0.455.0> Making sure data directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists 2025-01-27 02:07:14.520 [info] <0.455.0> Starting message stores for vhost '/' 2025-01-27 02:07:14.521 [info] <0.459.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index 2025-01-27 02:07:14.553 [info] <0.455.0> Started message store of type transient for vhost '/' 2025-01-27 02:07:14.553 [info] <0.462.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index 2025-01-27 02:07:14.587 [info] <0.455.0> Started message store of type persistent for vhost '/' 2025-01-27 02:07:14.598 [info] <0.455.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2660-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.481.0> 2025-01-27 02:07:14.599 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2660-2.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2660-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.485.0> 2025-01-27 02:07:14.599 [info] <0.455.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2660-2..tftp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.489.0> 2025-01-27 02:07:14.599 [info] <0.455.0> Mirrored queue 'device_manager.cn-jenkins-deploy-platform-ansible-os-2660-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.493.0> 2025-01-27 02:07:14.599 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2660-3.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2660-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.497.0> 2025-01-27 02:07:14.600 [info] <0.455.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2660-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.501.0> 2025-01-27 02:07:14.600 [info] <0.455.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2660-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.505.0> 2025-01-27 02:07:14.612 [info] <0.455.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2660-1.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.509.0> 2025-01-27 02:07:14.613 [info] <0.455.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2660-1' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.513.0> 2025-01-27 02:07:14.613 [info] <0.455.0> Mirrored queue 'job_request_consumer' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.517.0> 2025-01-27 02:07:14.613 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2660-1.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2660-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.521.0> 2025-01-27 02:07:14.614 [info] <0.455.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2660-2.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.525.0> 2025-01-27 02:07:14.614 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2660-1.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2660-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.529.0> 2025-01-27 02:07:14.614 [info] <0.455.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2660-2' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.533.0> 2025-01-27 02:07:14.614 [info] <0.455.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2660-1' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.537.0> 2025-01-27 02:07:14.615 [info] <0.455.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2660-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.541.0> 2025-01-27 02:07:14.615 [info] <0.455.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2660-3' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.545.0> 2025-01-27 02:07:14.615 [info] <0.455.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2660-3.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.549.0> 2025-01-27 02:07:14.615 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2660-3.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2660-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.553.0> 2025-01-27 02:07:14.616 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2660-2.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2660-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.557.0> 2025-01-27 02:07:14.616 [info] <0.455.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2660-2..config_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.561.0> 2025-01-27 02:07:14.616 [info] <0.455.0> Mirrored queue 'device_manager_ztp.ztp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.565.0> 2025-01-27 02:07:14.617 [info] <0.455.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2660-2' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.569.0> 2025-01-27 02:07:14.617 [info] <0.455.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2660-3' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.573.0> 2025-01-27 02:07:14.617 [info] <0.455.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2660-1..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.577.0> 2025-01-27 02:07:14.617 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2660-1.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2660-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.581.0> 2025-01-27 02:07:14.648 [info] <0.299.0> Running boot step load_definitions defined by app rabbitmq_management 2025-01-27 02:07:14.669 [info] <0.299.0> Applying definitions from /etc/rabbitmq/definitions.json 2025-01-27 02:07:14.669 [info] <0.299.0> Asked to import definitions. Acting user: rmq-internal 2025-01-27 02:07:14.669 [info] <0.299.0> Importing users... 2025-01-27 02:07:14.730 [info] <0.299.0> Setting user tags for user 'guest' to [administrator] 2025-01-27 02:07:14.799 [info] <0.299.0> Importing vhosts... 2025-01-27 02:07:14.799 [info] <0.299.0> Importing user permissions... 2025-01-27 02:07:14.799 [info] <0.299.0> Setting permissions for 'guest' in '/' to '.*', '.*', '.*' 2025-01-27 02:07:14.843 [info] <0.299.0> Importing topic permissions... 2025-01-27 02:07:14.843 [info] <0.299.0> Importing parameters... 2025-01-27 02:07:14.843 [info] <0.299.0> Importing global parameters... 2025-01-27 02:07:14.843 [info] <0.299.0> Importing policies... 2025-01-27 02:07:14.853 [info] <0.299.0> Importing queues... 2025-01-27 02:07:14.853 [info] <0.299.0> Importing exchanges... 2025-01-27 02:07:14.853 [info] <0.299.0> Importing bindings... 2025-01-27 02:07:14.853 [info] <0.299.0> Running boot step empty_db_check defined by app rabbit 2025-01-27 02:07:14.853 [info] <0.299.0> Running boot step rabbit_looking_glass defined by app rabbit 2025-01-27 02:07:14.853 [info] <0.299.0> Running boot step rabbit_core_metrics_gc defined by app rabbit 2025-01-27 02:07:14.853 [info] <0.299.0> Running boot step background_gc defined by app rabbit 2025-01-27 02:07:14.854 [info] <0.299.0> Running boot step connection_tracking defined by app rabbit 2025-01-27 02:07:14.864 [info] <0.299.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2660-2' 2025-01-27 02:07:14.868 [info] <0.299.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2660-2' 2025-01-27 02:07:14.870 [info] <0.299.0> Running boot step routing_ready defined by app rabbit 2025-01-27 02:07:14.870 [info] <0.299.0> Running boot step pre_flight defined by app rabbit 2025-01-27 02:07:14.870 [info] <0.299.0> Running boot step notify_cluster defined by app rabbit 2025-01-27 02:07:14.871 [info] <0.299.0> Running boot step networking defined by app rabbit 2025-01-27 02:07:14.876 [info] <0.415.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-1' up 2025-01-27 02:07:14.889 [info] <0.685.0> started TCP listener on [::]:5673 2025-01-27 02:07:14.896 [info] <0.299.0> Running boot step cluster_name defined by app rabbit 2025-01-27 02:07:14.896 [info] <0.299.0> Running boot step direct_client defined by app rabbit 2025-01-27 02:07:14.896 [info] <0.299.0> Running boot step os_signal_handler defined by app rabbit 2025-01-27 02:07:14.896 [info] <0.688.0> Swapping OS signal event handler (erl_signal_server) for our own 2025-01-27 02:07:15.138 [info] <0.764.0> Management plugin: HTTP (non-TLS) listener started on port 15673 2025-01-27 02:07:15.139 [info] <0.870.0> Statistics database started. 2025-01-27 02:07:15.139 [info] <0.869.0> Starting worker pool 'management_worker_pool' with 3 processes in it 2025-01-27 02:07:15.140 [notice] <0.106.0> Changed loghwm of /var/log/contrail/config-database-rabbitmq/rabbitmq.log to 50 2025-01-27 02:07:15.651 [info] <0.8.0> Server startup complete; 3 plugins started. * rabbitmq_management * rabbitmq_web_dispatch * rabbitmq_management_agent 2025-01-27 02:07:19.242 [info] <0.974.0> RabbitMQ is asked to stop... 2025-01-27 02:07:19.501 [info] <0.974.0> Stopping RabbitMQ applications and their dependencies in the following order: rabbitmq_management amqp_client rabbitmq_web_dispatch cowboy cowlib rabbitmq_management_agent rabbit mnesia rabbit_common sysmon_handler os_mon 2025-01-27 02:07:19.501 [info] <0.974.0> Stopping application 'rabbitmq_management' 2025-01-27 02:07:19.503 [warning] <0.760.0> RabbitMQ HTTP listener registry could not find context rabbitmq_management_tls 2025-01-27 02:07:19.505 [info] <0.974.0> Stopping application 'amqp_client' 2025-01-27 02:07:19.505 [info] <0.43.0> Application rabbitmq_management exited with reason: stopped 2025-01-27 02:07:19.507 [info] <0.43.0> Application amqp_client exited with reason: stopped 2025-01-27 02:07:19.507 [info] <0.974.0> Stopping application 'rabbitmq_web_dispatch' 2025-01-27 02:07:19.508 [info] <0.974.0> Stopping application 'cowboy' 2025-01-27 02:07:19.508 [info] <0.43.0> Application rabbitmq_web_dispatch exited with reason: stopped 2025-01-27 02:07:19.510 [info] <0.43.0> Application cowboy exited with reason: stopped 2025-01-27 02:07:19.510 [info] <0.974.0> Stopping application 'cowlib' 2025-01-27 02:07:19.511 [info] <0.43.0> Application cowlib exited with reason: stopped 2025-01-27 02:07:19.511 [info] <0.974.0> Stopping application 'rabbitmq_management_agent' 2025-01-27 02:07:19.512 [info] <0.43.0> Application rabbitmq_management_agent exited with reason: stopped 2025-01-27 02:07:19.512 [info] <0.974.0> Stopping application 'rabbit' 2025-01-27 02:07:19.513 [info] <0.299.0> Will unregister with peer discovery backend rabbit_peer_discovery_classic_config 2025-01-27 02:07:19.514 [info] <0.685.0> stopped TCP listener on [::]:5673 2025-01-27 02:07:19.514 [info] <0.443.0> Closing all connections in vhost '/' on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2' because the vhost is stopping 2025-01-27 02:07:19.517 [info] <0.462.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' 2025-01-27 02:07:19.628 [info] <0.462.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' is stopped 2025-01-27 02:07:19.629 [info] <0.459.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' 2025-01-27 02:07:19.633 [info] <0.459.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' is stopped 2025-01-27 02:07:19.640 [info] <0.43.0> Application rabbit exited with reason: stopped 2025-01-27 02:07:19.640 [info] <0.974.0> Stopping application 'mnesia' 2025-01-27 02:07:19.646 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-01-27 02:07:19.646 [info] <0.974.0> Stopping application 'rabbit_common' 2025-01-27 02:07:19.646 [info] <0.43.0> Application rabbit_common exited with reason: stopped 2025-01-27 02:07:19.647 [info] <0.974.0> Stopping application 'sysmon_handler' 2025-01-27 02:07:19.648 [info] <0.43.0> Application sysmon_handler exited with reason: stopped 2025-01-27 02:07:19.648 [info] <0.974.0> Stopping application 'os_mon' 2025-01-27 02:07:19.649 [info] <0.43.0> Application os_mon exited with reason: stopped 2025-01-27 02:07:19.649 [info] <0.974.0> Successfully stopped RabbitMQ and its dependencies 2025-01-27 02:07:19.649 [info] <0.974.0> Halting Erlang VM with the following applications: jsx ranch ssl public_key asn1 observer_cli xmerl tools credentials_obfuscation crypto stdout_formatter recon inets lager goldrush compiler syntax_tools sasl stdlib kernel 2025-01-27 02:07:38.195 [info] <0.8.0> Log file opened with Lager 2025-01-27 02:07:43.847 [info] <0.8.0> Feature flags: list of feature flags found: 2025-01-27 02:07:43.848 [info] <0.8.0> Feature flags: feature flag states written to disk: yes 2025-01-27 02:07:43.992 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-01-27 02:07:44.120 [info] <0.311.0> Starting RabbitMQ 3.7.28 on Erlang 21.3.8.21 Copyright (c) 2007-2020 Pivotal Software, Inc. Licensed under the MPL. See https://www.rabbitmq.com/ 2025-01-27 02:07:44.121 [info] <0.311.0> node : contrail@cn-jenkins-deploy-platform-ansible-os-2660-2 home dir : /var/lib/rabbitmq config file(s) : /etc/rabbitmq/rabbitmq.config cookie hash : esZsDxSN6VGbi9JkMSxNZA== log(s) : /var/log/contrail/config-database-rabbitmq/rabbitmq.log : /var/log/rabbitmq/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2_upgrade.log database dir : /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2 2025-01-27 02:07:44.138 [info] <0.311.0> Running boot step pre_boot defined by app rabbit 2025-01-27 02:07:44.138 [info] <0.311.0> Running boot step rabbit_core_metrics defined by app rabbit 2025-01-27 02:07:44.138 [info] <0.311.0> Running boot step rabbit_alarm defined by app rabbit 2025-01-27 02:07:44.165 [info] <0.317.0> Memory high watermark set to 12637 MiB (13251084288 bytes) of 15796 MiB (16563855360 bytes) total 2025-01-27 02:07:44.174 [info] <0.340.0> Enabling free disk space monitoring 2025-01-27 02:07:44.174 [info] <0.340.0> Disk free limit set to 50MB 2025-01-27 02:07:44.179 [info] <0.311.0> Running boot step code_server_cache defined by app rabbit 2025-01-27 02:07:44.180 [info] <0.311.0> Running boot step file_handle_cache defined by app rabbit 2025-01-27 02:07:44.185 [info] <0.358.0> Limiting to approx 65436 file handles (58890 sockets) 2025-01-27 02:07:44.185 [info] <0.359.0> FHC read buffering: OFF 2025-01-27 02:07:44.185 [info] <0.359.0> FHC write buffering: ON 2025-01-27 02:07:44.186 [info] <0.311.0> Running boot step worker_pool defined by app rabbit 2025-01-27 02:07:44.187 [info] <0.312.0> Will use 4 processes for default worker pool 2025-01-27 02:07:44.187 [info] <0.312.0> Starting worker pool 'worker_pool' with 4 processes in it 2025-01-27 02:07:44.187 [info] <0.311.0> Running boot step database defined by app rabbit 2025-01-27 02:07:44.249 [info] <0.311.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-01-27 02:07:44.249 [info] <0.311.0> Successfully synced tables from a peer 2025-01-27 02:07:44.251 [info] <0.311.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-01-27 02:07:44.251 [info] <0.311.0> Successfully synced tables from a peer 2025-01-27 02:07:44.292 [info] <0.311.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-01-27 02:07:44.292 [info] <0.311.0> Successfully synced tables from a peer 2025-01-27 02:07:44.292 [info] <0.311.0> Will register with peer discovery backend rabbit_peer_discovery_classic_config 2025-01-27 02:07:44.292 [info] <0.311.0> Running boot step database_sync defined by app rabbit 2025-01-27 02:07:44.293 [info] <0.311.0> Running boot step feature_flags defined by app rabbit 2025-01-27 02:07:44.293 [info] <0.311.0> Running boot step codec_correctness_check defined by app rabbit 2025-01-27 02:07:44.293 [info] <0.311.0> Running boot step external_infrastructure defined by app rabbit 2025-01-27 02:07:44.293 [info] <0.311.0> Running boot step rabbit_registry defined by app rabbit 2025-01-27 02:07:44.293 [info] <0.311.0> Running boot step rabbit_auth_mechanism_cr_demo defined by app rabbit 2025-01-27 02:07:44.293 [info] <0.311.0> Running boot step rabbit_queue_location_random defined by app rabbit 2025-01-27 02:07:44.293 [info] <0.311.0> Running boot step rabbit_event defined by app rabbit 2025-01-27 02:07:44.293 [info] <0.311.0> Running boot step rabbit_auth_mechanism_amqplain defined by app rabbit 2025-01-27 02:07:44.293 [info] <0.311.0> Running boot step rabbit_auth_mechanism_plain defined by app rabbit 2025-01-27 02:07:44.293 [info] <0.311.0> Running boot step rabbit_exchange_type_direct defined by app rabbit 2025-01-27 02:07:44.293 [info] <0.311.0> Running boot step rabbit_exchange_type_fanout defined by app rabbit 2025-01-27 02:07:44.293 [info] <0.311.0> Running boot step rabbit_exchange_type_headers defined by app rabbit 2025-01-27 02:07:44.293 [info] <0.311.0> Running boot step rabbit_exchange_type_topic defined by app rabbit 2025-01-27 02:07:44.294 [info] <0.311.0> Running boot step rabbit_mirror_queue_mode_all defined by app rabbit 2025-01-27 02:07:44.294 [info] <0.311.0> Running boot step rabbit_mirror_queue_mode_exactly defined by app rabbit 2025-01-27 02:07:44.294 [info] <0.311.0> Running boot step rabbit_mirror_queue_mode_nodes defined by app rabbit 2025-01-27 02:07:44.294 [info] <0.311.0> Running boot step rabbit_priority_queue defined by app rabbit 2025-01-27 02:07:44.294 [info] <0.311.0> Priority queues enabled, real BQ is rabbit_variable_queue 2025-01-27 02:07:44.294 [info] <0.311.0> Running boot step rabbit_queue_location_client_local defined by app rabbit 2025-01-27 02:07:44.294 [info] <0.311.0> Running boot step rabbit_queue_location_min_masters defined by app rabbit 2025-01-27 02:07:44.294 [info] <0.311.0> Running boot step kernel_ready defined by app rabbit 2025-01-27 02:07:44.294 [info] <0.311.0> Running boot step rabbit_sysmon_minder defined by app rabbit 2025-01-27 02:07:44.294 [info] <0.311.0> Running boot step rabbit_epmd_monitor defined by app rabbit 2025-01-27 02:07:44.295 [info] <0.414.0> epmd monitor knows us, inter-node communication (distribution) port: 25673 2025-01-27 02:07:44.295 [info] <0.311.0> Running boot step guid_generator defined by app rabbit 2025-01-27 02:07:44.302 [info] <0.311.0> Running boot step rabbit_node_monitor defined by app rabbit 2025-01-27 02:07:44.303 [info] <0.418.0> Starting rabbit_node_monitor 2025-01-27 02:07:44.311 [info] <0.311.0> Running boot step delegate_sup defined by app rabbit 2025-01-27 02:07:44.312 [info] <0.311.0> Running boot step rabbit_memory_monitor defined by app rabbit 2025-01-27 02:07:44.312 [info] <0.311.0> Running boot step core_initialized defined by app rabbit 2025-01-27 02:07:44.312 [info] <0.311.0> Running boot step upgrade_queues defined by app rabbit 2025-01-27 02:07:44.353 [info] <0.311.0> Running boot step rabbit_connection_tracking defined by app rabbit 2025-01-27 02:07:44.353 [info] <0.311.0> Running boot step rabbit_connection_tracking_handler defined by app rabbit 2025-01-27 02:07:44.353 [info] <0.311.0> Running boot step rabbit_exchange_parameters defined by app rabbit 2025-01-27 02:07:44.353 [info] <0.311.0> Running boot step rabbit_mirror_queue_misc defined by app rabbit 2025-01-27 02:07:44.353 [info] <0.311.0> Running boot step rabbit_policies defined by app rabbit 2025-01-27 02:07:44.354 [info] <0.311.0> Running boot step rabbit_policy defined by app rabbit 2025-01-27 02:07:44.354 [info] <0.311.0> Running boot step rabbit_queue_location_validator defined by app rabbit 2025-01-27 02:07:44.354 [info] <0.311.0> Running boot step rabbit_vhost_limit defined by app rabbit 2025-01-27 02:07:44.354 [info] <0.311.0> Running boot step rabbit_mgmt_reset_handler defined by app rabbitmq_management 2025-01-27 02:07:44.354 [info] <0.311.0> Running boot step rabbit_mgmt_db_handler defined by app rabbitmq_management_agent 2025-01-27 02:07:44.354 [warning] <0.311.0> force_fine_statistics set to true; ignored. Replaced by {rates_mode, none} in the rabbitmq_management application. 2025-01-27 02:07:44.354 [info] <0.311.0> Management plugin: using rates mode 'basic' 2025-01-27 02:07:44.359 [info] <0.311.0> Running boot step recovery defined by app rabbit 2025-01-27 02:07:44.360 [info] <0.455.0> Making sure data directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists 2025-01-27 02:07:44.392 [info] <0.455.0> Starting message stores for vhost '/' 2025-01-27 02:07:44.392 [info] <0.459.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index 2025-01-27 02:07:44.399 [info] <0.455.0> Started message store of type transient for vhost '/' 2025-01-27 02:07:44.399 [info] <0.462.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index 2025-01-27 02:07:44.410 [info] <0.455.0> Started message store of type persistent for vhost '/' 2025-01-27 02:07:44.412 [info] <0.455.0> Mirrored queue 'job_status_consumer.dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.481.0> 2025-01-27 02:07:44.412 [info] <0.455.0> Mirrored queue 'svc_monitor.cn-jenkins-deploy-platform-ansible-os-2660-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.485.0> 2025-01-27 02:07:44.412 [info] <0.455.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2660-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.489.0> 2025-01-27 02:07:44.412 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2660-2.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2660-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.493.0> 2025-01-27 02:07:44.412 [info] <0.455.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2660-2..tftp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.497.0> 2025-01-27 02:07:44.413 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2660-3.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2660-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.501.0> 2025-01-27 02:07:44.413 [info] <0.455.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2660-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.505.0> 2025-01-27 02:07:44.413 [info] <0.455.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2660-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.509.0> 2025-01-27 02:07:44.413 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2660-3.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2660-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.513.0> 2025-01-27 02:07:44.413 [info] <0.455.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2660-1.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.517.0> 2025-01-27 02:07:44.414 [info] <0.455.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2660-1' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.521.0> 2025-01-27 02:07:44.414 [info] <0.455.0> Mirrored queue 'job_request_consumer' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.525.0> 2025-01-27 02:07:44.414 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2660-1.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2660-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.529.0> 2025-01-27 02:07:44.414 [info] <0.455.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2660-2.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.533.0> 2025-01-27 02:07:44.414 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2660-1.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2660-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.537.0> 2025-01-27 02:07:44.415 [info] <0.455.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2660-2' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.541.0> 2025-01-27 02:07:44.415 [info] <0.455.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2660-1' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.545.0> 2025-01-27 02:07:44.415 [info] <0.455.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2660-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.549.0> 2025-01-27 02:07:44.415 [info] <0.455.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2660-3' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.553.0> 2025-01-27 02:07:44.416 [info] <0.455.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2660-3.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.557.0> 2025-01-27 02:07:44.416 [info] <0.455.0> Mirrored queue 'schema_transformer.cn-jenkins-deploy-platform-ansible-os-2660-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.561.0> 2025-01-27 02:07:44.416 [info] <0.455.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2660-3..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.569.0> 2025-01-27 02:07:44.417 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2660-3.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2660-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.573.0> 2025-01-27 02:07:44.417 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2660-2.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2660-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.577.0> 2025-01-27 02:07:44.417 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2660-2.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2660-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.581.0> 2025-01-27 02:07:44.417 [info] <0.455.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2660-2..config_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.585.0> 2025-01-27 02:07:44.418 [info] <0.455.0> Mirrored queue 'device_manager_ztp.ztp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.589.0> 2025-01-27 02:07:44.418 [info] <0.455.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2660-2' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.593.0> 2025-01-27 02:07:44.418 [info] <0.455.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2660-3' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.597.0> 2025-01-27 02:07:44.419 [info] <0.455.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2660-1..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.601.0> 2025-01-27 02:07:44.419 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2660-1.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2660-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.605.0> 2025-01-27 02:07:44.420 [info] <0.455.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2660-2..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.612.0> 2025-01-27 02:07:44.421 [info] <0.311.0> Running boot step load_definitions defined by app rabbitmq_management 2025-01-27 02:07:44.421 [info] <0.311.0> Applying definitions from /etc/rabbitmq/definitions.json 2025-01-27 02:07:44.421 [info] <0.311.0> Asked to import definitions. Acting user: rmq-internal 2025-01-27 02:07:44.421 [info] <0.311.0> Importing users... 2025-01-27 02:07:44.439 [info] <0.311.0> Setting user tags for user 'guest' to [administrator] 2025-01-27 02:07:44.478 [info] <0.311.0> Importing vhosts... 2025-01-27 02:07:44.478 [info] <0.311.0> Importing user permissions... 2025-01-27 02:07:44.478 [info] <0.311.0> Setting permissions for 'guest' in '/' to '.*', '.*', '.*' 2025-01-27 02:07:44.546 [info] <0.311.0> Importing topic permissions... 2025-01-27 02:07:44.547 [info] <0.311.0> Importing parameters... 2025-01-27 02:07:44.547 [info] <0.311.0> Importing global parameters... 2025-01-27 02:07:44.547 [info] <0.311.0> Importing policies... 2025-01-27 02:07:44.583 [info] <0.311.0> Importing queues... 2025-01-27 02:07:44.583 [info] <0.311.0> Importing exchanges... 2025-01-27 02:07:44.583 [info] <0.311.0> Importing bindings... 2025-01-27 02:07:44.583 [info] <0.311.0> Running boot step empty_db_check defined by app rabbit 2025-01-27 02:07:44.583 [info] <0.311.0> Running boot step rabbit_looking_glass defined by app rabbit 2025-01-27 02:07:44.583 [info] <0.311.0> Running boot step rabbit_core_metrics_gc defined by app rabbit 2025-01-27 02:07:44.583 [info] <0.311.0> Running boot step background_gc defined by app rabbit 2025-01-27 02:07:44.583 [info] <0.311.0> Running boot step connection_tracking defined by app rabbit 2025-01-27 02:07:44.587 [info] <0.311.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2660-2' 2025-01-27 02:07:44.588 [info] <0.311.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2660-2' 2025-01-27 02:07:44.588 [info] <0.311.0> Running boot step routing_ready defined by app rabbit 2025-01-27 02:07:44.588 [info] <0.311.0> Running boot step pre_flight defined by app rabbit 2025-01-27 02:07:44.588 [info] <0.311.0> Running boot step notify_cluster defined by app rabbit 2025-01-27 02:07:44.588 [info] <0.311.0> Running boot step networking defined by app rabbit 2025-01-27 02:07:44.588 [info] <0.418.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-1' up 2025-01-27 02:07:44.590 [info] <0.727.0> started TCP listener on [::]:5673 2025-01-27 02:07:44.590 [info] <0.311.0> Running boot step cluster_name defined by app rabbit 2025-01-27 02:07:44.591 [info] <0.311.0> Running boot step direct_client defined by app rabbit 2025-01-27 02:07:44.591 [info] <0.311.0> Running boot step os_signal_handler defined by app rabbit 2025-01-27 02:07:44.591 [info] <0.729.0> Swapping OS signal event handler (erl_signal_server) for our own 2025-01-27 02:07:44.667 [info] <0.786.0> Management plugin: HTTP (non-TLS) listener started on port 15673 2025-01-27 02:07:44.667 [info] <0.918.0> Statistics database started. 2025-01-27 02:07:44.667 [info] <0.917.0> Starting worker pool 'management_worker_pool' with 3 processes in it 2025-01-27 02:07:44.669 [notice] <0.106.0> Changed loghwm of /var/log/contrail/config-database-rabbitmq/rabbitmq.log to 50 2025-01-27 02:07:45.386 [info] <0.8.0> Server startup complete; 3 plugins started. * rabbitmq_management * rabbitmq_web_dispatch * rabbitmq_management_agent 2025-01-27 02:07:46.328 [info] <0.418.0> node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-3' up 2025-01-27 02:07:53.572 [info] <0.418.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-3' up 2025-01-27 02:14:15.411 [info] <0.1631.0> accepting AMQP connection <0.1631.0> (10.0.0.37:42546 -> 10.0.0.32:5673) 2025-01-27 02:14:15.412 [info] <0.1634.0> accepting AMQP connection <0.1634.0> (10.0.0.37:42550 -> 10.0.0.32:5673) 2025-01-27 02:14:15.413 [info] <0.1637.0> accepting AMQP connection <0.1637.0> (10.0.0.37:42584 -> 10.0.0.32:5673) 2025-01-27 02:14:15.413 [info] <0.1640.0> accepting AMQP connection <0.1640.0> (10.0.0.37:42558 -> 10.0.0.32:5673) 2025-01-27 02:14:15.414 [info] <0.1643.0> accepting AMQP connection <0.1643.0> (10.0.0.22:56236 -> 10.0.0.32:5673) 2025-01-27 02:14:15.415 [info] <0.1649.0> accepting AMQP connection <0.1649.0> (10.0.0.37:42572 -> 10.0.0.32:5673) 2025-01-27 02:14:15.415 [info] <0.1652.0> accepting AMQP connection <0.1652.0> (10.0.0.37:42574 -> 10.0.0.32:5673) 2025-01-27 02:14:15.416 [info] <0.1655.0> accepting AMQP connection <0.1655.0> (10.0.0.32:59154 -> 10.0.0.32:5673) 2025-01-27 02:14:15.416 [info] <0.1646.0> accepting AMQP connection <0.1646.0> (10.0.0.32:59148 -> 10.0.0.32:5673) 2025-01-27 02:14:15.418 [info] <0.1634.0> connection <0.1634.0> (10.0.0.37:42550 -> 10.0.0.32:5673): user 'guest' authenticated and granted access to vhost '/' 2025-01-27 02:14:15.419 [info] <0.1658.0> accepting AMQP connection <0.1658.0> (10.0.0.32:59170 -> 10.0.0.32:5673) 2025-01-27 02:14:15.419 [info] <0.1661.0> accepting AMQP connection <0.1661.0> (10.0.0.32:59176 -> 10.0.0.32:5673) 2025-01-27 02:14:15.419 [info] <0.1667.0> accepting AMQP connection <0.1667.0> (10.0.0.32:59184 -> 10.0.0.32:5673) 2025-01-27 02:14:15.419 [info] <0.1670.0> accepting AMQP connection <0.1670.0> (10.0.0.32:59186 -> 10.0.0.32:5673) 2025-01-27 02:14:15.419 [info] <0.1664.0> accepting AMQP connection <0.1664.0> (10.0.0.37:42598 -> 10.0.0.32:5673) 2025-01-27 02:14:15.419 [info] <0.1673.0> accepting AMQP connection <0.1673.0> (10.0.0.32:59198 -> 10.0.0.32:5673) 2025-01-27 02:14:15.419 [info] <0.1679.0> accepting AMQP connection <0.1679.0> (10.0.0.22:56244 -> 10.0.0.32:5673) 2025-01-27 02:14:15.419 [info] <0.1682.0> accepting AMQP connection <0.1682.0> (10.0.0.22:56256 -> 10.0.0.32:5673) 2025-01-27 02:14:15.419 [info] <0.1676.0> accepting AMQP connection <0.1676.0> (10.0.0.32:59202 -> 10.0.0.32:5673) 2025-01-27 02:14:15.419 [info] <0.1631.0> connection <0.1631.0> (10.0.0.37:42546 -> 10.0.0.32:5673): user 'guest' authenticated and granted access to vhost '/' 2025-01-27 02:14:15.421 [info] <0.1697.0> accepting AMQP connection <0.1697.0> (10.0.0.22:56266 -> 10.0.0.32:5673) 2025-01-27 02:14:15.421 [info] <0.1700.0> accepting AMQP connection <0.1700.0> (10.0.0.32:59204 -> 10.0.0.32:5673) 2025-01-27 02:14:15.421 [info] <0.1652.0> connection <0.1652.0> (10.0.0.37:42574 -> 10.0.0.32:5673): user 'guest' authenticated and granted access to vhost '/' 2025-01-27 02:14:15.422 [info] <0.1646.0> connection <0.1646.0> (10.0.0.32:59148 -> 10.0.0.32:5673): user 'guest' authenticated and granted access to vhost '/' 2025-01-27 02:14:15.427 [info] <0.1637.0> connection <0.1637.0> (10.0.0.37:42584 -> 10.0.0.32:5673): user 'guest' authenticated and granted access to vhost '/' 2025-01-27 02:14:15.427 [info] <0.1710.0> accepting AMQP connection <0.1710.0> (10.0.0.32:59210 -> 10.0.0.32:5673) 2025-01-27 02:14:15.427 [info] <0.1676.0> connection <0.1676.0> (10.0.0.32:59202 -> 10.0.0.32:5673): user 'guest' authenticated and granted access to vhost '/' 2025-01-27 02:14:15.427 [info] <0.1721.0> accepting AMQP connection <0.1721.0> (10.0.0.22:56262 -> 10.0.0.32:5673) 2025-01-27 02:14:15.428 [info] <0.1664.0> connection <0.1664.0> (10.0.0.37:42598 -> 10.0.0.32:5673): user 'guest' authenticated and granted access to vhost '/' 2025-01-27 02:14:15.430 [info] <0.1640.0> connection <0.1640.0> (10.0.0.37:42558 -> 10.0.0.32:5673): user 'guest' authenticated and granted access to vhost '/' 2025-01-27 02:14:15.430 [info] <0.1673.0> connection <0.1673.0> (10.0.0.32:59198 -> 10.0.0.32:5673): user 'guest' authenticated and granted access to vhost '/' 2025-01-27 02:14:15.430 [info] <0.1682.0> connection <0.1682.0> (10.0.0.22:56256 -> 10.0.0.32:5673): user 'guest' authenticated and granted access to vhost '/' 2025-01-27 02:14:15.431 [info] <0.1658.0> connection <0.1658.0> (10.0.0.32:59170 -> 10.0.0.32:5673): user 'guest' authenticated and granted access to vhost '/' 2025-01-27 02:14:15.433 [info] <0.1649.0> connection <0.1649.0> (10.0.0.37:42572 -> 10.0.0.32:5673): user 'guest' authenticated and granted access to vhost '/' 2025-01-27 02:14:15.433 [info] <0.1725.0> accepting AMQP connection <0.1725.0> (10.0.0.22:56272 -> 10.0.0.32:5673) 2025-01-27 02:14:15.434 [info] <0.1667.0> connection <0.1667.0> (10.0.0.32:59184 -> 10.0.0.32:5673): user 'guest' authenticated and granted access to vhost '/' 2025-01-27 02:14:15.435 [info] <0.1643.0> connection <0.1643.0> (10.0.0.22:56236 -> 10.0.0.32:5673): user 'guest' authenticated and granted access to vhost '/' 2025-01-27 02:14:15.435 [info] <0.1786.0> accepting AMQP connection <0.1786.0> (10.0.0.22:56270 -> 10.0.0.32:5673) 2025-01-27 02:14:15.435 [info] <0.1655.0> connection <0.1655.0> (10.0.0.32:59154 -> 10.0.0.32:5673): user 'guest' authenticated and granted access to vhost '/' 2025-01-27 02:14:15.437 [info] <0.1661.0> connection <0.1661.0> (10.0.0.32:59176 -> 10.0.0.32:5673): user 'guest' authenticated and granted access to vhost '/' 2025-01-27 02:14:15.437 [info] <0.1700.0> connection <0.1700.0> (10.0.0.32:59204 -> 10.0.0.32:5673): user 'guest' authenticated and granted access to vhost '/' 2025-01-27 02:14:15.441 [info] <0.1679.0> connection <0.1679.0> (10.0.0.22:56244 -> 10.0.0.32:5673): user 'guest' authenticated and granted access to vhost '/' 2025-01-27 02:14:15.441 [info] <0.1721.0> connection <0.1721.0> (10.0.0.22:56262 -> 10.0.0.32:5673): user 'guest' authenticated and granted access to vhost '/' 2025-01-27 02:14:15.441 [info] <0.1725.0> connection <0.1725.0> (10.0.0.22:56272 -> 10.0.0.32:5673): user 'guest' authenticated and granted access to vhost '/' 2025-01-27 02:14:15.443 [info] <0.1697.0> connection <0.1697.0> (10.0.0.22:56266 -> 10.0.0.32:5673): user 'guest' authenticated and granted access to vhost '/' 2025-01-27 02:14:15.443 [info] <0.1670.0> connection <0.1670.0> (10.0.0.32:59186 -> 10.0.0.32:5673): user 'guest' authenticated and granted access to vhost '/' 2025-01-27 02:14:15.444 [info] <0.1710.0> connection <0.1710.0> (10.0.0.32:59210 -> 10.0.0.32:5673): user 'guest' authenticated and granted access to vhost '/' 2025-01-27 02:14:15.445 [info] <0.1786.0> connection <0.1786.0> (10.0.0.22:56270 -> 10.0.0.32:5673): user 'guest' authenticated and granted access to vhost '/' 2025-01-27 02:14:15.507 [info] <0.418.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-1' down 2025-01-27 02:14:15.516 [info] <0.418.0> Keeping contrail@cn-jenkins-deploy-platform-ansible-os-2660-1 listeners: the node is already back 2025-01-27 02:14:16.113 [warning] lager_file_backend dropped 58 messages in the last second that exceeded the limit of 50 messages/sec 2025-01-27 02:14:22.288 [info] <0.2115.0> accepting AMQP connection <0.2115.0> (10.0.0.22:55608 -> 10.0.0.32:5673) 2025-01-27 02:14:22.290 [info] <0.2115.0> connection <0.2115.0> (10.0.0.22:55608 -> 10.0.0.32:5673): user 'guest' authenticated and granted access to vhost '/' 2025-01-27 02:14:22.311 [info] <0.2124.0> Mirrored queue 'schema_transformer.cn-jenkins-deploy-platform-ansible-os-2660-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-3': <5876.1840.0> 2025-01-27 02:14:22.316 [info] <0.2124.0> Mirrored queue 'schema_transformer.cn-jenkins-deploy-platform-ansible-os-2660-3.' in vhost '/': Synchronising: 0 messages to synchronise 2025-01-27 02:14:22.316 [info] <0.2124.0> Mirrored queue 'schema_transformer.cn-jenkins-deploy-platform-ansible-os-2660-3.' in vhost '/': Synchronising: batch size: 5 2025-01-27 02:14:22.325 [info] <0.2131.0> Mirrored queue 'schema_transformer.cn-jenkins-deploy-platform-ansible-os-2660-3.' in vhost '/': Synchronising: all slaves already synced 2025-01-27 02:14:22.339 [info] <0.2135.0> accepting AMQP connection <0.2135.0> (10.0.0.22:55610 -> 10.0.0.32:5673) 2025-01-27 02:14:22.341 [info] <0.2135.0> connection <0.2135.0> (10.0.0.22:55610 -> 10.0.0.32:5673): user 'guest' authenticated and granted access to vhost '/' 2025-01-27 02:14:22.357 [info] <0.2115.0> closing AMQP connection <0.2115.0> (10.0.0.22:55608 -> 10.0.0.32:5673, vhost: '/', user: 'guest') 2025-01-27 02:14:22.358 [info] <0.2135.0> closing AMQP connection <0.2135.0> (10.0.0.22:55610 -> 10.0.0.32:5673, vhost: '/', user: 'guest') 2025-01-27 02:14:22.618 [info] <0.2153.0> accepting AMQP connection <0.2153.0> (10.0.0.22:55618 -> 10.0.0.32:5673) 2025-01-27 02:14:22.620 [info] <0.2153.0> connection <0.2153.0> (10.0.0.22:55618 -> 10.0.0.32:5673): user 'guest' authenticated and granted access to vhost '/' 2025-01-27 02:14:22.646 [info] <0.2162.0> Mirrored queue 'svc_monitor.cn-jenkins-deploy-platform-ansible-os-2660-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-3': <5876.1855.0> 2025-01-27 02:14:22.655 [info] <0.2162.0> Mirrored queue 'svc_monitor.cn-jenkins-deploy-platform-ansible-os-2660-3.' in vhost '/': Synchronising: 0 messages to synchronise 2025-01-27 02:14:22.656 [info] <0.2162.0> Mirrored queue 'svc_monitor.cn-jenkins-deploy-platform-ansible-os-2660-3.' in vhost '/': Synchronising: batch size: 5 2025-01-27 02:14:22.658 [info] <0.2170.0> Mirrored queue 'svc_monitor.cn-jenkins-deploy-platform-ansible-os-2660-3.' in vhost '/': Synchronising: all slaves already synced 2025-01-27 02:14:22.661 [info] <0.2173.0> accepting AMQP connection <0.2173.0> (10.0.0.22:55622 -> 10.0.0.32:5673) 2025-01-27 02:14:22.663 [info] <0.2173.0> connection <0.2173.0> (10.0.0.22:55622 -> 10.0.0.32:5673): user 'guest' authenticated and granted access to vhost '/' 2025-01-27 02:14:22.691 [info] <0.2153.0> closing AMQP connection <0.2153.0> (10.0.0.22:55618 -> 10.0.0.32:5673, vhost: '/', user: 'guest') 2025-01-27 02:14:22.691 [info] <0.2173.0> closing AMQP connection <0.2173.0> (10.0.0.22:55622 -> 10.0.0.32:5673, vhost: '/', user: 'guest') 2025-01-27 02:14:45.604 [info] <0.418.0> node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-1' up 2025-01-27 02:14:54.366 [info] <0.1898.0> Mirrored queue 'job_status_consumer.dummy' in vhost '/': Synchronising: 0 messages to synchronise 2025-01-27 02:14:54.366 [info] <0.497.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2660-2..tftp_queue' in vhost '/': Synchronising: 0 messages to synchronise 2025-01-27 02:14:54.366 [info] <0.1898.0> Mirrored queue 'job_status_consumer.dummy' in vhost '/': Synchronising: batch size: 5 2025-01-27 02:14:54.366 [info] <0.497.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2660-2..tftp_queue' in vhost '/': Synchronising: batch size: 5 2025-01-27 02:14:54.396 [info] <0.505.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2660-2.' in vhost '/': Synchronising: 0 messages to synchronise 2025-01-27 02:14:54.396 [info] <0.505.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2660-2.' in vhost '/': Synchronising: batch size: 5 2025-01-27 02:14:54.421 [info] <0.525.0> Mirrored queue 'job_request_consumer' in vhost '/': Synchronising: 0 messages to synchronise 2025-01-27 02:14:54.421 [info] <0.525.0> Mirrored queue 'job_request_consumer' in vhost '/': Synchronising: batch size: 5 2025-01-27 02:14:54.422 [info] <0.2337.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2660-2..tftp_queue' in vhost '/': Synchronising: all slaves already synced 2025-01-27 02:14:54.422 [info] <0.2338.0> Mirrored queue 'job_status_consumer.dummy' in vhost '/': Synchronising: all slaves already synced 2025-01-27 02:14:54.429 [info] <0.585.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2660-2..config_queue' in vhost '/': Synchronising: 0 messages to synchronise 2025-01-27 02:14:54.429 [info] <0.585.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2660-2..config_queue' in vhost '/': Synchronising: batch size: 5 2025-01-27 02:14:54.434 [info] <0.2343.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2660-2.' in vhost '/': Synchronising: all slaves already synced 2025-01-27 02:14:54.435 [info] <0.589.0> Mirrored queue 'device_manager_ztp.ztp_queue' in vhost '/': Synchronising: 0 messages to synchronise 2025-01-27 02:14:54.435 [info] <0.589.0> Mirrored queue 'device_manager_ztp.ztp_queue' in vhost '/': Synchronising: batch size: 5 2025-01-27 02:14:54.454 [info] <0.2351.0> Mirrored queue 'job_request_consumer' in vhost '/': Synchronising: all slaves already synced 2025-01-27 02:14:54.464 [info] <0.1910.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2660-1..dummy' in vhost '/': Synchronising: 0 messages to synchronise 2025-01-27 02:14:54.464 [info] <0.1905.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2660-2..dummy' in vhost '/': Synchronising: 0 messages to synchronise 2025-01-27 02:14:54.464 [info] <0.1910.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2660-1..dummy' in vhost '/': Synchronising: batch size: 5 2025-01-27 02:14:54.464 [info] <0.1905.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2660-2..dummy' in vhost '/': Synchronising: batch size: 5 2025-01-27 02:14:54.466 [info] <0.2352.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2660-2..config_queue' in vhost '/': Synchronising: all slaves already synced 2025-01-27 02:14:54.482 [info] <0.2353.0> Mirrored queue 'device_manager_ztp.ztp_queue' in vhost '/': Synchronising: all slaves already synced 2025-01-27 02:14:54.482 [info] <0.1914.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2660-3..dummy' in vhost '/': Synchronising: 0 messages to synchronise 2025-01-27 02:14:54.482 [info] <0.1914.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2660-3..dummy' in vhost '/': Synchronising: batch size: 5 2025-01-27 02:14:54.489 [info] <0.2356.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2660-3..dummy' in vhost '/': Synchronising: all slaves already synced 2025-01-27 02:14:54.502 [info] <0.2354.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2660-1..dummy' in vhost '/': Synchronising: all slaves already synced 2025-01-27 02:14:54.502 [info] <0.2355.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2660-2..dummy' in vhost '/': Synchronising: all slaves already synced 2025-01-27 02:14:54.566 [info] <0.418.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-1' up 2025-01-27 02:15:46.636 [info] <0.2405.0> RabbitMQ is asked to stop... 2025-01-27 02:15:46.730 [info] <0.2405.0> Stopping RabbitMQ applications and their dependencies in the following order: rabbitmq_management amqp_client rabbitmq_web_dispatch cowboy cowlib rabbitmq_management_agent rabbit mnesia rabbit_common sysmon_handler os_mon 2025-01-27 02:15:46.730 [info] <0.2405.0> Stopping application 'rabbitmq_management' 2025-01-27 02:15:46.734 [warning] <0.782.0> RabbitMQ HTTP listener registry could not find context rabbitmq_management_tls 2025-01-27 02:15:46.736 [info] <0.43.0> Application rabbitmq_management exited with reason: stopped 2025-01-27 02:15:46.736 [info] <0.2405.0> Stopping application 'amqp_client' 2025-01-27 02:15:46.737 [info] <0.43.0> Application amqp_client exited with reason: stopped 2025-01-27 02:15:46.737 [info] <0.2405.0> Stopping application 'rabbitmq_web_dispatch' 2025-01-27 02:15:46.739 [info] <0.2405.0> Stopping application 'cowboy' 2025-01-27 02:15:46.739 [info] <0.43.0> Application rabbitmq_web_dispatch exited with reason: stopped 2025-01-27 02:15:46.740 [info] <0.43.0> Application cowboy exited with reason: stopped 2025-01-27 02:15:46.740 [info] <0.2405.0> Stopping application 'cowlib' 2025-01-27 02:15:46.740 [info] <0.2405.0> Stopping application 'rabbitmq_management_agent' 2025-01-27 02:15:46.740 [info] <0.43.0> Application cowlib exited with reason: stopped 2025-01-27 02:15:46.741 [info] <0.43.0> Application rabbitmq_management_agent exited with reason: stopped 2025-01-27 02:15:46.741 [info] <0.2405.0> Stopping application 'rabbit' 2025-01-27 02:15:46.741 [info] <0.311.0> Will unregister with peer discovery backend rabbit_peer_discovery_classic_config 2025-01-27 02:15:46.741 [info] <0.727.0> stopped TCP listener on [::]:5673 2025-01-27 02:15:46.749 [error] <0.1652.0> Error on AMQP connection <0.1652.0> (10.0.0.37:42574 -> 10.0.0.32:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-01-27 02:15:46.749 [error] <0.1637.0> Error on AMQP connection <0.1637.0> (10.0.0.37:42584 -> 10.0.0.32:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-01-27 02:15:46.749 [error] <0.1863.0> Error on AMQP connection <0.1863.0> (10.0.0.32:59212 -> 10.0.0.32:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-01-27 02:15:46.749 [error] <0.1664.0> Error on AMQP connection <0.1664.0> (10.0.0.37:42598 -> 10.0.0.32:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-01-27 02:15:46.749 [error] <0.1655.0> Error on AMQP connection <0.1655.0> (10.0.0.32:59154 -> 10.0.0.32:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-01-27 02:15:46.749 [error] <0.1649.0> Error on AMQP connection <0.1649.0> (10.0.0.37:42572 -> 10.0.0.32:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-01-27 02:15:46.749 [error] <0.1679.0> Error on AMQP connection <0.1679.0> (10.0.0.22:56244 -> 10.0.0.32:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-01-27 02:15:46.749 [error] <0.1700.0> Error on AMQP connection <0.1700.0> (10.0.0.32:59204 -> 10.0.0.32:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-01-27 02:15:46.749 [error] <0.1646.0> Error on AMQP connection <0.1646.0> (10.0.0.32:59148 -> 10.0.0.32:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-01-27 02:15:46.749 [error] <0.1661.0> Error on AMQP connection <0.1661.0> (10.0.0.32:59176 -> 10.0.0.32:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-01-27 02:15:46.749 [error] <0.1640.0> Error on AMQP connection <0.1640.0> (10.0.0.37:42558 -> 10.0.0.32:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-01-27 02:15:46.749 [error] <0.1786.0> Error on AMQP connection <0.1786.0> (10.0.0.22:56270 -> 10.0.0.32:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-01-27 02:15:46.749 [error] <0.1631.0> Error on AMQP connection <0.1631.0> (10.0.0.37:42546 -> 10.0.0.32:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-01-27 02:15:46.749 [error] <0.1866.0> Error on AMQP connection <0.1866.0> (10.0.0.32:59226 -> 10.0.0.32:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-01-27 02:15:46.749 [error] <0.1643.0> Error on AMQP connection <0.1643.0> (10.0.0.22:56236 -> 10.0.0.32:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-01-27 02:15:46.749 [error] <0.1658.0> Error on AMQP connection <0.1658.0> (10.0.0.32:59170 -> 10.0.0.32:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-01-27 02:15:46.749 [error] <0.1710.0> Error on AMQP connection <0.1710.0> (10.0.0.32:59210 -> 10.0.0.32:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-01-27 02:15:46.749 [error] <0.1869.0> Error on AMQP connection <0.1869.0> (10.0.0.22:56282 -> 10.0.0.32:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-01-27 02:15:46.749 [error] <0.1676.0> Error on AMQP connection <0.1676.0> (10.0.0.32:59202 -> 10.0.0.32:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-01-27 02:15:46.749 [error] <0.1634.0> Error on AMQP connection <0.1634.0> (10.0.0.37:42550 -> 10.0.0.32:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-01-27 02:15:46.749 [error] <0.1670.0> Error on AMQP connection <0.1670.0> (10.0.0.32:59186 -> 10.0.0.32:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-01-27 02:15:46.749 [error] <0.1725.0> Error on AMQP connection <0.1725.0> (10.0.0.22:56272 -> 10.0.0.32:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-01-27 02:15:46.749 [error] <0.1860.0> Error on AMQP connection <0.1860.0> (10.0.0.37:42614 -> 10.0.0.32:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-01-27 02:15:46.749 [error] <0.1673.0> Error on AMQP connection <0.1673.0> (10.0.0.32:59198 -> 10.0.0.32:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-01-27 02:15:46.749 [error] <0.1697.0> Error on AMQP connection <0.1697.0> (10.0.0.22:56266 -> 10.0.0.32:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-01-27 02:15:46.749 [error] <0.1682.0> Error on AMQP connection <0.1682.0> (10.0.0.22:56256 -> 10.0.0.32:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-01-27 02:15:46.749 [error] <0.1667.0> Error on AMQP connection <0.1667.0> (10.0.0.32:59184 -> 10.0.0.32:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-01-27 02:15:46.749 [error] <0.1721.0> Error on AMQP connection <0.1721.0> (10.0.0.22:56262 -> 10.0.0.32:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-01-27 02:15:46.765 [error] <0.1829.0> Supervisor {<0.1829.0>,rabbit_channel_sup_sup} had child channel_sup started with rabbit_channel_sup:start_link() at undefined exit with reason shutdown in context shutdown_error 2025-01-27 02:15:46.780 [error] <0.1811.0> Supervisor {<0.1811.0>,rabbit_channel_sup_sup} had child channel_sup started with rabbit_channel_sup:start_link() at undefined exit with reason shutdown in context shutdown_error 2025-01-27 02:15:46.856 [info] <0.443.0> Closing all connections in vhost '/' on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2' because the vhost is stopping 2025-01-27 02:15:46.879 [info] <0.462.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' 2025-01-27 02:16:21.002 [info] <0.8.0> Log file opened with Lager 2025-01-27 02:16:24.451 [info] <0.8.0> Feature flags: list of feature flags found: 2025-01-27 02:16:24.451 [info] <0.8.0> Feature flags: feature flag states written to disk: yes 2025-01-27 02:16:24.482 [info] <0.249.0> Starting RabbitMQ 3.7.28 on Erlang 21.3.8.21 Copyright (c) 2007-2020 Pivotal Software, Inc. Licensed under the MPL. See https://www.rabbitmq.com/ 2025-01-27 02:16:24.483 [info] <0.249.0> node : contrail@cn-jenkins-deploy-platform-ansible-os-2660-2 home dir : /var/lib/rabbitmq config file(s) : /etc/rabbitmq/rabbitmq.config cookie hash : esZsDxSN6VGbi9JkMSxNZA== log(s) : /var/log/contrail/config-database-rabbitmq/rabbitmq.log : /var/log/rabbitmq/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2_upgrade.log database dir : /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2 2025-01-27 02:16:24.501 [info] <0.249.0> Running boot step pre_boot defined by app rabbit 2025-01-27 02:16:24.501 [info] <0.249.0> Running boot step rabbit_core_metrics defined by app rabbit 2025-01-27 02:16:24.501 [info] <0.249.0> Running boot step rabbit_alarm defined by app rabbit 2025-01-27 02:16:24.512 [info] <0.255.0> Memory high watermark set to 12637 MiB (13251084288 bytes) of 15796 MiB (16563855360 bytes) total 2025-01-27 02:16:24.521 [info] <0.257.0> Enabling free disk space monitoring 2025-01-27 02:16:24.521 [info] <0.257.0> Disk free limit set to 50MB 2025-01-27 02:16:24.530 [info] <0.249.0> Running boot step code_server_cache defined by app rabbit 2025-01-27 02:16:24.531 [info] <0.249.0> Running boot step file_handle_cache defined by app rabbit 2025-01-27 02:16:24.531 [info] <0.260.0> Limiting to approx 65436 file handles (58890 sockets) 2025-01-27 02:16:24.531 [info] <0.261.0> FHC read buffering: OFF 2025-01-27 02:16:24.531 [info] <0.261.0> FHC write buffering: ON 2025-01-27 02:16:24.531 [info] <0.249.0> Running boot step worker_pool defined by app rabbit 2025-01-27 02:16:24.531 [info] <0.250.0> Will use 4 processes for default worker pool 2025-01-27 02:16:24.531 [info] <0.250.0> Starting worker pool 'worker_pool' with 4 processes in it 2025-01-27 02:16:24.531 [info] <0.249.0> Running boot step database defined by app rabbit 2025-01-27 02:16:24.532 [info] <0.249.0> Node database directory at /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2 is empty. Assuming we need to join an existing cluster or initialise from scratch... 2025-01-27 02:16:24.532 [info] <0.249.0> Configured peer discovery backend: rabbit_peer_discovery_classic_config 2025-01-27 02:16:24.532 [info] <0.249.0> Will try to lock with peer discovery backend rabbit_peer_discovery_classic_config 2025-01-27 02:16:24.532 [info] <0.249.0> Peer discovery backend does not support locking, falling back to randomized delay 2025-01-27 02:16:24.532 [info] <0.249.0> Peer discovery backend rabbit_peer_discovery_classic_config supports registration. 2025-01-27 02:16:24.532 [info] <0.249.0> Will wait for 57036 milliseconds before proceeding with registration... 2025-01-27 02:17:07.516 [info] <0.303.0> RabbitMQ hasn't finished starting yet. Waiting for startup to finish before stopping... 2025-01-27 02:17:21.568 [info] <0.249.0> All discovered existing cluster peers: contrail@cn-jenkins-deploy-platform-ansible-os-2660-1, contrail@cn-jenkins-deploy-platform-ansible-os-2660-2, contrail@cn-jenkins-deploy-platform-ansible-os-2660-3 2025-01-27 02:17:21.568 [info] <0.249.0> Peer nodes we can cluster with: contrail@cn-jenkins-deploy-platform-ansible-os-2660-1, contrail@cn-jenkins-deploy-platform-ansible-os-2660-3 2025-01-27 02:17:21.570 [info] <0.249.0> Node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-1' selected for auto-clustering 2025-01-27 02:17:21.595 [info] <0.249.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-01-27 02:17:21.628 [info] <0.249.0> Successfully synced tables from a peer 2025-01-27 02:17:21.629 [info] <0.249.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-01-27 02:17:21.629 [info] <0.249.0> Successfully synced tables from a peer 2025-01-27 02:17:21.661 [info] <0.249.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-01-27 02:17:21.662 [info] <0.249.0> Successfully synced tables from a peer 2025-01-27 02:17:21.663 [info] <0.249.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2660-2' 2025-01-27 02:17:21.663 [info] <0.249.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2660-2' 2025-01-27 02:17:21.664 [info] <0.249.0> Will register with peer discovery backend rabbit_peer_discovery_classic_config 2025-01-27 02:17:21.665 [info] <0.249.0> Running boot step database_sync defined by app rabbit 2025-01-27 02:17:21.665 [info] <0.249.0> Running boot step feature_flags defined by app rabbit 2025-01-27 02:17:21.667 [info] <0.249.0> Running boot step codec_correctness_check defined by app rabbit 2025-01-27 02:17:21.667 [info] <0.249.0> Running boot step external_infrastructure defined by app rabbit 2025-01-27 02:17:21.667 [info] <0.249.0> Running boot step rabbit_registry defined by app rabbit 2025-01-27 02:17:21.668 [info] <0.249.0> Running boot step rabbit_auth_mechanism_cr_demo defined by app rabbit 2025-01-27 02:17:21.668 [info] <0.249.0> Running boot step rabbit_queue_location_random defined by app rabbit 2025-01-27 02:17:21.669 [info] <0.249.0> Running boot step rabbit_event defined by app rabbit 2025-01-27 02:17:21.670 [info] <0.249.0> Running boot step rabbit_auth_mechanism_amqplain defined by app rabbit 2025-01-27 02:17:21.670 [info] <0.249.0> Running boot step rabbit_auth_mechanism_plain defined by app rabbit 2025-01-27 02:17:21.671 [info] <0.249.0> Running boot step rabbit_exchange_type_direct defined by app rabbit 2025-01-27 02:17:21.671 [info] <0.249.0> Running boot step rabbit_exchange_type_fanout defined by app rabbit 2025-01-27 02:17:21.672 [info] <0.249.0> Running boot step rabbit_exchange_type_headers defined by app rabbit 2025-01-27 02:17:21.672 [info] <0.249.0> Running boot step rabbit_exchange_type_topic defined by app rabbit 2025-01-27 02:17:21.672 [info] <0.249.0> Running boot step rabbit_mirror_queue_mode_all defined by app rabbit 2025-01-27 02:17:21.673 [info] <0.249.0> Running boot step rabbit_mirror_queue_mode_exactly defined by app rabbit 2025-01-27 02:17:21.673 [info] <0.249.0> Running boot step rabbit_mirror_queue_mode_nodes defined by app rabbit 2025-01-27 02:17:21.673 [info] <0.249.0> Running boot step rabbit_priority_queue defined by app rabbit 2025-01-27 02:17:21.674 [info] <0.249.0> Priority queues enabled, real BQ is rabbit_variable_queue 2025-01-27 02:17:21.674 [info] <0.249.0> Running boot step rabbit_queue_location_client_local defined by app rabbit 2025-01-27 02:17:21.674 [info] <0.249.0> Running boot step rabbit_queue_location_min_masters defined by app rabbit 2025-01-27 02:17:21.674 [info] <0.249.0> Running boot step kernel_ready defined by app rabbit 2025-01-27 02:17:21.674 [info] <0.249.0> Running boot step rabbit_sysmon_minder defined by app rabbit 2025-01-27 02:17:21.674 [info] <0.249.0> Running boot step rabbit_epmd_monitor defined by app rabbit 2025-01-27 02:17:21.675 [info] <0.425.0> epmd monitor knows us, inter-node communication (distribution) port: 25673 2025-01-27 02:17:21.676 [info] <0.249.0> Running boot step guid_generator defined by app rabbit 2025-01-27 02:17:21.679 [info] <0.249.0> Running boot step rabbit_node_monitor defined by app rabbit 2025-01-27 02:17:21.679 [info] <0.429.0> Starting rabbit_node_monitor 2025-01-27 02:17:21.680 [info] <0.249.0> Running boot step delegate_sup defined by app rabbit 2025-01-27 02:17:21.680 [info] <0.249.0> Running boot step rabbit_memory_monitor defined by app rabbit 2025-01-27 02:17:21.680 [info] <0.249.0> Running boot step core_initialized defined by app rabbit 2025-01-27 02:17:21.680 [info] <0.249.0> Running boot step upgrade_queues defined by app rabbit 2025-01-27 02:17:21.701 [info] <0.249.0> message_store upgrades: 1 to apply 2025-01-27 02:17:21.701 [info] <0.249.0> message_store upgrades: Applying rabbit_variable_queue:move_messages_to_vhost_store 2025-01-27 02:17:21.701 [info] <0.249.0> message_store upgrades: No durable queues found. Skipping message store migration 2025-01-27 02:17:21.702 [info] <0.249.0> message_store upgrades: Removing the old message store data 2025-01-27 02:17:21.702 [info] <0.249.0> message_store upgrades: All upgrades applied successfully 2025-01-27 02:17:21.726 [info] <0.249.0> Running boot step rabbit_connection_tracking defined by app rabbit 2025-01-27 02:17:21.726 [info] <0.249.0> Running boot step rabbit_connection_tracking_handler defined by app rabbit 2025-01-27 02:17:21.726 [info] <0.249.0> Running boot step rabbit_exchange_parameters defined by app rabbit 2025-01-27 02:17:21.727 [info] <0.249.0> Running boot step rabbit_mirror_queue_misc defined by app rabbit 2025-01-27 02:17:21.727 [info] <0.249.0> Running boot step rabbit_policies defined by app rabbit 2025-01-27 02:17:21.727 [info] <0.249.0> Running boot step rabbit_policy defined by app rabbit 2025-01-27 02:17:21.727 [info] <0.249.0> Running boot step rabbit_queue_location_validator defined by app rabbit 2025-01-27 02:17:21.727 [info] <0.249.0> Running boot step rabbit_vhost_limit defined by app rabbit 2025-01-27 02:17:21.728 [info] <0.249.0> Running boot step rabbit_mgmt_reset_handler defined by app rabbitmq_management 2025-01-27 02:17:21.728 [info] <0.249.0> Running boot step rabbit_mgmt_db_handler defined by app rabbitmq_management_agent 2025-01-27 02:17:21.728 [warning] <0.249.0> force_fine_statistics set to true; ignored. Replaced by {rates_mode, none} in the rabbitmq_management application. 2025-01-27 02:17:21.728 [info] <0.249.0> Management plugin: using rates mode 'basic' 2025-01-27 02:17:21.730 [info] <0.249.0> Running boot step recovery defined by app rabbit 2025-01-27 02:17:21.730 [info] <0.466.0> Making sure data directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists 2025-01-27 02:17:21.734 [info] <0.466.0> Starting message stores for vhost '/' 2025-01-27 02:17:21.735 [info] <0.470.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index 2025-01-27 02:17:21.736 [info] <0.466.0> Started message store of type transient for vhost '/' 2025-01-27 02:17:21.736 [info] <0.473.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index 2025-01-27 02:17:21.736 [warning] <0.473.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": rebuilding indices from scratch 2025-01-27 02:17:21.737 [info] <0.466.0> Started message store of type persistent for vhost '/' 2025-01-27 02:17:21.738 [info] <0.466.0> Mirrored queue 'job_status_consumer.dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.492.0> 2025-01-27 02:17:21.738 [info] <0.466.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2660-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.496.0> 2025-01-27 02:17:21.739 [info] <0.466.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2660-2.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2660-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.500.0> 2025-01-27 02:17:21.739 [info] <0.466.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2660-2..tftp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.504.0> 2025-01-27 02:17:21.739 [info] <0.466.0> Mirrored queue 'device_manager.cn-jenkins-deploy-platform-ansible-os-2660-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.508.0> 2025-01-27 02:17:21.740 [info] <0.466.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2660-3.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2660-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.512.0> 2025-01-27 02:17:21.740 [info] <0.466.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2660-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.516.0> 2025-01-27 02:17:21.740 [info] <0.466.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2660-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.520.0> 2025-01-27 02:17:21.740 [info] <0.466.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2660-3.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2660-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.524.0> 2025-01-27 02:17:21.740 [info] <0.466.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2660-1.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.528.0> 2025-01-27 02:17:21.740 [info] <0.466.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2660-1' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.532.0> 2025-01-27 02:17:21.741 [info] <0.466.0> Mirrored queue 'job_request_consumer' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.536.0> 2025-01-27 02:17:21.741 [info] <0.466.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2660-1.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2660-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.540.0> 2025-01-27 02:17:21.741 [info] <0.466.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2660-2.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.544.0> 2025-01-27 02:17:21.741 [info] <0.466.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2660-1.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2660-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.548.0> 2025-01-27 02:17:21.741 [info] <0.466.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2660-2' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.552.0> 2025-01-27 02:17:21.742 [info] <0.466.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2660-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.556.0> 2025-01-27 02:17:21.742 [info] <0.466.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2660-1' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.560.0> 2025-01-27 02:17:21.742 [info] <0.466.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2660-3' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.564.0> 2025-01-27 02:17:21.742 [info] <0.466.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2660-3.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.568.0> 2025-01-27 02:17:21.743 [info] <0.466.0> Mirrored queue 'schema_transformer.cn-jenkins-deploy-platform-ansible-os-2660-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.572.0> 2025-01-27 02:17:21.743 [info] <0.466.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2660-3..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.576.0> 2025-01-27 02:17:21.743 [info] <0.466.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2660-3.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2660-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.580.0> 2025-01-27 02:17:21.743 [info] <0.466.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2660-2.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2660-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.584.0> 2025-01-27 02:17:21.744 [info] <0.466.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2660-2.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2660-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.588.0> 2025-01-27 02:17:21.744 [info] <0.466.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2660-2..config_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.592.0> 2025-01-27 02:17:21.744 [info] <0.466.0> Mirrored queue 'svc_monitor.cn-jenkins-deploy-platform-ansible-os-2660-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.596.0> 2025-01-27 02:17:21.744 [info] <0.466.0> Mirrored queue 'device_manager_ztp.ztp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.600.0> 2025-01-27 02:17:21.745 [info] <0.466.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2660-2' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.604.0> 2025-01-27 02:17:21.745 [info] <0.466.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2660-3' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.608.0> 2025-01-27 02:17:21.745 [info] <0.466.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2660-1..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.612.0> 2025-01-27 02:17:21.745 [info] <0.466.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2660-1.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2660-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.616.0> 2025-01-27 02:17:21.745 [info] <0.466.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2660-2..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.620.0> 2025-01-27 02:17:21.746 [info] <0.249.0> Running boot step load_definitions defined by app rabbitmq_management 2025-01-27 02:17:21.746 [info] <0.249.0> Applying definitions from /etc/rabbitmq/definitions.json 2025-01-27 02:17:21.746 [info] <0.249.0> Asked to import definitions. Acting user: rmq-internal 2025-01-27 02:17:21.746 [info] <0.249.0> Importing users... 2025-01-27 02:17:21.754 [info] <0.249.0> Setting user tags for user 'guest' to [administrator] 2025-01-27 02:17:21.763 [info] <0.249.0> Importing vhosts... 2025-01-27 02:17:21.763 [info] <0.249.0> Importing user permissions... 2025-01-27 02:17:21.763 [info] <0.249.0> Setting permissions for 'guest' in '/' to '.*', '.*', '.*' 2025-01-27 02:17:21.798 [info] <0.249.0> Importing topic permissions... 2025-01-27 02:17:21.798 [info] <0.249.0> Importing parameters... 2025-01-27 02:17:21.799 [info] <0.249.0> Importing global parameters... 2025-01-27 02:17:21.799 [info] <0.249.0> Importing policies... 2025-01-27 02:17:21.813 [info] <0.249.0> Importing queues... 2025-01-27 02:17:21.813 [info] <0.249.0> Importing exchanges... 2025-01-27 02:17:21.813 [info] <0.249.0> Importing bindings... 2025-01-27 02:17:21.813 [info] <0.249.0> Running boot step empty_db_check defined by app rabbit 2025-01-27 02:17:21.813 [info] <0.249.0> Running boot step rabbit_looking_glass defined by app rabbit 2025-01-27 02:17:21.813 [info] <0.249.0> Running boot step rabbit_core_metrics_gc defined by app rabbit 2025-01-27 02:17:21.814 [info] <0.249.0> Running boot step background_gc defined by app rabbit 2025-01-27 02:17:21.814 [info] <0.249.0> Running boot step connection_tracking defined by app rabbit 2025-01-27 02:17:21.815 [info] <0.249.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2660-2' 2025-01-27 02:17:21.815 [info] <0.249.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2660-2' 2025-01-27 02:17:21.816 [info] <0.249.0> Running boot step routing_ready defined by app rabbit 2025-01-27 02:17:21.816 [info] <0.249.0> Running boot step pre_flight defined by app rabbit 2025-01-27 02:17:21.816 [info] <0.249.0> Running boot step notify_cluster defined by app rabbit 2025-01-27 02:17:21.816 [info] <0.249.0> Running boot step networking defined by app rabbit 2025-01-27 02:17:21.816 [info] <0.429.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-1' up 2025-01-27 02:17:21.818 [info] <0.745.0> started TCP listener on [::]:5673 2025-01-27 02:17:21.818 [info] <0.249.0> Running boot step cluster_name defined by app rabbit 2025-01-27 02:17:21.819 [info] <0.249.0> Running boot step direct_client defined by app rabbit 2025-01-27 02:17:21.819 [info] <0.249.0> Running boot step os_signal_handler defined by app rabbit 2025-01-27 02:17:21.819 [info] <0.747.0> Swapping OS signal event handler (erl_signal_server) for our own 2025-01-27 02:17:21.850 [info] <0.802.0> Management plugin: HTTP (non-TLS) listener started on port 15673 2025-01-27 02:17:21.850 [info] <0.937.0> Statistics database started. 2025-01-27 02:17:21.851 [info] <0.936.0> Starting worker pool 'management_worker_pool' with 3 processes in it 2025-01-27 02:17:21.851 [notice] <0.114.0> Changed loghwm of /var/log/contrail/config-database-rabbitmq/rabbitmq.log to 50 2025-01-27 02:17:21.944 [info] <0.8.0> Server startup complete; 3 plugins started. * rabbitmq_management * rabbitmq_web_dispatch * rabbitmq_management_agent 2025-01-27 02:17:21.959 [info] <0.303.0> RabbitMQ is asked to stop... 2025-01-27 02:17:22.010 [info] <0.303.0> Stopping RabbitMQ applications and their dependencies in the following order: rabbitmq_management amqp_client rabbitmq_web_dispatch cowboy cowlib rabbitmq_management_agent rabbit mnesia rabbit_common sysmon_handler os_mon 2025-01-27 02:17:22.010 [info] <0.303.0> Stopping application 'rabbitmq_management' 2025-01-27 02:17:22.018 [warning] <0.797.0> RabbitMQ HTTP listener registry could not find context rabbitmq_management_tls 2025-01-27 02:17:22.020 [info] <0.303.0> Stopping application 'amqp_client' 2025-01-27 02:17:22.020 [info] <0.43.0> Application rabbitmq_management exited with reason: stopped 2025-01-27 02:17:22.021 [info] <0.43.0> Application amqp_client exited with reason: stopped 2025-01-27 02:17:22.021 [info] <0.303.0> Stopping application 'rabbitmq_web_dispatch' 2025-01-27 02:17:22.022 [info] <0.43.0> Application rabbitmq_web_dispatch exited with reason: stopped 2025-01-27 02:17:22.022 [info] <0.303.0> Stopping application 'cowboy' 2025-01-27 02:17:22.023 [info] <0.303.0> Stopping application 'cowlib' 2025-01-27 02:17:22.023 [info] <0.43.0> Application cowboy exited with reason: stopped 2025-01-27 02:17:22.023 [info] <0.43.0> Application cowlib exited with reason: stopped 2025-01-27 02:17:22.023 [info] <0.303.0> Stopping application 'rabbitmq_management_agent' 2025-01-27 02:17:22.025 [info] <0.43.0> Application rabbitmq_management_agent exited with reason: stopped 2025-01-27 02:17:22.025 [info] <0.303.0> Stopping application 'rabbit' 2025-01-27 02:17:22.025 [info] <0.249.0> Will unregister with peer discovery backend rabbit_peer_discovery_classic_config 2025-01-27 02:17:22.026 [info] <0.745.0> stopped TCP listener on [::]:5673 2025-01-27 02:17:22.026 [info] <0.454.0> Closing all connections in vhost '/' on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2' because the vhost is stopping 2025-01-27 02:17:22.027 [info] <0.473.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' 2025-01-27 02:17:22.031 [info] <0.473.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' is stopped 2025-01-27 02:17:22.031 [info] <0.470.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' 2025-01-27 02:17:22.041 [info] <0.470.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' is stopped 2025-01-27 02:17:22.048 [info] <0.43.0> Application rabbit exited with reason: stopped 2025-01-27 02:17:22.048 [info] <0.303.0> Stopping application 'mnesia' 2025-01-27 02:17:22.051 [info] <0.303.0> Stopping application 'rabbit_common' 2025-01-27 02:17:22.051 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-01-27 02:17:22.051 [info] <0.43.0> Application rabbit_common exited with reason: stopped 2025-01-27 02:17:22.051 [info] <0.303.0> Stopping application 'sysmon_handler' 2025-01-27 02:17:22.052 [info] <0.43.0> Application sysmon_handler exited with reason: stopped 2025-01-27 02:17:22.052 [info] <0.303.0> Stopping application 'os_mon' 2025-01-27 02:17:22.053 [info] <0.43.0> Application os_mon exited with reason: stopped 2025-01-27 02:17:22.053 [info] <0.303.0> Successfully stopped RabbitMQ and its dependencies 2025-01-27 02:17:31.824 [info] <0.960.0> Resetting Rabbit forcefully 2025-01-27 02:17:32.692 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-01-27 02:17:32.699 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-01-27 02:17:32.804 [info] <0.966.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-01-27 02:17:32.804 [info] <0.966.0> Successfully synced tables from a peer 2025-01-27 02:17:32.837 [info] <0.966.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-01-27 02:17:32.837 [info] <0.966.0> Successfully synced tables from a peer 2025-01-27 02:17:32.866 [info] <0.966.0> Waiting for Mnesia tables for 30000 ms, 0 retries left 2025-01-27 02:17:32.866 [info] <0.966.0> Successfully synced tables from a peer 2025-01-27 02:17:32.872 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-01-27 02:17:32.880 [info] <0.966.0> Clustering with ['contrail@cn-jenkins-deploy-platform-ansible-os-2660-3','contrail@cn-jenkins-deploy-platform-ansible-os-2660-1'] as disc node 2025-01-27 02:17:32.898 [info] <0.966.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-01-27 02:17:32.898 [info] <0.966.0> Successfully synced tables from a peer 2025-01-27 02:17:33.094 [info] <0.966.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-01-27 02:17:33.094 [info] <0.966.0> Successfully synced tables from a peer 2025-01-27 02:17:33.126 [info] <0.966.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-01-27 02:17:33.126 [info] <0.966.0> Successfully synced tables from a peer 2025-01-27 02:17:33.131 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-01-27 02:17:33.131 [error] <0.966.0> Failed to create a tracked connection table for node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': {node_not_running,'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2'} 2025-01-27 02:17:33.131 [info] <0.966.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2660-2' 2025-01-27 02:17:33.131 [error] <0.966.0> Failed to create a per-vhost tracked connection table for node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': {node_not_running,'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2'} 2025-01-27 02:17:33.131 [info] <0.966.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2660-2' 2025-01-27 02:17:34.042 [info] <0.1328.0> RabbitMQ is asked to stop... 2025-01-27 02:17:34.111 [info] <0.1328.0> Stopping RabbitMQ applications and their dependencies in the following order: rabbitmq_management amqp_client rabbitmq_web_dispatch rabbitmq_management_agent rabbit mnesia rabbit_common sysmon_handler os_mon 2025-01-27 02:17:34.112 [info] <0.1328.0> Stopping application 'rabbitmq_management' 2025-01-27 02:17:34.112 [info] <0.1328.0> Stopping application 'amqp_client' 2025-01-27 02:17:34.112 [info] <0.1328.0> Stopping application 'rabbitmq_web_dispatch' 2025-01-27 02:17:34.112 [info] <0.1328.0> Stopping application 'rabbitmq_management_agent' 2025-01-27 02:17:34.112 [info] <0.1328.0> Stopping application 'rabbit' 2025-01-27 02:17:34.112 [info] <0.1328.0> Stopping application 'mnesia' 2025-01-27 02:17:34.112 [info] <0.1328.0> Stopping application 'rabbit_common' 2025-01-27 02:17:34.112 [info] <0.1328.0> Stopping application 'sysmon_handler' 2025-01-27 02:17:34.112 [info] <0.1328.0> Stopping application 'os_mon' 2025-01-27 02:17:34.112 [info] <0.1328.0> Successfully stopped RabbitMQ and its dependencies 2025-01-27 02:17:34.112 [info] <0.1328.0> Halting Erlang VM with the following applications: jsx ranch ssl public_key asn1 observer_cli tools credentials_obfuscation crypto stdout_formatter xmerl recon inets lager goldrush compiler syntax_tools sasl stdlib kernel 2025-01-27 02:17:39.182 [info] <0.8.0> Log file opened with Lager 2025-01-27 02:17:40.709 [info] <0.8.0> Feature flags: list of feature flags found: 2025-01-27 02:17:40.709 [info] <0.8.0> Feature flags: feature flag states written to disk: yes 2025-01-27 02:17:40.779 [info] <0.8.0> upgrades: Backing up mnesia dir to "/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2-upgrade-backup" 2025-01-27 02:17:40.781 [info] <0.8.0> upgrades: Mnesia dir backed up to "/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2-upgrade-backup" 2025-01-27 02:17:40.785 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-01-27 02:17:40.828 [info] <0.301.0> Starting RabbitMQ 3.7.28 on Erlang 21.3.8.21 Copyright (c) 2007-2020 Pivotal Software, Inc. Licensed under the MPL. See https://www.rabbitmq.com/ 2025-01-27 02:17:40.829 [info] <0.301.0> node : contrail@cn-jenkins-deploy-platform-ansible-os-2660-2 home dir : /var/lib/rabbitmq config file(s) : /etc/rabbitmq/rabbitmq.config cookie hash : esZsDxSN6VGbi9JkMSxNZA== log(s) : /var/log/contrail/config-database-rabbitmq/rabbitmq.log : /var/log/rabbitmq/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2_upgrade.log database dir : /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2 2025-01-27 02:17:40.841 [info] <0.301.0> Running boot step pre_boot defined by app rabbit 2025-01-27 02:17:40.841 [info] <0.301.0> Running boot step rabbit_core_metrics defined by app rabbit 2025-01-27 02:17:40.842 [info] <0.301.0> Running boot step rabbit_alarm defined by app rabbit 2025-01-27 02:17:40.848 [info] <0.330.0> Memory high watermark set to 12637 MiB (13251084288 bytes) of 15796 MiB (16563855360 bytes) total 2025-01-27 02:17:40.856 [info] <0.348.0> Enabling free disk space monitoring 2025-01-27 02:17:40.856 [info] <0.348.0> Disk free limit set to 50MB 2025-01-27 02:17:40.863 [info] <0.301.0> Running boot step code_server_cache defined by app rabbit 2025-01-27 02:17:40.863 [info] <0.301.0> Running boot step file_handle_cache defined by app rabbit 2025-01-27 02:17:40.864 [info] <0.375.0> Limiting to approx 65436 file handles (58890 sockets) 2025-01-27 02:17:40.864 [info] <0.378.0> FHC read buffering: OFF 2025-01-27 02:17:40.864 [info] <0.378.0> FHC write buffering: ON 2025-01-27 02:17:40.864 [info] <0.301.0> Running boot step worker_pool defined by app rabbit 2025-01-27 02:17:40.864 [info] <0.303.0> Will use 4 processes for default worker pool 2025-01-27 02:17:40.864 [info] <0.303.0> Starting worker pool 'worker_pool' with 4 processes in it 2025-01-27 02:17:40.865 [info] <0.301.0> Running boot step database defined by app rabbit 2025-01-27 02:17:40.944 [info] <0.301.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-01-27 02:17:40.946 [info] <0.301.0> Successfully synced tables from a peer 2025-01-27 02:17:40.948 [info] <0.301.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-01-27 02:17:40.948 [info] <0.301.0> Successfully synced tables from a peer 2025-01-27 02:17:40.982 [info] <0.301.0> upgrades: Mnesia backup removed 2025-01-27 02:17:40.982 [info] <0.301.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-01-27 02:17:40.982 [info] <0.301.0> Successfully synced tables from a peer 2025-01-27 02:17:40.982 [info] <0.301.0> Will register with peer discovery backend rabbit_peer_discovery_classic_config 2025-01-27 02:17:40.982 [info] <0.301.0> Running boot step database_sync defined by app rabbit 2025-01-27 02:17:40.982 [info] <0.301.0> Running boot step feature_flags defined by app rabbit 2025-01-27 02:17:40.983 [info] <0.301.0> Running boot step codec_correctness_check defined by app rabbit 2025-01-27 02:17:40.983 [info] <0.301.0> Running boot step external_infrastructure defined by app rabbit 2025-01-27 02:17:40.983 [info] <0.301.0> Running boot step rabbit_registry defined by app rabbit 2025-01-27 02:17:40.983 [info] <0.301.0> Running boot step rabbit_auth_mechanism_cr_demo defined by app rabbit 2025-01-27 02:17:40.983 [info] <0.301.0> Running boot step rabbit_queue_location_random defined by app rabbit 2025-01-27 02:17:40.983 [info] <0.301.0> Running boot step rabbit_event defined by app rabbit 2025-01-27 02:17:40.984 [info] <0.301.0> Running boot step rabbit_auth_mechanism_amqplain defined by app rabbit 2025-01-27 02:17:40.984 [info] <0.301.0> Running boot step rabbit_auth_mechanism_plain defined by app rabbit 2025-01-27 02:17:40.984 [info] <0.301.0> Running boot step rabbit_exchange_type_direct defined by app rabbit 2025-01-27 02:17:40.984 [info] <0.301.0> Running boot step rabbit_exchange_type_fanout defined by app rabbit 2025-01-27 02:17:40.984 [info] <0.301.0> Running boot step rabbit_exchange_type_headers defined by app rabbit 2025-01-27 02:17:40.984 [info] <0.301.0> Running boot step rabbit_exchange_type_topic defined by app rabbit 2025-01-27 02:17:40.984 [info] <0.301.0> Running boot step rabbit_mirror_queue_mode_all defined by app rabbit 2025-01-27 02:17:40.984 [info] <0.301.0> Running boot step rabbit_mirror_queue_mode_exactly defined by app rabbit 2025-01-27 02:17:40.984 [info] <0.301.0> Running boot step rabbit_mirror_queue_mode_nodes defined by app rabbit 2025-01-27 02:17:40.985 [info] <0.301.0> Running boot step rabbit_priority_queue defined by app rabbit 2025-01-27 02:17:40.985 [info] <0.301.0> Priority queues enabled, real BQ is rabbit_variable_queue 2025-01-27 02:17:40.985 [info] <0.301.0> Running boot step rabbit_queue_location_client_local defined by app rabbit 2025-01-27 02:17:40.985 [info] <0.301.0> Running boot step rabbit_queue_location_min_masters defined by app rabbit 2025-01-27 02:17:40.985 [info] <0.301.0> Running boot step kernel_ready defined by app rabbit 2025-01-27 02:17:40.985 [info] <0.301.0> Running boot step rabbit_sysmon_minder defined by app rabbit 2025-01-27 02:17:40.985 [info] <0.301.0> Running boot step rabbit_epmd_monitor defined by app rabbit 2025-01-27 02:17:40.986 [info] <0.407.0> epmd monitor knows us, inter-node communication (distribution) port: 25673 2025-01-27 02:17:40.986 [info] <0.301.0> Running boot step guid_generator defined by app rabbit 2025-01-27 02:17:40.991 [info] <0.301.0> Running boot step rabbit_node_monitor defined by app rabbit 2025-01-27 02:17:40.993 [info] <0.411.0> Starting rabbit_node_monitor 2025-01-27 02:17:40.994 [info] <0.301.0> Running boot step delegate_sup defined by app rabbit 2025-01-27 02:17:40.994 [info] <0.301.0> Running boot step rabbit_memory_monitor defined by app rabbit 2025-01-27 02:17:40.994 [info] <0.301.0> Running boot step core_initialized defined by app rabbit 2025-01-27 02:17:40.994 [info] <0.301.0> Running boot step upgrade_queues defined by app rabbit 2025-01-27 02:17:41.016 [info] <0.301.0> message_store upgrades: 1 to apply 2025-01-27 02:17:41.017 [info] <0.301.0> message_store upgrades: Applying rabbit_variable_queue:move_messages_to_vhost_store 2025-01-27 02:17:41.017 [info] <0.301.0> message_store upgrades: No durable queues found. Skipping message store migration 2025-01-27 02:17:41.017 [info] <0.301.0> message_store upgrades: Removing the old message store data 2025-01-27 02:17:41.017 [info] <0.301.0> message_store upgrades: All upgrades applied successfully 2025-01-27 02:17:41.043 [info] <0.301.0> Running boot step rabbit_connection_tracking defined by app rabbit 2025-01-27 02:17:41.043 [info] <0.301.0> Running boot step rabbit_connection_tracking_handler defined by app rabbit 2025-01-27 02:17:41.043 [info] <0.301.0> Running boot step rabbit_exchange_parameters defined by app rabbit 2025-01-27 02:17:41.043 [info] <0.301.0> Running boot step rabbit_mirror_queue_misc defined by app rabbit 2025-01-27 02:17:41.044 [info] <0.301.0> Running boot step rabbit_policies defined by app rabbit 2025-01-27 02:17:41.044 [info] <0.301.0> Running boot step rabbit_policy defined by app rabbit 2025-01-27 02:17:41.044 [info] <0.301.0> Running boot step rabbit_queue_location_validator defined by app rabbit 2025-01-27 02:17:41.045 [info] <0.301.0> Running boot step rabbit_vhost_limit defined by app rabbit 2025-01-27 02:17:41.045 [info] <0.301.0> Running boot step rabbit_mgmt_reset_handler defined by app rabbitmq_management 2025-01-27 02:17:41.045 [info] <0.301.0> Running boot step rabbit_mgmt_db_handler defined by app rabbitmq_management_agent 2025-01-27 02:17:41.045 [warning] <0.301.0> force_fine_statistics set to true; ignored. Replaced by {rates_mode, none} in the rabbitmq_management application. 2025-01-27 02:17:41.045 [info] <0.301.0> Management plugin: using rates mode 'basic' 2025-01-27 02:17:41.047 [info] <0.301.0> Running boot step recovery defined by app rabbit 2025-01-27 02:17:41.048 [info] <0.448.0> Making sure data directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2660-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists 2025-01-27 02:17:41.053 [info] <0.448.0> Starting message stores for vhost '/' 2025-01-27 02:17:41.053 [info] <0.452.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index 2025-01-27 02:17:41.054 [info] <0.448.0> Started message store of type transient for vhost '/' 2025-01-27 02:17:41.054 [info] <0.455.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index 2025-01-27 02:17:41.055 [warning] <0.455.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": rebuilding indices from scratch 2025-01-27 02:17:41.055 [info] <0.448.0> Started message store of type persistent for vhost '/' 2025-01-27 02:17:41.057 [info] <0.448.0> Mirrored queue 'job_status_consumer.dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.474.0> 2025-01-27 02:17:41.057 [info] <0.448.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2660-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.478.0> 2025-01-27 02:17:41.058 [info] <0.448.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2660-2.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2660-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.482.0> 2025-01-27 02:17:41.058 [info] <0.448.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2660-2..tftp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.486.0> 2025-01-27 02:17:41.058 [info] <0.448.0> Mirrored queue 'device_manager.cn-jenkins-deploy-platform-ansible-os-2660-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.490.0> 2025-01-27 02:17:41.058 [info] <0.448.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2660-3.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2660-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.494.0> 2025-01-27 02:17:41.059 [info] <0.448.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2660-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.498.0> 2025-01-27 02:17:41.059 [info] <0.448.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2660-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.502.0> 2025-01-27 02:17:41.059 [info] <0.448.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2660-3.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2660-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.506.0> 2025-01-27 02:17:41.059 [info] <0.448.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2660-1.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.510.0> 2025-01-27 02:17:41.059 [info] <0.448.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2660-1' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.514.0> 2025-01-27 02:17:41.060 [info] <0.448.0> Mirrored queue 'job_request_consumer' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.518.0> 2025-01-27 02:17:41.060 [info] <0.448.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2660-1.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2660-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.522.0> 2025-01-27 02:17:41.061 [info] <0.448.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2660-2.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.526.0> 2025-01-27 02:17:41.061 [info] <0.448.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2660-1.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2660-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.531.0> 2025-01-27 02:17:41.061 [info] <0.448.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2660-2' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.536.0> 2025-01-27 02:17:41.062 [info] <0.448.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2660-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.542.0> 2025-01-27 02:17:41.062 [info] <0.448.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2660-1' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.546.0> 2025-01-27 02:17:41.063 [info] <0.448.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2660-3' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.551.0> 2025-01-27 02:17:41.064 [info] <0.448.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2660-3.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.559.0> 2025-01-27 02:17:41.065 [info] <0.448.0> Mirrored queue 'schema_transformer.cn-jenkins-deploy-platform-ansible-os-2660-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.567.0> 2025-01-27 02:17:41.065 [info] <0.448.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2660-3..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.572.0> 2025-01-27 02:17:41.065 [info] <0.448.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2660-3.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2660-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.576.0> 2025-01-27 02:17:41.065 [info] <0.448.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2660-2.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2660-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.580.0> 2025-01-27 02:17:41.066 [info] <0.448.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2660-2.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2660-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.584.0> 2025-01-27 02:17:41.066 [info] <0.448.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2660-2..config_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.588.0> 2025-01-27 02:17:41.067 [info] <0.448.0> Mirrored queue 'svc_monitor.cn-jenkins-deploy-platform-ansible-os-2660-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.594.0> 2025-01-27 02:17:41.067 [info] <0.448.0> Mirrored queue 'device_manager_ztp.ztp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.599.0> 2025-01-27 02:17:41.067 [info] <0.448.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2660-2' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.603.0> 2025-01-27 02:17:41.067 [info] <0.448.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2660-3' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.610.0> 2025-01-27 02:17:41.068 [info] <0.448.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2660-1..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.614.0> 2025-01-27 02:17:41.068 [info] <0.448.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2660-1.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2660-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.618.0> 2025-01-27 02:17:41.068 [info] <0.448.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2660-2..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-2': <0.622.0> 2025-01-27 02:17:41.069 [info] <0.301.0> Running boot step load_definitions defined by app rabbitmq_management 2025-01-27 02:17:41.069 [info] <0.301.0> Applying definitions from /etc/rabbitmq/definitions.json 2025-01-27 02:17:41.069 [info] <0.301.0> Asked to import definitions. Acting user: rmq-internal 2025-01-27 02:17:41.069 [info] <0.301.0> Importing users... 2025-01-27 02:17:41.095 [info] <0.301.0> Setting user tags for user 'guest' to [administrator] 2025-01-27 02:17:41.116 [info] <0.301.0> Importing vhosts... 2025-01-27 02:17:41.116 [info] <0.301.0> Importing user permissions... 2025-01-27 02:17:41.116 [info] <0.301.0> Setting permissions for 'guest' in '/' to '.*', '.*', '.*' 2025-01-27 02:17:41.138 [info] <0.301.0> Importing topic permissions... 2025-01-27 02:17:41.138 [info] <0.301.0> Importing parameters... 2025-01-27 02:17:41.138 [info] <0.301.0> Importing global parameters... 2025-01-27 02:17:41.138 [info] <0.301.0> Importing policies... 2025-01-27 02:17:41.151 [info] <0.301.0> Importing queues... 2025-01-27 02:17:41.152 [info] <0.301.0> Importing exchanges... 2025-01-27 02:17:41.152 [info] <0.301.0> Importing bindings... 2025-01-27 02:17:41.152 [info] <0.301.0> Running boot step empty_db_check defined by app rabbit 2025-01-27 02:17:41.152 [info] <0.301.0> Running boot step rabbit_looking_glass defined by app rabbit 2025-01-27 02:17:41.152 [info] <0.301.0> Running boot step rabbit_core_metrics_gc defined by app rabbit 2025-01-27 02:17:41.152 [info] <0.301.0> Running boot step background_gc defined by app rabbit 2025-01-27 02:17:41.152 [info] <0.301.0> Running boot step connection_tracking defined by app rabbit 2025-01-27 02:17:41.180 [info] <0.301.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2660-2' 2025-01-27 02:17:41.201 [info] <0.301.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2660-2' 2025-01-27 02:17:41.201 [info] <0.301.0> Running boot step routing_ready defined by app rabbit 2025-01-27 02:17:41.201 [info] <0.301.0> Running boot step pre_flight defined by app rabbit 2025-01-27 02:17:41.201 [info] <0.301.0> Running boot step notify_cluster defined by app rabbit 2025-01-27 02:17:41.201 [info] <0.301.0> Running boot step networking defined by app rabbit 2025-01-27 02:17:41.202 [info] <0.411.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-1' up 2025-01-27 02:17:41.204 [info] <0.733.0> started TCP listener on [::]:5673 2025-01-27 02:17:41.205 [info] <0.301.0> Running boot step cluster_name defined by app rabbit 2025-01-27 02:17:41.205 [info] <0.301.0> Running boot step direct_client defined by app rabbit 2025-01-27 02:17:41.205 [info] <0.301.0> Running boot step os_signal_handler defined by app rabbit 2025-01-27 02:17:41.205 [info] <0.735.0> Swapping OS signal event handler (erl_signal_server) for our own 2025-01-27 02:17:41.237 [info] <0.789.0> Management plugin: HTTP (non-TLS) listener started on port 15673 2025-01-27 02:17:41.237 [info] <0.925.0> Statistics database started. 2025-01-27 02:17:41.237 [info] <0.924.0> Starting worker pool 'management_worker_pool' with 3 processes in it 2025-01-27 02:17:41.238 [notice] <0.106.0> Changed loghwm of /var/log/contrail/config-database-rabbitmq/rabbitmq.log to 50 2025-01-27 02:17:41.333 [info] <0.8.0> Server startup complete; 3 plugins started. * rabbitmq_management * rabbitmq_management_agent * rabbitmq_web_dispatch 2025-01-27 02:17:51.783 [info] <0.411.0> node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-3' up 2025-01-27 02:18:34.376 [info] <0.411.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-3' up 2025-01-27 02:18:41.574 [info] <0.411.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-3' down 2025-01-27 02:18:41.582 [info] <0.411.0> Keeping contrail@cn-jenkins-deploy-platform-ansible-os-2660-3 listeners: the node is already back 2025-01-27 02:18:41.681 [info] <0.474.0> Mirrored queue 'job_status_consumer.dummy' in vhost '/': Slave saw deaths of mirrors 2025-01-27 02:18:41.681 [info] <0.588.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2660-2..config_queue' in vhost '/': Slave saw deaths of mirrors 2025-01-27 02:18:41.682 [info] <0.618.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2660-1.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2660-1.' in vhost '/': Slave saw deaths of mirrors 2025-01-27 02:18:41.684 [info] <0.526.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2660-2.-8082-0' in vhost '/': Slave saw deaths of mirrors 2025-01-27 02:18:41.684 [info] <0.594.0> Mirrored queue 'svc_monitor.cn-jenkins-deploy-platform-ansible-os-2660-2.' in vhost '/': Slave saw deaths of mirrors 2025-01-27 02:18:41.685 [info] <0.490.0> Mirrored queue 'device_manager.cn-jenkins-deploy-platform-ansible-os-2660-2.' in vhost '/': Slave saw deaths of mirrors 2025-01-27 02:18:41.685 [info] <0.518.0> Mirrored queue 'job_request_consumer' in vhost '/': Slave saw deaths of mirrors 2025-01-27 02:18:41.689 [info] <0.486.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2660-2..tftp_queue' in vhost '/': Slave saw deaths of mirrors 2025-01-27 02:18:41.698 [info] <0.506.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2660-3.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2660-3.' in vhost '/': Slave saw deaths of mirrors 2025-01-27 02:18:41.698 [info] <0.559.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2660-3.-8082-0' in vhost '/': Slave saw deaths of mirrors 2025-01-27 02:18:41.698 [info] <0.502.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2660-2.' in vhost '/': Slave saw deaths of mirrors 2025-01-27 02:18:41.718 [info] <0.494.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2660-3.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2660-3.' in vhost '/': Slave saw deaths of mirrors 2025-01-27 02:18:41.719 [info] <0.536.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2660-2' in vhost '/': Slave saw deaths of mirrors 2025-01-27 02:18:41.719 [info] <0.522.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2660-1.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2660-1.' in vhost '/': Slave saw deaths of mirrors 2025-01-27 02:18:41.720 [info] <0.584.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2660-2.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2660-2.' in vhost '/': Slave saw deaths of mirrors 2025-01-27 02:18:41.721 [info] <0.572.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2660-3..dummy' in vhost '/': Slave saw deaths of mirrors 2025-01-27 02:18:41.721 [info] <0.580.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2660-2.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2660-2.' in vhost '/': Slave saw deaths of mirrors 2025-01-27 02:18:41.722 [info] <0.567.0> Mirrored queue 'schema_transformer.cn-jenkins-deploy-platform-ansible-os-2660-2.' in vhost '/': Slave saw deaths of mirrors 2025-01-27 02:18:41.757 [info] <0.542.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2660-2.' in vhost '/': Slave saw deaths of mirrors 2025-01-27 02:18:52.471 [info] <0.411.0> node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-3' down: connection_closed 2025-01-27 02:18:53.368 [info] <0.411.0> node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-3' up 2025-01-27 02:18:55.976 [info] <0.411.0> node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-3' down: connection_closed 2025-01-27 02:19:01.735 [info] <0.411.0> node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-3' up 2025-01-27 02:19:02.131 [info] <0.411.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2660-3' up