2025-02-06 00:46:31.750 [info] <0.8.0> Log file opened with Lager 2025-02-06 00:46:33.831 [info] <0.8.0> Feature flags: list of feature flags found: 2025-02-06 00:46:33.831 [info] <0.8.0> Feature flags: feature flag states written to disk: yes 2025-02-06 00:46:33.848 [info] <0.240.0> Starting RabbitMQ 3.7.28 on Erlang 21.3.8.21 Copyright (c) 2007-2020 Pivotal Software, Inc. Licensed under the MPL. See https://www.rabbitmq.com/ 2025-02-06 00:46:33.849 [info] <0.240.0> node : contrail@cn-jenkins-deploy-platform-ansible-os-2696-2 home dir : /var/lib/rabbitmq config file(s) : /etc/rabbitmq/rabbitmq.config cookie hash : esZsDxSN6VGbi9JkMSxNZA== log(s) : /var/log/contrail/config-database-rabbitmq/rabbitmq.log : /var/log/rabbitmq/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2_upgrade.log database dir : /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2 2025-02-06 00:46:33.861 [info] <0.240.0> Running boot step pre_boot defined by app rabbit 2025-02-06 00:46:33.861 [info] <0.240.0> Running boot step rabbit_core_metrics defined by app rabbit 2025-02-06 00:46:33.861 [info] <0.240.0> Running boot step rabbit_alarm defined by app rabbit 2025-02-06 00:46:33.866 [info] <0.246.0> Memory high watermark set to 12637 MiB (13251077734 bytes) of 15796 MiB (16563847168 bytes) total 2025-02-06 00:46:33.870 [info] <0.248.0> Enabling free disk space monitoring 2025-02-06 00:46:33.870 [info] <0.248.0> Disk free limit set to 50MB 2025-02-06 00:46:33.874 [info] <0.240.0> Running boot step code_server_cache defined by app rabbit 2025-02-06 00:46:33.874 [info] <0.240.0> Running boot step file_handle_cache defined by app rabbit 2025-02-06 00:46:33.874 [info] <0.251.0> Limiting to approx 65436 file handles (58890 sockets) 2025-02-06 00:46:33.875 [info] <0.252.0> FHC read buffering: OFF 2025-02-06 00:46:33.875 [info] <0.252.0> FHC write buffering: ON 2025-02-06 00:46:33.875 [info] <0.240.0> Running boot step worker_pool defined by app rabbit 2025-02-06 00:46:33.875 [info] <0.241.0> Will use 4 processes for default worker pool 2025-02-06 00:46:33.875 [info] <0.241.0> Starting worker pool 'worker_pool' with 4 processes in it 2025-02-06 00:46:33.876 [info] <0.240.0> Running boot step database defined by app rabbit 2025-02-06 00:46:33.876 [info] <0.240.0> Node database directory at /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2 is empty. Assuming we need to join an existing cluster or initialise from scratch... 2025-02-06 00:46:33.876 [info] <0.240.0> Configured peer discovery backend: rabbit_peer_discovery_classic_config 2025-02-06 00:46:33.876 [info] <0.240.0> Will try to lock with peer discovery backend rabbit_peer_discovery_classic_config 2025-02-06 00:46:33.876 [info] <0.240.0> Peer discovery backend does not support locking, falling back to randomized delay 2025-02-06 00:46:33.876 [info] <0.240.0> Peer discovery backend rabbit_peer_discovery_classic_config supports registration. 2025-02-06 00:46:33.876 [info] <0.240.0> Will wait for 19706 milliseconds before proceeding with registration... 2025-02-06 00:46:53.583 [info] <0.240.0> All discovered existing cluster peers: contrail@cn-jenkins-deploy-platform-ansible-os-2696-1, contrail@cn-jenkins-deploy-platform-ansible-os-2696-2, contrail@cn-jenkins-deploy-platform-ansible-os-2696-3 2025-02-06 00:46:53.583 [info] <0.240.0> Peer nodes we can cluster with: contrail@cn-jenkins-deploy-platform-ansible-os-2696-1, contrail@cn-jenkins-deploy-platform-ansible-os-2696-3 2025-02-06 00:46:53.587 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-2696-1: {error,tables_not_present} 2025-02-06 00:46:53.589 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-2696-3: {error,tables_not_present} 2025-02-06 00:46:53.589 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 9 retries left... 2025-02-06 00:46:54.091 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-2696-1: {error,tables_not_present} 2025-02-06 00:46:54.092 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-2696-3: {error,tables_not_present} 2025-02-06 00:46:54.092 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 8 retries left... 2025-02-06 00:46:54.593 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-2696-1: {error,tables_not_present} 2025-02-06 00:46:54.594 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-2696-3: {error,tables_not_present} 2025-02-06 00:46:54.594 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 7 retries left... 2025-02-06 00:46:55.095 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-2696-1: {error,tables_not_present} 2025-02-06 00:46:55.096 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-2696-3: {error,tables_not_present} 2025-02-06 00:46:55.096 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 6 retries left... 2025-02-06 00:46:55.597 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-2696-1: {error,tables_not_present} 2025-02-06 00:46:55.598 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-2696-3: {error,tables_not_present} 2025-02-06 00:46:55.598 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 5 retries left... 2025-02-06 00:46:56.099 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-2696-1: {error,tables_not_present} 2025-02-06 00:46:56.100 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-2696-3: {error,tables_not_present} 2025-02-06 00:46:56.100 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 4 retries left... 2025-02-06 00:46:56.602 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-2696-1: {error,tables_not_present} 2025-02-06 00:46:56.602 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-2696-3: {error,tables_not_present} 2025-02-06 00:46:56.602 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 3 retries left... 2025-02-06 00:46:57.103 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-2696-1: {error,tables_not_present} 2025-02-06 00:46:57.104 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-2696-3: {error,tables_not_present} 2025-02-06 00:46:57.104 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 2 retries left... 2025-02-06 00:46:57.605 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-2696-1: {error,tables_not_present} 2025-02-06 00:46:57.606 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-2696-3: {error,tables_not_present} 2025-02-06 00:46:57.606 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 1 retries left... 2025-02-06 00:46:58.107 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-2696-1: {error,tables_not_present} 2025-02-06 00:46:58.108 [warning] <0.240.0> Could not auto-cluster with node contrail@cn-jenkins-deploy-platform-ansible-os-2696-3: {error,tables_not_present} 2025-02-06 00:46:58.108 [error] <0.240.0> Trying to join discovered peers failed. Will retry after a delay of 500 ms, 0 retries left... 2025-02-06 00:46:58.609 [warning] <0.240.0> Could not successfully contact any node of: contrail@cn-jenkins-deploy-platform-ansible-os-2696-1,contrail@cn-jenkins-deploy-platform-ansible-os-2696-3 (as in Erlang distribution). Starting as a blank standalone node... 2025-02-06 00:46:58.610 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-02-06 00:46:58.688 [info] <0.240.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-02-06 00:46:58.688 [info] <0.240.0> Successfully synced tables from a peer 2025-02-06 00:46:58.725 [info] <0.240.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-02-06 00:46:58.725 [info] <0.240.0> Successfully synced tables from a peer 2025-02-06 00:46:58.754 [info] <0.240.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-02-06 00:46:58.754 [info] <0.240.0> Successfully synced tables from a peer 2025-02-06 00:46:58.754 [info] <0.240.0> Will register with peer discovery backend rabbit_peer_discovery_classic_config 2025-02-06 00:46:58.754 [info] <0.240.0> Running boot step database_sync defined by app rabbit 2025-02-06 00:46:58.754 [info] <0.240.0> Running boot step feature_flags defined by app rabbit 2025-02-06 00:46:58.755 [info] <0.240.0> Running boot step codec_correctness_check defined by app rabbit 2025-02-06 00:46:58.755 [info] <0.240.0> Running boot step external_infrastructure defined by app rabbit 2025-02-06 00:46:58.755 [info] <0.240.0> Running boot step rabbit_registry defined by app rabbit 2025-02-06 00:46:58.755 [info] <0.240.0> Running boot step rabbit_auth_mechanism_cr_demo defined by app rabbit 2025-02-06 00:46:58.755 [info] <0.240.0> Running boot step rabbit_queue_location_random defined by app rabbit 2025-02-06 00:46:58.755 [info] <0.240.0> Running boot step rabbit_event defined by app rabbit 2025-02-06 00:46:58.755 [info] <0.240.0> Running boot step rabbit_auth_mechanism_amqplain defined by app rabbit 2025-02-06 00:46:58.755 [info] <0.240.0> Running boot step rabbit_auth_mechanism_plain defined by app rabbit 2025-02-06 00:46:58.755 [info] <0.240.0> Running boot step rabbit_exchange_type_direct defined by app rabbit 2025-02-06 00:46:58.755 [info] <0.240.0> Running boot step rabbit_exchange_type_fanout defined by app rabbit 2025-02-06 00:46:58.755 [info] <0.240.0> Running boot step rabbit_exchange_type_headers defined by app rabbit 2025-02-06 00:46:58.755 [info] <0.240.0> Running boot step rabbit_exchange_type_topic defined by app rabbit 2025-02-06 00:46:58.756 [info] <0.240.0> Running boot step rabbit_mirror_queue_mode_all defined by app rabbit 2025-02-06 00:46:58.756 [info] <0.240.0> Running boot step rabbit_mirror_queue_mode_exactly defined by app rabbit 2025-02-06 00:46:58.756 [info] <0.240.0> Running boot step rabbit_mirror_queue_mode_nodes defined by app rabbit 2025-02-06 00:46:58.756 [info] <0.240.0> Running boot step rabbit_priority_queue defined by app rabbit 2025-02-06 00:46:58.756 [info] <0.240.0> Priority queues enabled, real BQ is rabbit_variable_queue 2025-02-06 00:46:58.756 [info] <0.240.0> Running boot step rabbit_queue_location_client_local defined by app rabbit 2025-02-06 00:46:58.756 [info] <0.240.0> Running boot step rabbit_queue_location_min_masters defined by app rabbit 2025-02-06 00:46:58.756 [info] <0.240.0> Running boot step kernel_ready defined by app rabbit 2025-02-06 00:46:58.756 [info] <0.240.0> Running boot step rabbit_sysmon_minder defined by app rabbit 2025-02-06 00:46:58.756 [info] <0.240.0> Running boot step rabbit_epmd_monitor defined by app rabbit 2025-02-06 00:46:58.757 [info] <0.440.0> epmd monitor knows us, inter-node communication (distribution) port: 25673 2025-02-06 00:46:58.757 [info] <0.240.0> Running boot step guid_generator defined by app rabbit 2025-02-06 00:46:58.760 [info] <0.240.0> Running boot step rabbit_node_monitor defined by app rabbit 2025-02-06 00:46:58.760 [info] <0.444.0> Starting rabbit_node_monitor 2025-02-06 00:46:58.760 [info] <0.240.0> Running boot step delegate_sup defined by app rabbit 2025-02-06 00:46:58.760 [info] <0.240.0> Running boot step rabbit_memory_monitor defined by app rabbit 2025-02-06 00:46:58.760 [info] <0.240.0> Running boot step core_initialized defined by app rabbit 2025-02-06 00:46:58.760 [info] <0.240.0> Running boot step upgrade_queues defined by app rabbit 2025-02-06 00:46:58.779 [info] <0.240.0> message_store upgrades: 1 to apply 2025-02-06 00:46:58.780 [info] <0.240.0> message_store upgrades: Applying rabbit_variable_queue:move_messages_to_vhost_store 2025-02-06 00:46:58.780 [info] <0.240.0> message_store upgrades: No durable queues found. Skipping message store migration 2025-02-06 00:46:58.780 [info] <0.240.0> message_store upgrades: Removing the old message store data 2025-02-06 00:46:58.780 [info] <0.240.0> message_store upgrades: All upgrades applied successfully 2025-02-06 00:46:58.801 [info] <0.240.0> Running boot step rabbit_connection_tracking defined by app rabbit 2025-02-06 00:46:58.802 [info] <0.240.0> Running boot step rabbit_connection_tracking_handler defined by app rabbit 2025-02-06 00:46:58.802 [info] <0.240.0> Running boot step rabbit_exchange_parameters defined by app rabbit 2025-02-06 00:46:58.802 [info] <0.240.0> Running boot step rabbit_mirror_queue_misc defined by app rabbit 2025-02-06 00:46:58.802 [info] <0.240.0> Running boot step rabbit_policies defined by app rabbit 2025-02-06 00:46:58.803 [info] <0.240.0> Running boot step rabbit_policy defined by app rabbit 2025-02-06 00:46:58.803 [info] <0.240.0> Running boot step rabbit_queue_location_validator defined by app rabbit 2025-02-06 00:46:58.803 [info] <0.240.0> Running boot step rabbit_vhost_limit defined by app rabbit 2025-02-06 00:46:58.803 [info] <0.240.0> Running boot step rabbit_mgmt_reset_handler defined by app rabbitmq_management 2025-02-06 00:46:58.803 [info] <0.240.0> Running boot step rabbit_mgmt_db_handler defined by app rabbitmq_management_agent 2025-02-06 00:46:58.803 [warning] <0.240.0> force_fine_statistics set to true; ignored. Replaced by {rates_mode, none} in the rabbitmq_management application. 2025-02-06 00:46:58.803 [info] <0.240.0> Management plugin: using rates mode 'basic' 2025-02-06 00:46:58.803 [info] <0.240.0> Running boot step recovery defined by app rabbit 2025-02-06 00:46:58.804 [info] <0.240.0> Running boot step load_definitions defined by app rabbitmq_management 2025-02-06 00:46:58.804 [info] <0.240.0> Applying definitions from /etc/rabbitmq/definitions.json 2025-02-06 00:46:58.804 [info] <0.240.0> Asked to import definitions. Acting user: rmq-internal 2025-02-06 00:46:58.804 [info] <0.240.0> Importing users... 2025-02-06 00:46:58.804 [info] <0.240.0> Creating user 'guest' 2025-02-06 00:46:58.809 [info] <0.240.0> Setting user tags for user 'guest' to [administrator] 2025-02-06 00:46:58.811 [info] <0.240.0> Importing vhosts... 2025-02-06 00:46:58.811 [info] <0.240.0> Adding vhost '/' 2025-02-06 00:46:58.827 [info] <0.492.0> Making sure data directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists 2025-02-06 00:46:58.831 [info] <0.492.0> Starting message stores for vhost '/' 2025-02-06 00:46:58.831 [info] <0.496.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index 2025-02-06 00:46:58.832 [info] <0.492.0> Started message store of type transient for vhost '/' 2025-02-06 00:46:58.832 [info] <0.499.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index 2025-02-06 00:46:58.832 [warning] <0.499.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": rebuilding indices from scratch 2025-02-06 00:46:58.832 [info] <0.492.0> Started message store of type persistent for vhost '/' 2025-02-06 00:46:58.833 [info] <0.240.0> Importing user permissions... 2025-02-06 00:46:58.833 [info] <0.240.0> Setting permissions for 'guest' in '/' to '.*', '.*', '.*' 2025-02-06 00:46:58.836 [info] <0.240.0> Importing topic permissions... 2025-02-06 00:46:58.836 [info] <0.240.0> Importing parameters... 2025-02-06 00:46:58.836 [info] <0.240.0> Importing global parameters... 2025-02-06 00:46:58.836 [info] <0.240.0> Importing policies... 2025-02-06 00:46:58.841 [info] <0.240.0> Importing queues... 2025-02-06 00:46:58.841 [info] <0.240.0> Importing exchanges... 2025-02-06 00:46:58.841 [info] <0.240.0> Importing bindings... 2025-02-06 00:46:58.841 [info] <0.240.0> Running boot step empty_db_check defined by app rabbit 2025-02-06 00:46:58.841 [info] <0.240.0> Running boot step rabbit_looking_glass defined by app rabbit 2025-02-06 00:46:58.841 [info] <0.240.0> Running boot step rabbit_core_metrics_gc defined by app rabbit 2025-02-06 00:46:58.841 [info] <0.240.0> Running boot step background_gc defined by app rabbit 2025-02-06 00:46:58.841 [info] <0.240.0> Running boot step connection_tracking defined by app rabbit 2025-02-06 00:46:58.845 [info] <0.240.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2696-2' 2025-02-06 00:46:58.848 [info] <0.240.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2696-2' 2025-02-06 00:46:58.848 [info] <0.240.0> Running boot step routing_ready defined by app rabbit 2025-02-06 00:46:58.848 [info] <0.240.0> Running boot step pre_flight defined by app rabbit 2025-02-06 00:46:58.848 [info] <0.240.0> Running boot step notify_cluster defined by app rabbit 2025-02-06 00:46:58.848 [info] <0.240.0> Running boot step networking defined by app rabbit 2025-02-06 00:46:58.849 [info] <0.545.0> started TCP listener on [::]:5673 2025-02-06 00:46:58.849 [info] <0.240.0> Running boot step cluster_name defined by app rabbit 2025-02-06 00:46:58.850 [info] <0.240.0> Running boot step direct_client defined by app rabbit 2025-02-06 00:46:58.850 [info] <0.240.0> Running boot step os_signal_handler defined by app rabbit 2025-02-06 00:46:58.850 [info] <0.547.0> Swapping OS signal event handler (erl_signal_server) for our own 2025-02-06 00:46:58.875 [info] <0.597.0> Management plugin: HTTP (non-TLS) listener started on port 15673 2025-02-06 00:46:58.875 [info] <0.703.0> Statistics database started. 2025-02-06 00:46:58.875 [info] <0.702.0> Starting worker pool 'management_worker_pool' with 3 processes in it 2025-02-06 00:46:58.876 [notice] <0.106.0> Changed loghwm of /var/log/contrail/config-database-rabbitmq/rabbitmq.log to 50 2025-02-06 00:46:58.958 [info] <0.8.0> Server startup complete; 3 plugins started. * rabbitmq_management * rabbitmq_management_agent * rabbitmq_web_dispatch 2025-02-06 00:46:59.898 [info] <0.727.0> RabbitMQ is asked to stop... 2025-02-06 00:46:59.941 [info] <0.727.0> Stopping RabbitMQ applications and their dependencies in the following order: rabbitmq_management amqp_client rabbitmq_web_dispatch cowboy cowlib rabbitmq_management_agent rabbit mnesia rabbit_common sysmon_handler os_mon 2025-02-06 00:46:59.941 [info] <0.727.0> Stopping application 'rabbitmq_management' 2025-02-06 00:46:59.942 [warning] <0.553.0> RabbitMQ HTTP listener registry could not find context rabbitmq_management_tls 2025-02-06 00:46:59.944 [info] <0.43.0> Application rabbitmq_management exited with reason: stopped 2025-02-06 00:46:59.944 [info] <0.727.0> Stopping application 'amqp_client' 2025-02-06 00:46:59.945 [info] <0.43.0> Application amqp_client exited with reason: stopped 2025-02-06 00:46:59.945 [info] <0.727.0> Stopping application 'rabbitmq_web_dispatch' 2025-02-06 00:46:59.946 [info] <0.43.0> Application rabbitmq_web_dispatch exited with reason: stopped 2025-02-06 00:46:59.946 [info] <0.727.0> Stopping application 'cowboy' 2025-02-06 00:46:59.947 [info] <0.43.0> Application cowboy exited with reason: stopped 2025-02-06 00:46:59.947 [info] <0.727.0> Stopping application 'cowlib' 2025-02-06 00:46:59.947 [info] <0.727.0> Stopping application 'rabbitmq_management_agent' 2025-02-06 00:46:59.947 [info] <0.43.0> Application cowlib exited with reason: stopped 2025-02-06 00:46:59.948 [info] <0.43.0> Application rabbitmq_management_agent exited with reason: stopped 2025-02-06 00:46:59.948 [info] <0.727.0> Stopping application 'rabbit' 2025-02-06 00:46:59.949 [info] <0.240.0> Will unregister with peer discovery backend rabbit_peer_discovery_classic_config 2025-02-06 00:46:59.949 [info] <0.545.0> stopped TCP listener on [::]:5673 2025-02-06 00:46:59.949 [info] <0.469.0> Closing all connections in vhost '/' on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2' because the vhost is stopping 2025-02-06 00:46:59.949 [info] <0.499.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' 2025-02-06 00:46:59.978 [info] <0.499.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' is stopped 2025-02-06 00:46:59.978 [info] <0.496.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' 2025-02-06 00:46:59.984 [info] <0.496.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' is stopped 2025-02-06 00:46:59.987 [info] <0.43.0> Application rabbit exited with reason: stopped 2025-02-06 00:46:59.987 [info] <0.727.0> Stopping application 'mnesia' 2025-02-06 00:46:59.988 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-02-06 00:46:59.988 [info] <0.727.0> Stopping application 'rabbit_common' 2025-02-06 00:46:59.988 [info] <0.43.0> Application rabbit_common exited with reason: stopped 2025-02-06 00:46:59.988 [info] <0.727.0> Stopping application 'sysmon_handler' 2025-02-06 00:46:59.989 [info] <0.43.0> Application sysmon_handler exited with reason: stopped 2025-02-06 00:46:59.989 [info] <0.727.0> Stopping application 'os_mon' 2025-02-06 00:46:59.990 [info] <0.43.0> Application os_mon exited with reason: stopped 2025-02-06 00:46:59.990 [info] <0.727.0> Successfully stopped RabbitMQ and its dependencies 2025-02-06 00:47:10.015 [info] <0.742.0> Resetting Rabbit forcefully 2025-02-06 00:47:11.321 [info] <0.754.0> RabbitMQ is asked to stop... 2025-02-06 00:47:11.374 [info] <0.754.0> Stopping RabbitMQ applications and their dependencies in the following order: rabbitmq_management amqp_client rabbitmq_management_agent rabbitmq_web_dispatch rabbit mnesia rabbit_common sysmon_handler os_mon 2025-02-06 00:47:11.374 [info] <0.754.0> Stopping application 'rabbitmq_management' 2025-02-06 00:47:11.374 [info] <0.754.0> Stopping application 'amqp_client' 2025-02-06 00:47:11.374 [info] <0.754.0> Stopping application 'rabbitmq_management_agent' 2025-02-06 00:47:11.374 [info] <0.754.0> Stopping application 'rabbitmq_web_dispatch' 2025-02-06 00:47:11.374 [info] <0.754.0> Stopping application 'rabbit' 2025-02-06 00:47:11.374 [info] <0.754.0> Stopping application 'mnesia' 2025-02-06 00:47:11.374 [info] <0.754.0> Stopping application 'rabbit_common' 2025-02-06 00:47:11.374 [info] <0.754.0> Stopping application 'sysmon_handler' 2025-02-06 00:47:11.374 [info] <0.754.0> Stopping application 'os_mon' 2025-02-06 00:47:11.375 [info] <0.754.0> Successfully stopped RabbitMQ and its dependencies 2025-02-06 00:47:11.375 [info] <0.754.0> Halting Erlang VM with the following applications: ranch ssl public_key asn1 observer_cli tools credentials_obfuscation crypto stdout_formatter recon inets jsx xmerl lager goldrush compiler syntax_tools sasl stdlib kernel 2025-02-06 00:47:15.381 [info] <0.8.0> Log file opened with Lager 2025-02-06 00:47:16.983 [info] <0.8.0> Feature flags: list of feature flags found: 2025-02-06 00:47:16.984 [info] <0.8.0> Feature flags: feature flag states written to disk: yes 2025-02-06 00:47:17.001 [info] <0.253.0> Starting RabbitMQ 3.7.28 on Erlang 21.3.8.21 Copyright (c) 2007-2020 Pivotal Software, Inc. Licensed under the MPL. See https://www.rabbitmq.com/ 2025-02-06 00:47:17.001 [info] <0.253.0> node : contrail@cn-jenkins-deploy-platform-ansible-os-2696-2 home dir : /var/lib/rabbitmq config file(s) : /etc/rabbitmq/rabbitmq.config cookie hash : esZsDxSN6VGbi9JkMSxNZA== log(s) : /var/log/contrail/config-database-rabbitmq/rabbitmq.log : /var/log/rabbitmq/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2_upgrade.log database dir : /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2 2025-02-06 00:47:17.012 [info] <0.253.0> Running boot step pre_boot defined by app rabbit 2025-02-06 00:47:17.012 [info] <0.253.0> Running boot step rabbit_core_metrics defined by app rabbit 2025-02-06 00:47:17.013 [info] <0.253.0> Running boot step rabbit_alarm defined by app rabbit 2025-02-06 00:47:17.017 [info] <0.259.0> Memory high watermark set to 12637 MiB (13251077734 bytes) of 15796 MiB (16563847168 bytes) total 2025-02-06 00:47:17.021 [info] <0.261.0> Enabling free disk space monitoring 2025-02-06 00:47:17.021 [info] <0.261.0> Disk free limit set to 50MB 2025-02-06 00:47:17.024 [info] <0.253.0> Running boot step code_server_cache defined by app rabbit 2025-02-06 00:47:17.024 [info] <0.253.0> Running boot step file_handle_cache defined by app rabbit 2025-02-06 00:47:17.025 [info] <0.264.0> Limiting to approx 65436 file handles (58890 sockets) 2025-02-06 00:47:17.025 [info] <0.265.0> FHC read buffering: OFF 2025-02-06 00:47:17.025 [info] <0.265.0> FHC write buffering: ON 2025-02-06 00:47:17.025 [info] <0.253.0> Running boot step worker_pool defined by app rabbit 2025-02-06 00:47:17.025 [info] <0.254.0> Will use 4 processes for default worker pool 2025-02-06 00:47:17.025 [info] <0.254.0> Starting worker pool 'worker_pool' with 4 processes in it 2025-02-06 00:47:17.025 [info] <0.253.0> Running boot step database defined by app rabbit 2025-02-06 00:47:17.025 [info] <0.253.0> Node database directory at /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2 is empty. Assuming we need to join an existing cluster or initialise from scratch... 2025-02-06 00:47:17.025 [info] <0.253.0> Configured peer discovery backend: rabbit_peer_discovery_classic_config 2025-02-06 00:47:17.025 [info] <0.253.0> Will try to lock with peer discovery backend rabbit_peer_discovery_classic_config 2025-02-06 00:47:17.025 [info] <0.253.0> Peer discovery backend does not support locking, falling back to randomized delay 2025-02-06 00:47:17.025 [info] <0.253.0> Peer discovery backend rabbit_peer_discovery_classic_config supports registration. 2025-02-06 00:47:17.025 [info] <0.253.0> Will wait for 43643 milliseconds before proceeding with registration... 2025-02-06 00:48:00.669 [info] <0.253.0> All discovered existing cluster peers: contrail@cn-jenkins-deploy-platform-ansible-os-2696-1, contrail@cn-jenkins-deploy-platform-ansible-os-2696-2, contrail@cn-jenkins-deploy-platform-ansible-os-2696-3 2025-02-06 00:48:00.669 [info] <0.253.0> Peer nodes we can cluster with: contrail@cn-jenkins-deploy-platform-ansible-os-2696-1, contrail@cn-jenkins-deploy-platform-ansible-os-2696-3 2025-02-06 00:48:00.671 [info] <0.253.0> Node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-1' selected for auto-clustering 2025-02-06 00:48:00.698 [info] <0.253.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-02-06 00:48:00.699 [info] <0.253.0> Successfully synced tables from a peer 2025-02-06 00:48:01.002 [info] <0.253.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-02-06 00:48:01.002 [info] <0.253.0> Successfully synced tables from a peer 2025-02-06 00:48:01.043 [info] <0.253.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-02-06 00:48:01.043 [info] <0.253.0> Successfully synced tables from a peer 2025-02-06 00:48:01.052 [info] <0.253.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2696-2' 2025-02-06 00:48:01.058 [info] <0.253.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2696-2' 2025-02-06 00:48:01.058 [info] <0.253.0> Will register with peer discovery backend rabbit_peer_discovery_classic_config 2025-02-06 00:48:01.058 [info] <0.253.0> Running boot step database_sync defined by app rabbit 2025-02-06 00:48:01.058 [info] <0.253.0> Running boot step feature_flags defined by app rabbit 2025-02-06 00:48:01.058 [info] <0.253.0> Running boot step codec_correctness_check defined by app rabbit 2025-02-06 00:48:01.058 [info] <0.253.0> Running boot step external_infrastructure defined by app rabbit 2025-02-06 00:48:01.058 [info] <0.253.0> Running boot step rabbit_registry defined by app rabbit 2025-02-06 00:48:01.059 [info] <0.253.0> Running boot step rabbit_auth_mechanism_cr_demo defined by app rabbit 2025-02-06 00:48:01.059 [info] <0.253.0> Running boot step rabbit_queue_location_random defined by app rabbit 2025-02-06 00:48:01.059 [info] <0.253.0> Running boot step rabbit_event defined by app rabbit 2025-02-06 00:48:01.059 [info] <0.253.0> Running boot step rabbit_auth_mechanism_amqplain defined by app rabbit 2025-02-06 00:48:01.059 [info] <0.253.0> Running boot step rabbit_auth_mechanism_plain defined by app rabbit 2025-02-06 00:48:01.059 [info] <0.253.0> Running boot step rabbit_exchange_type_direct defined by app rabbit 2025-02-06 00:48:01.059 [info] <0.253.0> Running boot step rabbit_exchange_type_fanout defined by app rabbit 2025-02-06 00:48:01.059 [info] <0.253.0> Running boot step rabbit_exchange_type_headers defined by app rabbit 2025-02-06 00:48:01.059 [info] <0.253.0> Running boot step rabbit_exchange_type_topic defined by app rabbit 2025-02-06 00:48:01.059 [info] <0.253.0> Running boot step rabbit_mirror_queue_mode_all defined by app rabbit 2025-02-06 00:48:01.059 [info] <0.253.0> Running boot step rabbit_mirror_queue_mode_exactly defined by app rabbit 2025-02-06 00:48:01.059 [info] <0.253.0> Running boot step rabbit_mirror_queue_mode_nodes defined by app rabbit 2025-02-06 00:48:01.060 [info] <0.253.0> Running boot step rabbit_priority_queue defined by app rabbit 2025-02-06 00:48:01.060 [info] <0.253.0> Priority queues enabled, real BQ is rabbit_variable_queue 2025-02-06 00:48:01.060 [info] <0.253.0> Running boot step rabbit_queue_location_client_local defined by app rabbit 2025-02-06 00:48:01.060 [info] <0.253.0> Running boot step rabbit_queue_location_min_masters defined by app rabbit 2025-02-06 00:48:01.060 [info] <0.253.0> Running boot step kernel_ready defined by app rabbit 2025-02-06 00:48:01.060 [info] <0.253.0> Running boot step rabbit_sysmon_minder defined by app rabbit 2025-02-06 00:48:01.060 [info] <0.253.0> Running boot step rabbit_epmd_monitor defined by app rabbit 2025-02-06 00:48:01.061 [info] <0.470.0> epmd monitor knows us, inter-node communication (distribution) port: 25673 2025-02-06 00:48:01.061 [info] <0.253.0> Running boot step guid_generator defined by app rabbit 2025-02-06 00:48:01.064 [info] <0.253.0> Running boot step rabbit_node_monitor defined by app rabbit 2025-02-06 00:48:01.064 [info] <0.474.0> Starting rabbit_node_monitor 2025-02-06 00:48:01.065 [info] <0.253.0> Running boot step delegate_sup defined by app rabbit 2025-02-06 00:48:01.065 [info] <0.253.0> Running boot step rabbit_memory_monitor defined by app rabbit 2025-02-06 00:48:01.065 [info] <0.253.0> Running boot step core_initialized defined by app rabbit 2025-02-06 00:48:01.065 [info] <0.253.0> Running boot step upgrade_queues defined by app rabbit 2025-02-06 00:48:01.087 [info] <0.253.0> message_store upgrades: 1 to apply 2025-02-06 00:48:01.087 [info] <0.253.0> message_store upgrades: Applying rabbit_variable_queue:move_messages_to_vhost_store 2025-02-06 00:48:01.087 [info] <0.253.0> message_store upgrades: No durable queues found. Skipping message store migration 2025-02-06 00:48:01.087 [info] <0.253.0> message_store upgrades: Removing the old message store data 2025-02-06 00:48:01.088 [info] <0.253.0> message_store upgrades: All upgrades applied successfully 2025-02-06 00:48:01.111 [info] <0.253.0> Running boot step rabbit_connection_tracking defined by app rabbit 2025-02-06 00:48:01.111 [info] <0.253.0> Running boot step rabbit_connection_tracking_handler defined by app rabbit 2025-02-06 00:48:01.111 [info] <0.253.0> Running boot step rabbit_exchange_parameters defined by app rabbit 2025-02-06 00:48:01.111 [info] <0.253.0> Running boot step rabbit_mirror_queue_misc defined by app rabbit 2025-02-06 00:48:01.111 [info] <0.253.0> Running boot step rabbit_policies defined by app rabbit 2025-02-06 00:48:01.112 [info] <0.253.0> Running boot step rabbit_policy defined by app rabbit 2025-02-06 00:48:01.112 [info] <0.253.0> Running boot step rabbit_queue_location_validator defined by app rabbit 2025-02-06 00:48:01.112 [info] <0.253.0> Running boot step rabbit_vhost_limit defined by app rabbit 2025-02-06 00:48:01.112 [info] <0.253.0> Running boot step rabbit_mgmt_reset_handler defined by app rabbitmq_management 2025-02-06 00:48:01.112 [info] <0.253.0> Running boot step rabbit_mgmt_db_handler defined by app rabbitmq_management_agent 2025-02-06 00:48:01.112 [warning] <0.253.0> force_fine_statistics set to true; ignored. Replaced by {rates_mode, none} in the rabbitmq_management application. 2025-02-06 00:48:01.112 [info] <0.253.0> Management plugin: using rates mode 'basic' 2025-02-06 00:48:01.117 [info] <0.253.0> Running boot step recovery defined by app rabbit 2025-02-06 00:48:01.121 [info] <0.511.0> Making sure data directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists 2025-02-06 00:48:01.128 [info] <0.511.0> Starting message stores for vhost '/' 2025-02-06 00:48:01.128 [info] <0.515.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index 2025-02-06 00:48:01.130 [info] <0.511.0> Started message store of type transient for vhost '/' 2025-02-06 00:48:01.130 [info] <0.518.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index 2025-02-06 00:48:01.132 [warning] <0.518.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": rebuilding indices from scratch 2025-02-06 00:48:01.133 [info] <0.511.0> Started message store of type persistent for vhost '/' 2025-02-06 00:48:01.135 [info] <0.253.0> Running boot step load_definitions defined by app rabbitmq_management 2025-02-06 00:48:01.135 [info] <0.253.0> Applying definitions from /etc/rabbitmq/definitions.json 2025-02-06 00:48:01.135 [info] <0.253.0> Asked to import definitions. Acting user: rmq-internal 2025-02-06 00:48:01.136 [info] <0.253.0> Importing users... 2025-02-06 00:48:01.145 [info] <0.253.0> Setting user tags for user 'guest' to [administrator] 2025-02-06 00:48:01.151 [info] <0.253.0> Importing vhosts... 2025-02-06 00:48:01.151 [info] <0.253.0> Importing user permissions... 2025-02-06 00:48:01.151 [info] <0.253.0> Setting permissions for 'guest' in '/' to '.*', '.*', '.*' 2025-02-06 00:48:01.153 [info] <0.253.0> Importing topic permissions... 2025-02-06 00:48:01.153 [info] <0.253.0> Importing parameters... 2025-02-06 00:48:01.153 [info] <0.253.0> Importing global parameters... 2025-02-06 00:48:01.154 [info] <0.253.0> Importing policies... 2025-02-06 00:48:01.157 [info] <0.253.0> Importing queues... 2025-02-06 00:48:01.157 [info] <0.253.0> Importing exchanges... 2025-02-06 00:48:01.157 [info] <0.253.0> Importing bindings... 2025-02-06 00:48:01.157 [info] <0.253.0> Running boot step empty_db_check defined by app rabbit 2025-02-06 00:48:01.157 [info] <0.253.0> Running boot step rabbit_looking_glass defined by app rabbit 2025-02-06 00:48:01.157 [info] <0.253.0> Running boot step rabbit_core_metrics_gc defined by app rabbit 2025-02-06 00:48:01.157 [info] <0.253.0> Running boot step background_gc defined by app rabbit 2025-02-06 00:48:01.157 [info] <0.253.0> Running boot step connection_tracking defined by app rabbit 2025-02-06 00:48:01.160 [info] <0.253.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2696-2' 2025-02-06 00:48:01.162 [info] <0.253.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2696-2' 2025-02-06 00:48:01.162 [info] <0.253.0> Running boot step routing_ready defined by app rabbit 2025-02-06 00:48:01.162 [info] <0.253.0> Running boot step pre_flight defined by app rabbit 2025-02-06 00:48:01.162 [info] <0.253.0> Running boot step notify_cluster defined by app rabbit 2025-02-06 00:48:01.162 [info] <0.253.0> Running boot step networking defined by app rabbit 2025-02-06 00:48:01.163 [info] <0.474.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-3' up 2025-02-06 00:48:01.165 [info] <0.559.0> started TCP listener on [::]:5673 2025-02-06 00:48:01.165 [info] <0.253.0> Running boot step cluster_name defined by app rabbit 2025-02-06 00:48:01.165 [info] <0.253.0> Running boot step direct_client defined by app rabbit 2025-02-06 00:48:01.165 [info] <0.253.0> Running boot step os_signal_handler defined by app rabbit 2025-02-06 00:48:01.165 [info] <0.561.0> Swapping OS signal event handler (erl_signal_server) for our own 2025-02-06 00:48:01.173 [info] <0.474.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-1' up 2025-02-06 00:48:01.210 [info] <0.613.0> Management plugin: HTTP (non-TLS) listener started on port 15673 2025-02-06 00:48:01.211 [info] <0.719.0> Statistics database started. 2025-02-06 00:48:01.211 [info] <0.718.0> Starting worker pool 'management_worker_pool' with 3 processes in it 2025-02-06 00:48:01.211 [notice] <0.106.0> Changed loghwm of /var/log/contrail/config-database-rabbitmq/rabbitmq.log to 50 2025-02-06 00:48:01.322 [info] <0.8.0> Server startup complete; 3 plugins started. * rabbitmq_management * rabbitmq_management_agent * rabbitmq_web_dispatch 2025-02-06 00:48:02.157 [info] <0.743.0> RabbitMQ is asked to stop... 2025-02-06 00:48:02.228 [info] <0.743.0> Stopping RabbitMQ applications and their dependencies in the following order: rabbitmq_management amqp_client rabbitmq_web_dispatch cowboy cowlib rabbitmq_management_agent rabbit mnesia rabbit_common sysmon_handler os_mon 2025-02-06 00:48:02.228 [info] <0.743.0> Stopping application 'rabbitmq_management' 2025-02-06 00:48:02.230 [warning] <0.567.0> RabbitMQ HTTP listener registry could not find context rabbitmq_management_tls 2025-02-06 00:48:02.231 [info] <0.743.0> Stopping application 'amqp_client' 2025-02-06 00:48:02.231 [info] <0.43.0> Application rabbitmq_management exited with reason: stopped 2025-02-06 00:48:02.233 [info] <0.743.0> Stopping application 'rabbitmq_web_dispatch' 2025-02-06 00:48:02.233 [info] <0.43.0> Application amqp_client exited with reason: stopped 2025-02-06 00:48:02.234 [info] <0.43.0> Application rabbitmq_web_dispatch exited with reason: stopped 2025-02-06 00:48:02.234 [info] <0.743.0> Stopping application 'cowboy' 2025-02-06 00:48:02.235 [info] <0.743.0> Stopping application 'cowlib' 2025-02-06 00:48:02.235 [info] <0.43.0> Application cowboy exited with reason: stopped 2025-02-06 00:48:02.235 [info] <0.43.0> Application cowlib exited with reason: stopped 2025-02-06 00:48:02.235 [info] <0.743.0> Stopping application 'rabbitmq_management_agent' 2025-02-06 00:48:02.236 [info] <0.743.0> Stopping application 'rabbit' 2025-02-06 00:48:02.236 [info] <0.43.0> Application rabbitmq_management_agent exited with reason: stopped 2025-02-06 00:48:02.236 [info] <0.253.0> Will unregister with peer discovery backend rabbit_peer_discovery_classic_config 2025-02-06 00:48:02.236 [info] <0.559.0> stopped TCP listener on [::]:5673 2025-02-06 00:48:02.237 [info] <0.499.0> Closing all connections in vhost '/' on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2' because the vhost is stopping 2025-02-06 00:48:02.237 [info] <0.518.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' 2025-02-06 00:48:02.241 [info] <0.518.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' is stopped 2025-02-06 00:48:02.241 [info] <0.515.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' 2025-02-06 00:48:02.245 [info] <0.515.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' is stopped 2025-02-06 00:48:02.247 [info] <0.43.0> Application rabbit exited with reason: stopped 2025-02-06 00:48:02.247 [info] <0.743.0> Stopping application 'mnesia' 2025-02-06 00:48:02.249 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-02-06 00:48:02.249 [info] <0.743.0> Stopping application 'rabbit_common' 2025-02-06 00:48:02.249 [info] <0.43.0> Application rabbit_common exited with reason: stopped 2025-02-06 00:48:02.249 [info] <0.743.0> Stopping application 'sysmon_handler' 2025-02-06 00:48:02.250 [info] <0.43.0> Application sysmon_handler exited with reason: stopped 2025-02-06 00:48:02.250 [info] <0.743.0> Stopping application 'os_mon' 2025-02-06 00:48:02.251 [info] <0.43.0> Application os_mon exited with reason: stopped 2025-02-06 00:48:02.251 [info] <0.743.0> Successfully stopped RabbitMQ and its dependencies 2025-02-06 00:48:02.251 [info] <0.743.0> Halting Erlang VM with the following applications: ranch ssl public_key asn1 observer_cli tools credentials_obfuscation crypto stdout_formatter recon inets jsx xmerl lager goldrush compiler syntax_tools sasl stdlib kernel 2025-02-06 00:48:06.105 [info] <0.8.0> Log file opened with Lager 2025-02-06 00:48:07.562 [info] <0.8.0> Feature flags: list of feature flags found: 2025-02-06 00:48:07.562 [info] <0.8.0> Feature flags: feature flag states written to disk: yes 2025-02-06 00:48:07.631 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-02-06 00:48:07.670 [info] <0.305.0> Starting RabbitMQ 3.7.28 on Erlang 21.3.8.21 Copyright (c) 2007-2020 Pivotal Software, Inc. Licensed under the MPL. See https://www.rabbitmq.com/ 2025-02-06 00:48:07.671 [info] <0.305.0> node : contrail@cn-jenkins-deploy-platform-ansible-os-2696-2 home dir : /var/lib/rabbitmq config file(s) : /etc/rabbitmq/rabbitmq.config cookie hash : esZsDxSN6VGbi9JkMSxNZA== log(s) : /var/log/contrail/config-database-rabbitmq/rabbitmq.log : /var/log/rabbitmq/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2_upgrade.log database dir : /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2 2025-02-06 00:48:07.682 [info] <0.305.0> Running boot step pre_boot defined by app rabbit 2025-02-06 00:48:07.682 [info] <0.305.0> Running boot step rabbit_core_metrics defined by app rabbit 2025-02-06 00:48:07.683 [info] <0.305.0> Running boot step rabbit_alarm defined by app rabbit 2025-02-06 00:48:07.689 [info] <0.330.0> Memory high watermark set to 12637 MiB (13251077734 bytes) of 15796 MiB (16563847168 bytes) total 2025-02-06 00:48:07.695 [info] <0.344.0> Enabling free disk space monitoring 2025-02-06 00:48:07.695 [info] <0.344.0> Disk free limit set to 50MB 2025-02-06 00:48:07.701 [info] <0.305.0> Running boot step code_server_cache defined by app rabbit 2025-02-06 00:48:07.701 [info] <0.305.0> Running boot step file_handle_cache defined by app rabbit 2025-02-06 00:48:07.702 [info] <0.370.0> Limiting to approx 65436 file handles (58890 sockets) 2025-02-06 00:48:07.702 [info] <0.372.0> FHC read buffering: OFF 2025-02-06 00:48:07.702 [info] <0.372.0> FHC write buffering: ON 2025-02-06 00:48:07.702 [info] <0.305.0> Running boot step worker_pool defined by app rabbit 2025-02-06 00:48:07.702 [info] <0.306.0> Will use 4 processes for default worker pool 2025-02-06 00:48:07.702 [info] <0.306.0> Starting worker pool 'worker_pool' with 4 processes in it 2025-02-06 00:48:07.702 [info] <0.305.0> Running boot step database defined by app rabbit 2025-02-06 00:48:07.729 [info] <0.305.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-02-06 00:48:07.731 [info] <0.305.0> Successfully synced tables from a peer 2025-02-06 00:48:07.732 [info] <0.305.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-02-06 00:48:07.732 [info] <0.305.0> Successfully synced tables from a peer 2025-02-06 00:48:07.758 [info] <0.305.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-02-06 00:48:07.758 [info] <0.305.0> Successfully synced tables from a peer 2025-02-06 00:48:07.758 [info] <0.305.0> Will register with peer discovery backend rabbit_peer_discovery_classic_config 2025-02-06 00:48:07.758 [info] <0.305.0> Running boot step database_sync defined by app rabbit 2025-02-06 00:48:07.758 [info] <0.305.0> Running boot step feature_flags defined by app rabbit 2025-02-06 00:48:07.758 [info] <0.305.0> Running boot step codec_correctness_check defined by app rabbit 2025-02-06 00:48:07.758 [info] <0.305.0> Running boot step external_infrastructure defined by app rabbit 2025-02-06 00:48:07.758 [info] <0.305.0> Running boot step rabbit_registry defined by app rabbit 2025-02-06 00:48:07.758 [info] <0.305.0> Running boot step rabbit_auth_mechanism_cr_demo defined by app rabbit 2025-02-06 00:48:07.758 [info] <0.305.0> Running boot step rabbit_queue_location_random defined by app rabbit 2025-02-06 00:48:07.758 [info] <0.305.0> Running boot step rabbit_event defined by app rabbit 2025-02-06 00:48:07.758 [info] <0.305.0> Running boot step rabbit_auth_mechanism_amqplain defined by app rabbit 2025-02-06 00:48:07.758 [info] <0.305.0> Running boot step rabbit_auth_mechanism_plain defined by app rabbit 2025-02-06 00:48:07.759 [info] <0.305.0> Running boot step rabbit_exchange_type_direct defined by app rabbit 2025-02-06 00:48:07.759 [info] <0.305.0> Running boot step rabbit_exchange_type_fanout defined by app rabbit 2025-02-06 00:48:07.759 [info] <0.305.0> Running boot step rabbit_exchange_type_headers defined by app rabbit 2025-02-06 00:48:07.759 [info] <0.305.0> Running boot step rabbit_exchange_type_topic defined by app rabbit 2025-02-06 00:48:07.759 [info] <0.305.0> Running boot step rabbit_mirror_queue_mode_all defined by app rabbit 2025-02-06 00:48:07.759 [info] <0.305.0> Running boot step rabbit_mirror_queue_mode_exactly defined by app rabbit 2025-02-06 00:48:07.759 [info] <0.305.0> Running boot step rabbit_mirror_queue_mode_nodes defined by app rabbit 2025-02-06 00:48:07.759 [info] <0.305.0> Running boot step rabbit_priority_queue defined by app rabbit 2025-02-06 00:48:07.759 [info] <0.305.0> Priority queues enabled, real BQ is rabbit_variable_queue 2025-02-06 00:48:07.759 [info] <0.305.0> Running boot step rabbit_queue_location_client_local defined by app rabbit 2025-02-06 00:48:07.759 [info] <0.305.0> Running boot step rabbit_queue_location_min_masters defined by app rabbit 2025-02-06 00:48:07.759 [info] <0.305.0> Running boot step kernel_ready defined by app rabbit 2025-02-06 00:48:07.759 [info] <0.305.0> Running boot step rabbit_sysmon_minder defined by app rabbit 2025-02-06 00:48:07.759 [info] <0.305.0> Running boot step rabbit_epmd_monitor defined by app rabbit 2025-02-06 00:48:07.760 [info] <0.414.0> epmd monitor knows us, inter-node communication (distribution) port: 25673 2025-02-06 00:48:07.760 [info] <0.305.0> Running boot step guid_generator defined by app rabbit 2025-02-06 00:48:07.763 [info] <0.305.0> Running boot step rabbit_node_monitor defined by app rabbit 2025-02-06 00:48:07.777 [info] <0.418.0> Starting rabbit_node_monitor 2025-02-06 00:48:07.777 [info] <0.305.0> Running boot step delegate_sup defined by app rabbit 2025-02-06 00:48:07.777 [info] <0.305.0> Running boot step rabbit_memory_monitor defined by app rabbit 2025-02-06 00:48:07.778 [info] <0.305.0> Running boot step core_initialized defined by app rabbit 2025-02-06 00:48:07.778 [info] <0.305.0> Running boot step upgrade_queues defined by app rabbit 2025-02-06 00:48:07.797 [info] <0.305.0> Running boot step rabbit_connection_tracking defined by app rabbit 2025-02-06 00:48:07.797 [info] <0.305.0> Running boot step rabbit_connection_tracking_handler defined by app rabbit 2025-02-06 00:48:07.797 [info] <0.305.0> Running boot step rabbit_exchange_parameters defined by app rabbit 2025-02-06 00:48:07.797 [info] <0.305.0> Running boot step rabbit_mirror_queue_misc defined by app rabbit 2025-02-06 00:48:07.797 [info] <0.305.0> Running boot step rabbit_policies defined by app rabbit 2025-02-06 00:48:07.798 [info] <0.305.0> Running boot step rabbit_policy defined by app rabbit 2025-02-06 00:48:07.798 [info] <0.305.0> Running boot step rabbit_queue_location_validator defined by app rabbit 2025-02-06 00:48:07.798 [info] <0.305.0> Running boot step rabbit_vhost_limit defined by app rabbit 2025-02-06 00:48:07.798 [info] <0.305.0> Running boot step rabbit_mgmt_reset_handler defined by app rabbitmq_management 2025-02-06 00:48:07.798 [info] <0.305.0> Running boot step rabbit_mgmt_db_handler defined by app rabbitmq_management_agent 2025-02-06 00:48:07.798 [warning] <0.305.0> force_fine_statistics set to true; ignored. Replaced by {rates_mode, none} in the rabbitmq_management application. 2025-02-06 00:48:07.798 [info] <0.305.0> Management plugin: using rates mode 'basic' 2025-02-06 00:48:07.799 [info] <0.305.0> Running boot step recovery defined by app rabbit 2025-02-06 00:48:07.800 [info] <0.455.0> Making sure data directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists 2025-02-06 00:48:07.805 [info] <0.455.0> Starting message stores for vhost '/' 2025-02-06 00:48:07.817 [info] <0.459.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index 2025-02-06 00:48:07.826 [info] <0.455.0> Started message store of type transient for vhost '/' 2025-02-06 00:48:07.826 [info] <0.462.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index 2025-02-06 00:48:07.827 [info] <0.455.0> Started message store of type persistent for vhost '/' 2025-02-06 00:48:07.828 [info] <0.305.0> Running boot step load_definitions defined by app rabbitmq_management 2025-02-06 00:48:07.828 [info] <0.305.0> Applying definitions from /etc/rabbitmq/definitions.json 2025-02-06 00:48:07.828 [info] <0.305.0> Asked to import definitions. Acting user: rmq-internal 2025-02-06 00:48:07.828 [info] <0.305.0> Importing users... 2025-02-06 00:48:07.840 [info] <0.305.0> Setting user tags for user 'guest' to [administrator] 2025-02-06 00:48:07.843 [info] <0.305.0> Importing vhosts... 2025-02-06 00:48:07.843 [info] <0.305.0> Importing user permissions... 2025-02-06 00:48:07.843 [info] <0.305.0> Setting permissions for 'guest' in '/' to '.*', '.*', '.*' 2025-02-06 00:48:07.869 [info] <0.305.0> Importing topic permissions... 2025-02-06 00:48:07.870 [info] <0.305.0> Importing parameters... 2025-02-06 00:48:07.870 [info] <0.305.0> Importing global parameters... 2025-02-06 00:48:07.870 [info] <0.305.0> Importing policies... 2025-02-06 00:48:07.872 [info] <0.305.0> Importing queues... 2025-02-06 00:48:07.873 [info] <0.305.0> Importing exchanges... 2025-02-06 00:48:07.873 [info] <0.305.0> Importing bindings... 2025-02-06 00:48:07.873 [info] <0.305.0> Running boot step empty_db_check defined by app rabbit 2025-02-06 00:48:07.873 [info] <0.305.0> Running boot step rabbit_looking_glass defined by app rabbit 2025-02-06 00:48:07.873 [info] <0.305.0> Running boot step rabbit_core_metrics_gc defined by app rabbit 2025-02-06 00:48:07.873 [info] <0.305.0> Running boot step background_gc defined by app rabbit 2025-02-06 00:48:07.873 [info] <0.305.0> Running boot step connection_tracking defined by app rabbit 2025-02-06 00:48:07.873 [info] <0.305.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2696-2' 2025-02-06 00:48:07.874 [info] <0.305.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2696-2' 2025-02-06 00:48:07.874 [info] <0.305.0> Running boot step routing_ready defined by app rabbit 2025-02-06 00:48:07.874 [info] <0.305.0> Running boot step pre_flight defined by app rabbit 2025-02-06 00:48:07.875 [info] <0.305.0> Running boot step notify_cluster defined by app rabbit 2025-02-06 00:48:07.875 [info] <0.305.0> Running boot step networking defined by app rabbit 2025-02-06 00:48:07.875 [info] <0.418.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-1' up 2025-02-06 00:48:07.876 [info] <0.503.0> started TCP listener on [::]:5673 2025-02-06 00:48:07.877 [info] <0.305.0> Running boot step cluster_name defined by app rabbit 2025-02-06 00:48:07.877 [info] <0.305.0> Running boot step direct_client defined by app rabbit 2025-02-06 00:48:07.877 [info] <0.305.0> Running boot step os_signal_handler defined by app rabbit 2025-02-06 00:48:07.877 [info] <0.505.0> Swapping OS signal event handler (erl_signal_server) for our own 2025-02-06 00:48:07.882 [info] <0.418.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-3' up 2025-02-06 00:48:07.910 [info] <0.557.0> Management plugin: HTTP (non-TLS) listener started on port 15673 2025-02-06 00:48:07.910 [info] <0.663.0> Statistics database started. 2025-02-06 00:48:07.911 [info] <0.662.0> Starting worker pool 'management_worker_pool' with 3 processes in it 2025-02-06 00:48:07.911 [notice] <0.106.0> Changed loghwm of /var/log/contrail/config-database-rabbitmq/rabbitmq.log to 50 2025-02-06 00:48:07.978 [info] <0.8.0> Server startup complete; 3 plugins started. * rabbitmq_management * rabbitmq_management_agent * rabbitmq_web_dispatch 2025-02-06 01:50:41.705 [info] <0.418.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-3' down 2025-02-06 01:50:41.716 [info] <0.418.0> Keeping contrail@cn-jenkins-deploy-platform-ansible-os-2696-3 listeners: the node is already back 2025-02-06 01:50:41.728 [info] <0.418.0> node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-3' down: connection_closed 2025-02-06 01:50:41.790 [info] <0.1271.0> RabbitMQ is asked to stop... 2025-02-06 01:50:41.831 [info] <0.1071.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2696-2.' in vhost '/': Slave saw deaths of mirrors 2025-02-06 01:50:41.831 [info] <0.749.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-3..tftp_queue' in vhost '/': Slave saw deaths of mirrors 2025-02-06 01:50:41.832 [info] <0.1062.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Slave saw deaths of mirrors 2025-02-06 01:50:41.833 [info] <0.903.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2696-1..dummy' in vhost '/': Slave saw deaths of mirrors 2025-02-06 01:50:41.834 [info] <0.1132.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-3.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Slave saw deaths of mirrors 2025-02-06 01:50:41.834 [info] <0.738.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-3..config_queue' in vhost '/': Slave saw deaths of mirrors 2025-02-06 01:50:41.834 [info] <0.1143.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-3.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Slave saw deaths of mirrors 2025-02-06 01:50:41.835 [info] <0.1177.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-1.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Slave saw deaths of mirrors 2025-02-06 01:50:41.836 [info] <0.952.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2696-2' in vhost '/': Slave saw deaths of mirrors 2025-02-06 01:50:41.836 [info] <0.760.0> Mirrored queue 'device_manager_ztp.ztp_queue' in vhost '/': Slave saw deaths of mirrors 2025-02-06 01:50:41.837 [info] <0.931.0> Mirrored queue 'device_manager.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Slave saw deaths of mirrors 2025-02-06 01:50:41.847 [info] <0.1025.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2696-3' in vhost '/': Slave saw deaths of mirrors 2025-02-06 01:50:41.854 [info] <0.880.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2696-1.-8082-0' in vhost '/': Slave saw deaths of mirrors 2025-02-06 01:50:41.854 [info] <0.1037.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2696-1' in vhost '/': Slave saw deaths of mirrors 2025-02-06 01:50:41.854 [info] <0.974.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2696-3' in vhost '/': Slave saw deaths of mirrors 2025-02-06 01:50:41.855 [info] <0.1165.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-2.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2696-2.' in vhost '/': Slave saw deaths of mirrors 2025-02-06 01:50:41.865 [info] <0.1188.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-1.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Slave saw deaths of mirrors 2025-02-06 01:50:41.873 [info] <0.1271.0> Stopping RabbitMQ applications and their dependencies in the following order: rabbitmq_management amqp_client rabbitmq_web_dispatch cowboy cowlib rabbitmq_management_agent rabbit mnesia rabbit_common sysmon_handler os_mon 2025-02-06 01:50:41.873 [info] <0.1271.0> Stopping application 'rabbitmq_management' 2025-02-06 01:50:41.875 [warning] <0.511.0> RabbitMQ HTTP listener registry could not find context rabbitmq_management_tls 2025-02-06 01:50:41.876 [info] <0.1271.0> Stopping application 'amqp_client' 2025-02-06 01:50:41.876 [info] <0.43.0> Application rabbitmq_management exited with reason: stopped 2025-02-06 01:50:41.878 [info] <0.1271.0> Stopping application 'rabbitmq_web_dispatch' 2025-02-06 01:50:41.878 [info] <0.43.0> Application amqp_client exited with reason: stopped 2025-02-06 01:50:41.879 [info] <0.43.0> Application rabbitmq_web_dispatch exited with reason: stopped 2025-02-06 01:50:41.879 [info] <0.1271.0> Stopping application 'cowboy' 2025-02-06 01:50:41.880 [info] <0.1271.0> Stopping application 'cowlib' 2025-02-06 01:50:41.880 [info] <0.43.0> Application cowboy exited with reason: stopped 2025-02-06 01:50:41.880 [info] <0.1271.0> Stopping application 'rabbitmq_management_agent' 2025-02-06 01:50:41.880 [info] <0.43.0> Application cowlib exited with reason: stopped 2025-02-06 01:50:41.881 [info] <0.43.0> Application rabbitmq_management_agent exited with reason: stopped 2025-02-06 01:50:41.881 [info] <0.1271.0> Stopping application 'rabbit' 2025-02-06 01:50:41.881 [info] <0.305.0> Will unregister with peer discovery backend rabbit_peer_discovery_classic_config 2025-02-06 01:50:41.881 [info] <0.503.0> stopped TCP listener on [::]:5673 2025-02-06 01:50:41.882 [info] <0.443.0> Closing all connections in vhost '/' on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2' because the vhost is stopping 2025-02-06 01:50:41.883 [info] <0.462.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' 2025-02-06 01:50:41.887 [info] <0.462.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' is stopped 2025-02-06 01:50:41.887 [info] <0.459.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' 2025-02-06 01:50:41.890 [info] <0.459.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' is stopped 2025-02-06 01:50:41.896 [info] <0.43.0> Application rabbit exited with reason: stopped 2025-02-06 01:50:41.896 [info] <0.1271.0> Stopping application 'mnesia' 2025-02-06 01:50:41.897 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-02-06 01:50:41.897 [info] <0.1271.0> Stopping application 'rabbit_common' 2025-02-06 01:50:41.897 [info] <0.43.0> Application rabbit_common exited with reason: stopped 2025-02-06 01:50:41.897 [info] <0.1271.0> Stopping application 'sysmon_handler' 2025-02-06 01:50:41.899 [info] <0.1271.0> Stopping application 'os_mon' 2025-02-06 01:51:06.562 [info] <0.8.0> Log file opened with Lager 2025-02-06 01:51:10.600 [info] <0.8.0> Feature flags: list of feature flags found: 2025-02-06 01:51:10.600 [info] <0.8.0> Feature flags: feature flag states written to disk: yes 2025-02-06 01:51:10.716 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-02-06 01:51:10.770 [info] <0.308.0> Starting RabbitMQ 3.7.28 on Erlang 21.3.8.21 Copyright (c) 2007-2020 Pivotal Software, Inc. Licensed under the MPL. See https://www.rabbitmq.com/ 2025-02-06 01:51:10.772 [info] <0.308.0> node : contrail@cn-jenkins-deploy-platform-ansible-os-2696-2 home dir : /var/lib/rabbitmq config file(s) : /etc/rabbitmq/rabbitmq.config cookie hash : esZsDxSN6VGbi9JkMSxNZA== log(s) : /var/log/contrail/config-database-rabbitmq/rabbitmq.log : /var/log/rabbitmq/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2_upgrade.log database dir : /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2 2025-02-06 01:51:10.818 [info] <0.308.0> Running boot step pre_boot defined by app rabbit 2025-02-06 01:51:10.818 [info] <0.308.0> Running boot step rabbit_core_metrics defined by app rabbit 2025-02-06 01:51:10.820 [info] <0.308.0> Running boot step rabbit_alarm defined by app rabbit 2025-02-06 01:51:10.830 [info] <0.369.0> Memory high watermark set to 12637 MiB (13251077734 bytes) of 15796 MiB (16563847168 bytes) total 2025-02-06 01:51:10.859 [info] <0.376.0> Enabling free disk space monitoring 2025-02-06 01:51:10.860 [info] <0.376.0> Disk free limit set to 50MB 2025-02-06 01:51:10.866 [info] <0.308.0> Running boot step code_server_cache defined by app rabbit 2025-02-06 01:51:10.866 [info] <0.308.0> Running boot step file_handle_cache defined by app rabbit 2025-02-06 01:51:10.867 [info] <0.384.0> FHC read buffering: OFF 2025-02-06 01:51:10.869 [info] <0.384.0> FHC write buffering: ON 2025-02-06 01:51:10.867 [info] <0.383.0> Limiting to approx 65436 file handles (58890 sockets) 2025-02-06 01:51:10.872 [info] <0.308.0> Running boot step worker_pool defined by app rabbit 2025-02-06 01:51:10.872 [info] <0.311.0> Will use 4 processes for default worker pool 2025-02-06 01:51:10.872 [info] <0.311.0> Starting worker pool 'worker_pool' with 4 processes in it 2025-02-06 01:51:10.872 [info] <0.308.0> Running boot step database defined by app rabbit 2025-02-06 01:51:10.892 [info] <0.308.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-02-06 01:51:10.897 [info] <0.308.0> Successfully synced tables from a peer 2025-02-06 01:51:10.900 [info] <0.308.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-02-06 01:51:10.900 [info] <0.308.0> Successfully synced tables from a peer 2025-02-06 01:51:10.958 [info] <0.308.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-02-06 01:51:10.958 [info] <0.308.0> Successfully synced tables from a peer 2025-02-06 01:51:10.958 [info] <0.308.0> Will register with peer discovery backend rabbit_peer_discovery_classic_config 2025-02-06 01:51:10.958 [info] <0.308.0> Running boot step database_sync defined by app rabbit 2025-02-06 01:51:10.958 [info] <0.308.0> Running boot step feature_flags defined by app rabbit 2025-02-06 01:51:10.958 [info] <0.308.0> Running boot step codec_correctness_check defined by app rabbit 2025-02-06 01:51:10.958 [info] <0.308.0> Running boot step external_infrastructure defined by app rabbit 2025-02-06 01:51:10.958 [info] <0.308.0> Running boot step rabbit_registry defined by app rabbit 2025-02-06 01:51:10.958 [info] <0.308.0> Running boot step rabbit_auth_mechanism_cr_demo defined by app rabbit 2025-02-06 01:51:10.958 [info] <0.308.0> Running boot step rabbit_queue_location_random defined by app rabbit 2025-02-06 01:51:10.958 [info] <0.308.0> Running boot step rabbit_event defined by app rabbit 2025-02-06 01:51:10.958 [info] <0.308.0> Running boot step rabbit_auth_mechanism_amqplain defined by app rabbit 2025-02-06 01:51:10.959 [info] <0.308.0> Running boot step rabbit_auth_mechanism_plain defined by app rabbit 2025-02-06 01:51:10.959 [info] <0.308.0> Running boot step rabbit_exchange_type_direct defined by app rabbit 2025-02-06 01:51:10.959 [info] <0.308.0> Running boot step rabbit_exchange_type_fanout defined by app rabbit 2025-02-06 01:51:10.959 [info] <0.308.0> Running boot step rabbit_exchange_type_headers defined by app rabbit 2025-02-06 01:51:10.959 [info] <0.308.0> Running boot step rabbit_exchange_type_topic defined by app rabbit 2025-02-06 01:51:10.959 [info] <0.308.0> Running boot step rabbit_mirror_queue_mode_all defined by app rabbit 2025-02-06 01:51:10.959 [info] <0.308.0> Running boot step rabbit_mirror_queue_mode_exactly defined by app rabbit 2025-02-06 01:51:10.959 [info] <0.308.0> Running boot step rabbit_mirror_queue_mode_nodes defined by app rabbit 2025-02-06 01:51:10.959 [info] <0.308.0> Running boot step rabbit_priority_queue defined by app rabbit 2025-02-06 01:51:10.959 [info] <0.308.0> Priority queues enabled, real BQ is rabbit_variable_queue 2025-02-06 01:51:10.959 [info] <0.308.0> Running boot step rabbit_queue_location_client_local defined by app rabbit 2025-02-06 01:51:10.959 [info] <0.308.0> Running boot step rabbit_queue_location_min_masters defined by app rabbit 2025-02-06 01:51:10.959 [info] <0.308.0> Running boot step kernel_ready defined by app rabbit 2025-02-06 01:51:10.959 [info] <0.308.0> Running boot step rabbit_sysmon_minder defined by app rabbit 2025-02-06 01:51:10.959 [info] <0.308.0> Running boot step rabbit_epmd_monitor defined by app rabbit 2025-02-06 01:51:10.960 [info] <0.409.0> epmd monitor knows us, inter-node communication (distribution) port: 25673 2025-02-06 01:51:10.960 [info] <0.308.0> Running boot step guid_generator defined by app rabbit 2025-02-06 01:51:11.048 [info] <0.308.0> Running boot step rabbit_node_monitor defined by app rabbit 2025-02-06 01:51:11.056 [info] <0.413.0> Starting rabbit_node_monitor 2025-02-06 01:51:11.056 [info] <0.308.0> Running boot step delegate_sup defined by app rabbit 2025-02-06 01:51:11.057 [info] <0.308.0> Running boot step rabbit_memory_monitor defined by app rabbit 2025-02-06 01:51:11.057 [info] <0.308.0> Running boot step core_initialized defined by app rabbit 2025-02-06 01:51:11.057 [info] <0.308.0> Running boot step upgrade_queues defined by app rabbit 2025-02-06 01:51:11.084 [info] <0.308.0> Running boot step rabbit_connection_tracking defined by app rabbit 2025-02-06 01:51:11.084 [info] <0.308.0> Running boot step rabbit_connection_tracking_handler defined by app rabbit 2025-02-06 01:51:11.084 [info] <0.308.0> Running boot step rabbit_exchange_parameters defined by app rabbit 2025-02-06 01:51:11.084 [info] <0.308.0> Running boot step rabbit_mirror_queue_misc defined by app rabbit 2025-02-06 01:51:11.085 [info] <0.308.0> Running boot step rabbit_policies defined by app rabbit 2025-02-06 01:51:11.085 [info] <0.308.0> Running boot step rabbit_policy defined by app rabbit 2025-02-06 01:51:11.085 [info] <0.308.0> Running boot step rabbit_queue_location_validator defined by app rabbit 2025-02-06 01:51:11.085 [info] <0.308.0> Running boot step rabbit_vhost_limit defined by app rabbit 2025-02-06 01:51:11.085 [info] <0.308.0> Running boot step rabbit_mgmt_reset_handler defined by app rabbitmq_management 2025-02-06 01:51:11.085 [info] <0.308.0> Running boot step rabbit_mgmt_db_handler defined by app rabbitmq_management_agent 2025-02-06 01:51:11.085 [warning] <0.308.0> force_fine_statistics set to true; ignored. Replaced by {rates_mode, none} in the rabbitmq_management application. 2025-02-06 01:51:11.085 [info] <0.308.0> Management plugin: using rates mode 'basic' 2025-02-06 01:51:11.087 [info] <0.308.0> Running boot step recovery defined by app rabbit 2025-02-06 01:51:11.088 [info] <0.450.0> Making sure data directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists 2025-02-06 01:51:11.100 [info] <0.450.0> Starting message stores for vhost '/' 2025-02-06 01:51:11.100 [info] <0.454.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index 2025-02-06 01:51:11.101 [info] <0.450.0> Started message store of type transient for vhost '/' 2025-02-06 01:51:11.101 [info] <0.457.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index 2025-02-06 01:51:11.103 [info] <0.450.0> Started message store of type persistent for vhost '/' 2025-02-06 01:51:11.104 [info] <0.450.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-1.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.476.0> 2025-02-06 01:51:11.104 [info] <0.450.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-2.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2696-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.480.0> 2025-02-06 01:51:11.104 [info] <0.450.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2696-1' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.484.0> 2025-02-06 01:51:11.105 [info] <0.450.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-2.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2696-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.488.0> 2025-02-06 01:51:11.105 [info] <0.450.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2696-2' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.492.0> 2025-02-06 01:51:11.105 [info] <0.450.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2696-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.496.0> 2025-02-06 01:51:11.105 [info] <0.450.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.500.0> 2025-02-06 01:51:11.106 [info] <0.450.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.504.0> 2025-02-06 01:51:11.106 [info] <0.450.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-3..config_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.508.0> 2025-02-06 01:51:11.106 [info] <0.450.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2696-1..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.512.0> 2025-02-06 01:51:11.107 [info] <0.450.0> Mirrored queue 'job_request_consumer' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.516.0> 2025-02-06 01:51:11.108 [info] <0.450.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2696-3.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.520.0> 2025-02-06 01:51:11.108 [info] <0.450.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2696-1.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.524.0> 2025-02-06 01:51:11.108 [info] <0.450.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2696-3' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.528.0> 2025-02-06 01:51:11.108 [info] <0.450.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2696-2' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.532.0> 2025-02-06 01:51:11.109 [info] <0.450.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2696-2.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.536.0> 2025-02-06 01:51:11.109 [info] <0.450.0> Mirrored queue 'device_manager.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.540.0> 2025-02-06 01:51:11.109 [info] <0.450.0> Mirrored queue 'device_manager_ztp.ztp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.544.0> 2025-02-06 01:51:11.109 [info] <0.450.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-3.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.548.0> 2025-02-06 01:51:11.109 [info] <0.450.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-1.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.552.0> 2025-02-06 01:51:11.110 [info] <0.450.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2696-1' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.556.0> 2025-02-06 01:51:11.110 [info] <0.450.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-3..tftp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.560.0> 2025-02-06 01:51:11.110 [info] <0.450.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2696-3' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.564.0> 2025-02-06 01:51:11.110 [info] <0.450.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-1.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.568.0> 2025-02-06 01:51:11.110 [info] <0.450.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.572.0> 2025-02-06 01:51:11.111 [info] <0.450.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-3.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.576.0> 2025-02-06 01:51:11.121 [info] <0.308.0> Running boot step load_definitions defined by app rabbitmq_management 2025-02-06 01:51:11.122 [info] <0.308.0> Applying definitions from /etc/rabbitmq/definitions.json 2025-02-06 01:51:11.122 [info] <0.308.0> Asked to import definitions. Acting user: rmq-internal 2025-02-06 01:51:11.122 [info] <0.308.0> Importing users... 2025-02-06 01:51:11.135 [info] <0.308.0> Setting user tags for user 'guest' to [administrator] 2025-02-06 01:51:11.165 [info] <0.308.0> Importing vhosts... 2025-02-06 01:51:11.165 [info] <0.308.0> Importing user permissions... 2025-02-06 01:51:11.165 [info] <0.308.0> Setting permissions for 'guest' in '/' to '.*', '.*', '.*' 2025-02-06 01:51:11.188 [info] <0.308.0> Importing topic permissions... 2025-02-06 01:51:11.189 [info] <0.308.0> Importing parameters... 2025-02-06 01:51:11.189 [info] <0.308.0> Importing global parameters... 2025-02-06 01:51:11.189 [info] <0.308.0> Importing policies... 2025-02-06 01:51:11.199 [info] <0.308.0> Importing queues... 2025-02-06 01:51:11.199 [info] <0.308.0> Importing exchanges... 2025-02-06 01:51:11.199 [info] <0.308.0> Importing bindings... 2025-02-06 01:51:11.199 [info] <0.308.0> Running boot step empty_db_check defined by app rabbit 2025-02-06 01:51:11.199 [info] <0.308.0> Running boot step rabbit_looking_glass defined by app rabbit 2025-02-06 01:51:11.199 [info] <0.308.0> Running boot step rabbit_core_metrics_gc defined by app rabbit 2025-02-06 01:51:11.199 [info] <0.308.0> Running boot step background_gc defined by app rabbit 2025-02-06 01:51:11.200 [info] <0.308.0> Running boot step connection_tracking defined by app rabbit 2025-02-06 01:51:11.201 [info] <0.308.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2696-2' 2025-02-06 01:51:11.201 [info] <0.308.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2696-2' 2025-02-06 01:51:11.202 [info] <0.308.0> Running boot step routing_ready defined by app rabbit 2025-02-06 01:51:11.202 [info] <0.308.0> Running boot step pre_flight defined by app rabbit 2025-02-06 01:51:11.202 [info] <0.308.0> Running boot step notify_cluster defined by app rabbit 2025-02-06 01:51:11.202 [info] <0.308.0> Running boot step networking defined by app rabbit 2025-02-06 01:51:11.202 [info] <0.413.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-1' up 2025-02-06 01:51:11.204 [info] <0.308.0> Running boot step cluster_name defined by app rabbit 2025-02-06 01:51:11.204 [info] <0.308.0> Running boot step direct_client defined by app rabbit 2025-02-06 01:51:11.204 [info] <0.308.0> Running boot step os_signal_handler defined by app rabbit 2025-02-06 01:51:11.204 [info] <0.682.0> Swapping OS signal event handler (erl_signal_server) for our own 2025-02-06 01:51:11.204 [info] <0.680.0> started TCP listener on [::]:5673 2025-02-06 01:51:11.327 [info] <0.762.0> Management plugin: HTTP (non-TLS) listener started on port 15673 2025-02-06 01:51:11.327 [info] <0.872.0> Statistics database started. 2025-02-06 01:51:11.327 [info] <0.871.0> Starting worker pool 'management_worker_pool' with 3 processes in it 2025-02-06 01:51:11.327 [notice] <0.110.0> Changed loghwm of /var/log/contrail/config-database-rabbitmq/rabbitmq.log to 50 2025-02-06 01:51:11.866 [info] <0.413.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-3' up 2025-02-06 01:51:12.376 [info] <0.8.0> Server startup complete; 3 plugins started. * rabbitmq_management * rabbitmq_web_dispatch * rabbitmq_management_agent 2025-02-06 01:51:18.141 [info] <0.1068.0> RabbitMQ is asked to stop... 2025-02-06 01:51:18.400 [info] <0.1068.0> Stopping RabbitMQ applications and their dependencies in the following order: rabbitmq_management amqp_client rabbitmq_web_dispatch cowboy cowlib rabbitmq_management_agent rabbit mnesia rabbit_common sysmon_handler os_mon 2025-02-06 01:51:18.400 [info] <0.1068.0> Stopping application 'rabbitmq_management' 2025-02-06 01:51:18.401 [warning] <0.758.0> RabbitMQ HTTP listener registry could not find context rabbitmq_management_tls 2025-02-06 01:51:18.403 [info] <0.1068.0> Stopping application 'amqp_client' 2025-02-06 01:51:18.403 [info] <0.43.0> Application rabbitmq_management exited with reason: stopped 2025-02-06 01:51:18.404 [info] <0.43.0> Application amqp_client exited with reason: stopped 2025-02-06 01:51:18.404 [info] <0.1068.0> Stopping application 'rabbitmq_web_dispatch' 2025-02-06 01:51:18.411 [info] <0.1068.0> Stopping application 'cowboy' 2025-02-06 01:51:18.411 [info] <0.43.0> Application rabbitmq_web_dispatch exited with reason: stopped 2025-02-06 01:51:18.418 [info] <0.43.0> Application cowboy exited with reason: stopped 2025-02-06 01:51:18.418 [info] <0.1068.0> Stopping application 'cowlib' 2025-02-06 01:51:18.418 [info] <0.1068.0> Stopping application 'rabbitmq_management_agent' 2025-02-06 01:51:18.428 [info] <0.43.0> Application cowlib exited with reason: stopped 2025-02-06 01:51:18.429 [info] <0.1068.0> Stopping application 'rabbit' 2025-02-06 01:51:18.429 [info] <0.43.0> Application rabbitmq_management_agent exited with reason: stopped 2025-02-06 01:51:18.429 [info] <0.308.0> Will unregister with peer discovery backend rabbit_peer_discovery_classic_config 2025-02-06 01:51:18.429 [info] <0.680.0> stopped TCP listener on [::]:5673 2025-02-06 01:51:18.430 [info] <0.438.0> Closing all connections in vhost '/' on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2' because the vhost is stopping 2025-02-06 01:51:18.433 [info] <0.457.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' 2025-02-06 01:51:18.447 [info] <0.457.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' is stopped 2025-02-06 01:51:18.447 [info] <0.454.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' 2025-02-06 01:51:18.456 [info] <0.454.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' is stopped 2025-02-06 01:51:18.462 [info] <0.43.0> Application rabbit exited with reason: stopped 2025-02-06 01:51:18.462 [info] <0.1068.0> Stopping application 'mnesia' 2025-02-06 01:51:18.477 [info] <0.1068.0> Stopping application 'rabbit_common' 2025-02-06 01:51:18.477 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-02-06 01:51:18.477 [info] <0.43.0> Application rabbit_common exited with reason: stopped 2025-02-06 01:51:18.477 [info] <0.1068.0> Stopping application 'sysmon_handler' 2025-02-06 01:51:18.479 [info] <0.43.0> Application sysmon_handler exited with reason: stopped 2025-02-06 01:51:18.479 [info] <0.1068.0> Stopping application 'os_mon' 2025-02-06 01:51:18.480 [info] <0.43.0> Application os_mon exited with reason: stopped 2025-02-06 01:51:18.480 [info] <0.1068.0> Successfully stopped RabbitMQ and its dependencies 2025-02-06 01:51:18.480 [info] <0.1068.0> Halting Erlang VM with the following applications: jsx inets ranch ssl public_key asn1 xmerl credentials_obfuscation crypto observer_cli recon tools stdout_formatter lager goldrush compiler syntax_tools sasl stdlib kernel 2025-02-06 01:51:31.033 [info] <0.8.0> Log file opened with Lager 2025-02-06 01:51:34.034 [info] <0.8.0> Feature flags: list of feature flags found: 2025-02-06 01:51:34.034 [info] <0.8.0> Feature flags: feature flag states written to disk: yes 2025-02-06 01:51:34.108 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-02-06 01:51:34.167 [info] <0.309.0> Starting RabbitMQ 3.7.28 on Erlang 21.3.8.21 Copyright (c) 2007-2020 Pivotal Software, Inc. Licensed under the MPL. See https://www.rabbitmq.com/ 2025-02-06 01:51:34.168 [info] <0.309.0> node : contrail@cn-jenkins-deploy-platform-ansible-os-2696-2 home dir : /var/lib/rabbitmq config file(s) : /etc/rabbitmq/rabbitmq.config cookie hash : esZsDxSN6VGbi9JkMSxNZA== log(s) : /var/log/contrail/config-database-rabbitmq/rabbitmq.log : /var/log/rabbitmq/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2_upgrade.log database dir : /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2 2025-02-06 01:51:34.188 [info] <0.309.0> Running boot step pre_boot defined by app rabbit 2025-02-06 01:51:34.188 [info] <0.309.0> Running boot step rabbit_core_metrics defined by app rabbit 2025-02-06 01:51:34.190 [info] <0.309.0> Running boot step rabbit_alarm defined by app rabbit 2025-02-06 01:51:34.197 [info] <0.344.0> Memory high watermark set to 12637 MiB (13251077734 bytes) of 15796 MiB (16563847168 bytes) total 2025-02-06 01:51:34.205 [info] <0.358.0> Enabling free disk space monitoring 2025-02-06 01:51:34.205 [info] <0.358.0> Disk free limit set to 50MB 2025-02-06 01:51:34.210 [info] <0.309.0> Running boot step code_server_cache defined by app rabbit 2025-02-06 01:51:34.210 [info] <0.309.0> Running boot step file_handle_cache defined by app rabbit 2025-02-06 01:51:34.211 [info] <0.373.0> Limiting to approx 65436 file handles (58890 sockets) 2025-02-06 01:51:34.211 [info] <0.374.0> FHC read buffering: OFF 2025-02-06 01:51:34.211 [info] <0.374.0> FHC write buffering: ON 2025-02-06 01:51:34.213 [info] <0.309.0> Running boot step worker_pool defined by app rabbit 2025-02-06 01:51:34.213 [info] <0.312.0> Will use 4 processes for default worker pool 2025-02-06 01:51:34.214 [info] <0.312.0> Starting worker pool 'worker_pool' with 4 processes in it 2025-02-06 01:51:34.214 [info] <0.309.0> Running boot step database defined by app rabbit 2025-02-06 01:51:34.248 [info] <0.309.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-02-06 01:51:34.251 [info] <0.309.0> Successfully synced tables from a peer 2025-02-06 01:51:34.254 [info] <0.309.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-02-06 01:51:34.254 [info] <0.309.0> Successfully synced tables from a peer 2025-02-06 01:51:34.290 [info] <0.309.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-02-06 01:51:34.290 [info] <0.309.0> Successfully synced tables from a peer 2025-02-06 01:51:34.290 [info] <0.309.0> Will register with peer discovery backend rabbit_peer_discovery_classic_config 2025-02-06 01:51:34.554 [info] <0.309.0> Running boot step database_sync defined by app rabbit 2025-02-06 01:51:34.554 [info] <0.309.0> Running boot step feature_flags defined by app rabbit 2025-02-06 01:51:34.554 [info] <0.309.0> Running boot step codec_correctness_check defined by app rabbit 2025-02-06 01:51:34.554 [info] <0.309.0> Running boot step external_infrastructure defined by app rabbit 2025-02-06 01:51:34.554 [info] <0.309.0> Running boot step rabbit_registry defined by app rabbit 2025-02-06 01:51:34.554 [info] <0.309.0> Running boot step rabbit_auth_mechanism_cr_demo defined by app rabbit 2025-02-06 01:51:34.554 [info] <0.309.0> Running boot step rabbit_queue_location_random defined by app rabbit 2025-02-06 01:51:34.554 [info] <0.309.0> Running boot step rabbit_event defined by app rabbit 2025-02-06 01:51:34.555 [info] <0.309.0> Running boot step rabbit_auth_mechanism_amqplain defined by app rabbit 2025-02-06 01:51:34.555 [info] <0.309.0> Running boot step rabbit_auth_mechanism_plain defined by app rabbit 2025-02-06 01:51:34.555 [info] <0.309.0> Running boot step rabbit_exchange_type_direct defined by app rabbit 2025-02-06 01:51:34.555 [info] <0.309.0> Running boot step rabbit_exchange_type_fanout defined by app rabbit 2025-02-06 01:51:34.555 [info] <0.309.0> Running boot step rabbit_exchange_type_headers defined by app rabbit 2025-02-06 01:51:34.555 [info] <0.309.0> Running boot step rabbit_exchange_type_topic defined by app rabbit 2025-02-06 01:51:34.555 [info] <0.309.0> Running boot step rabbit_mirror_queue_mode_all defined by app rabbit 2025-02-06 01:51:34.555 [info] <0.309.0> Running boot step rabbit_mirror_queue_mode_exactly defined by app rabbit 2025-02-06 01:51:34.555 [info] <0.309.0> Running boot step rabbit_mirror_queue_mode_nodes defined by app rabbit 2025-02-06 01:51:34.555 [info] <0.309.0> Running boot step rabbit_priority_queue defined by app rabbit 2025-02-06 01:51:34.555 [info] <0.309.0> Priority queues enabled, real BQ is rabbit_variable_queue 2025-02-06 01:51:34.555 [info] <0.309.0> Running boot step rabbit_queue_location_client_local defined by app rabbit 2025-02-06 01:51:34.555 [info] <0.309.0> Running boot step rabbit_queue_location_min_masters defined by app rabbit 2025-02-06 01:51:34.555 [info] <0.309.0> Running boot step kernel_ready defined by app rabbit 2025-02-06 01:51:34.555 [info] <0.309.0> Running boot step rabbit_sysmon_minder defined by app rabbit 2025-02-06 01:51:34.556 [info] <0.309.0> Running boot step rabbit_epmd_monitor defined by app rabbit 2025-02-06 01:51:34.557 [info] <0.413.0> epmd monitor knows us, inter-node communication (distribution) port: 25673 2025-02-06 01:51:34.557 [info] <0.309.0> Running boot step guid_generator defined by app rabbit 2025-02-06 01:51:34.566 [info] <0.309.0> Running boot step rabbit_node_monitor defined by app rabbit 2025-02-06 01:51:34.566 [info] <0.417.0> Starting rabbit_node_monitor 2025-02-06 01:51:34.566 [info] <0.309.0> Running boot step delegate_sup defined by app rabbit 2025-02-06 01:51:34.567 [info] <0.309.0> Running boot step rabbit_memory_monitor defined by app rabbit 2025-02-06 01:51:34.567 [info] <0.309.0> Running boot step core_initialized defined by app rabbit 2025-02-06 01:51:34.567 [info] <0.309.0> Running boot step upgrade_queues defined by app rabbit 2025-02-06 01:51:34.589 [info] <0.309.0> Running boot step rabbit_connection_tracking defined by app rabbit 2025-02-06 01:51:34.589 [info] <0.309.0> Running boot step rabbit_connection_tracking_handler defined by app rabbit 2025-02-06 01:51:34.589 [info] <0.309.0> Running boot step rabbit_exchange_parameters defined by app rabbit 2025-02-06 01:51:34.589 [info] <0.309.0> Running boot step rabbit_mirror_queue_misc defined by app rabbit 2025-02-06 01:51:34.589 [info] <0.309.0> Running boot step rabbit_policies defined by app rabbit 2025-02-06 01:51:34.590 [info] <0.309.0> Running boot step rabbit_policy defined by app rabbit 2025-02-06 01:51:34.590 [info] <0.309.0> Running boot step rabbit_queue_location_validator defined by app rabbit 2025-02-06 01:51:34.590 [info] <0.309.0> Running boot step rabbit_vhost_limit defined by app rabbit 2025-02-06 01:51:34.590 [info] <0.309.0> Running boot step rabbit_mgmt_reset_handler defined by app rabbitmq_management 2025-02-06 01:51:34.590 [info] <0.309.0> Running boot step rabbit_mgmt_db_handler defined by app rabbitmq_management_agent 2025-02-06 01:51:34.590 [warning] <0.309.0> force_fine_statistics set to true; ignored. Replaced by {rates_mode, none} in the rabbitmq_management application. 2025-02-06 01:51:34.590 [info] <0.309.0> Management plugin: using rates mode 'basic' 2025-02-06 01:51:34.603 [info] <0.309.0> Running boot step recovery defined by app rabbit 2025-02-06 01:51:34.604 [info] <0.454.0> Making sure data directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists 2025-02-06 01:51:34.608 [info] <0.454.0> Starting message stores for vhost '/' 2025-02-06 01:51:34.608 [info] <0.458.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index 2025-02-06 01:51:34.610 [info] <0.454.0> Started message store of type transient for vhost '/' 2025-02-06 01:51:34.610 [info] <0.461.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index 2025-02-06 01:51:34.612 [info] <0.454.0> Started message store of type persistent for vhost '/' 2025-02-06 01:51:34.614 [info] <0.454.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-1.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.480.0> 2025-02-06 01:51:34.614 [info] <0.454.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-2.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2696-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.484.0> 2025-02-06 01:51:34.614 [info] <0.454.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2696-1' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.488.0> 2025-02-06 01:51:34.615 [info] <0.454.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-2.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2696-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.492.0> 2025-02-06 01:51:34.615 [info] <0.454.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.496.0> 2025-02-06 01:51:34.615 [info] <0.454.0> Mirrored queue 'job_status_consumer.dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.500.0> 2025-02-06 01:51:34.615 [info] <0.454.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2696-2' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.504.0> 2025-02-06 01:51:34.616 [info] <0.454.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-1..config_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.508.0> 2025-02-06 01:51:34.616 [info] <0.454.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-3.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.512.0> 2025-02-06 01:51:34.616 [info] <0.454.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2696-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.516.0> 2025-02-06 01:51:34.616 [info] <0.454.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.520.0> 2025-02-06 01:51:34.616 [info] <0.454.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.524.0> 2025-02-06 01:51:34.616 [info] <0.454.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-3..config_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.528.0> 2025-02-06 01:51:34.617 [info] <0.454.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2696-1..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.532.0> 2025-02-06 01:51:34.617 [info] <0.454.0> Mirrored queue 'job_request_consumer' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.536.0> 2025-02-06 01:51:34.617 [info] <0.454.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2696-3.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.540.0> 2025-02-06 01:51:34.617 [info] <0.454.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-1..tftp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.544.0> 2025-02-06 01:51:34.617 [info] <0.454.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2696-1.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.548.0> 2025-02-06 01:51:34.617 [info] <0.454.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2696-3' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.552.0> 2025-02-06 01:51:34.618 [info] <0.454.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2696-2' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.556.0> 2025-02-06 01:51:34.618 [info] <0.454.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2696-2.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.560.0> 2025-02-06 01:51:34.618 [info] <0.454.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2696-2..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.564.0> 2025-02-06 01:51:34.618 [info] <0.454.0> Mirrored queue 'device_manager.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.568.0> 2025-02-06 01:51:34.618 [info] <0.454.0> Mirrored queue 'device_manager_ztp.ztp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.572.0> 2025-02-06 01:51:34.619 [info] <0.454.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-3.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.576.0> 2025-02-06 01:51:34.619 [info] <0.454.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-1.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.580.0> 2025-02-06 01:51:34.619 [info] <0.454.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-2.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2696-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.584.0> 2025-02-06 01:51:34.619 [info] <0.454.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2696-1' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.588.0> 2025-02-06 01:51:34.619 [info] <0.454.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-3..tftp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.592.0> 2025-02-06 01:51:34.620 [info] <0.454.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2696-3' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.596.0> 2025-02-06 01:51:34.620 [info] <0.454.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-1.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.600.0> 2025-02-06 01:51:34.620 [info] <0.454.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.604.0> 2025-02-06 01:51:34.620 [info] <0.454.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-3.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.608.0> 2025-02-06 01:51:34.620 [info] <0.454.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2696-3..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.612.0> 2025-02-06 01:51:34.620 [info] <0.309.0> Running boot step load_definitions defined by app rabbitmq_management 2025-02-06 01:51:34.621 [info] <0.309.0> Applying definitions from /etc/rabbitmq/definitions.json 2025-02-06 01:51:34.621 [info] <0.309.0> Asked to import definitions. Acting user: rmq-internal 2025-02-06 01:51:34.621 [info] <0.309.0> Importing users... 2025-02-06 01:51:34.633 [info] <0.309.0> Setting user tags for user 'guest' to [administrator] 2025-02-06 01:51:34.677 [info] <0.309.0> Importing vhosts... 2025-02-06 01:51:34.678 [info] <0.309.0> Importing user permissions... 2025-02-06 01:51:34.678 [info] <0.309.0> Setting permissions for 'guest' in '/' to '.*', '.*', '.*' 2025-02-06 01:51:34.711 [info] <0.309.0> Importing topic permissions... 2025-02-06 01:51:34.711 [info] <0.309.0> Importing parameters... 2025-02-06 01:51:34.711 [info] <0.309.0> Importing global parameters... 2025-02-06 01:51:34.711 [info] <0.309.0> Importing policies... 2025-02-06 01:51:34.759 [info] <0.309.0> Importing queues... 2025-02-06 01:51:34.759 [info] <0.309.0> Importing exchanges... 2025-02-06 01:51:34.759 [info] <0.309.0> Importing bindings... 2025-02-06 01:51:34.759 [info] <0.309.0> Running boot step empty_db_check defined by app rabbit 2025-02-06 01:51:34.759 [info] <0.309.0> Running boot step rabbit_looking_glass defined by app rabbit 2025-02-06 01:51:34.759 [info] <0.309.0> Running boot step rabbit_core_metrics_gc defined by app rabbit 2025-02-06 01:51:34.759 [info] <0.309.0> Running boot step background_gc defined by app rabbit 2025-02-06 01:51:34.759 [info] <0.309.0> Running boot step connection_tracking defined by app rabbit 2025-02-06 01:51:34.766 [info] <0.309.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2696-2' 2025-02-06 01:51:34.768 [info] <0.309.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2696-2' 2025-02-06 01:51:34.768 [info] <0.309.0> Running boot step routing_ready defined by app rabbit 2025-02-06 01:51:34.768 [info] <0.309.0> Running boot step pre_flight defined by app rabbit 2025-02-06 01:51:34.768 [info] <0.309.0> Running boot step notify_cluster defined by app rabbit 2025-02-06 01:51:34.768 [info] <0.309.0> Running boot step networking defined by app rabbit 2025-02-06 01:51:34.768 [info] <0.417.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-1' up 2025-02-06 01:51:34.770 [info] <0.740.0> started TCP listener on [::]:5673 2025-02-06 01:51:34.770 [info] <0.309.0> Running boot step cluster_name defined by app rabbit 2025-02-06 01:51:34.770 [info] <0.309.0> Running boot step direct_client defined by app rabbit 2025-02-06 01:51:34.770 [info] <0.309.0> Running boot step os_signal_handler defined by app rabbit 2025-02-06 01:51:34.770 [info] <0.742.0> Swapping OS signal event handler (erl_signal_server) for our own 2025-02-06 01:51:34.823 [info] <0.793.0> Management plugin: HTTP (non-TLS) listener started on port 15673 2025-02-06 01:51:34.823 [info] <0.901.0> Statistics database started. 2025-02-06 01:51:34.823 [info] <0.900.0> Starting worker pool 'management_worker_pool' with 3 processes in it 2025-02-06 01:51:34.824 [notice] <0.106.0> Changed loghwm of /var/log/contrail/config-database-rabbitmq/rabbitmq.log to 50 2025-02-06 01:51:34.953 [info] <0.8.0> Server startup complete; 3 plugins started. * rabbitmq_management * rabbitmq_web_dispatch * rabbitmq_management_agent 2025-02-06 01:51:37.014 [info] <0.417.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-3' up 2025-02-06 01:58:14.962 [info] <0.1737.0> accepting AMQP connection <0.1737.0> (10.0.0.41:59286 -> 10.0.0.41:5673) 2025-02-06 01:58:14.964 [info] <0.1737.0> connection <0.1737.0> (10.0.0.41:59286 -> 10.0.0.41:5673): user 'guest' authenticated and granted access to vhost '/' 2025-02-06 01:58:14.964 [info] <0.1745.0> accepting AMQP connection <0.1745.0> (10.0.0.23:42104 -> 10.0.0.41:5673) 2025-02-06 01:58:14.965 [info] <0.1745.0> connection <0.1745.0> (10.0.0.23:42104 -> 10.0.0.41:5673): user 'guest' authenticated and granted access to vhost '/' 2025-02-06 01:58:14.967 [info] <0.1754.0> accepting AMQP connection <0.1754.0> (10.0.0.23:42114 -> 10.0.0.41:5673) 2025-02-06 01:58:14.968 [info] <0.1758.0> accepting AMQP connection <0.1758.0> (10.0.0.38:48284 -> 10.0.0.41:5673) 2025-02-06 01:58:14.969 [info] <0.1761.0> accepting AMQP connection <0.1761.0> (10.0.0.38:48286 -> 10.0.0.41:5673) 2025-02-06 01:58:14.970 [info] <0.1764.0> accepting AMQP connection <0.1764.0> (10.0.0.41:59288 -> 10.0.0.41:5673) 2025-02-06 01:58:14.970 [info] <0.1767.0> accepting AMQP connection <0.1767.0> (10.0.0.38:48294 -> 10.0.0.41:5673) 2025-02-06 01:58:14.971 [info] <0.1773.0> accepting AMQP connection <0.1773.0> (10.0.0.23:42132 -> 10.0.0.41:5673) 2025-02-06 01:58:14.971 [info] <0.1776.0> accepting AMQP connection <0.1776.0> (10.0.0.41:59300 -> 10.0.0.41:5673) 2025-02-06 01:58:14.972 [info] <0.1770.0> accepting AMQP connection <0.1770.0> (10.0.0.23:42120 -> 10.0.0.41:5673) 2025-02-06 01:58:14.973 [info] <0.1779.0> accepting AMQP connection <0.1779.0> (10.0.0.41:59302 -> 10.0.0.41:5673) 2025-02-06 01:58:14.974 [info] <0.1784.0> accepting AMQP connection <0.1784.0> (10.0.0.41:59308 -> 10.0.0.41:5673) 2025-02-06 01:58:14.974 [info] <0.1787.0> accepting AMQP connection <0.1787.0> (10.0.0.38:48298 -> 10.0.0.41:5673) 2025-02-06 01:58:14.974 [info] <0.1758.0> connection <0.1758.0> (10.0.0.38:48284 -> 10.0.0.41:5673): user 'guest' authenticated and granted access to vhost '/' 2025-02-06 01:58:14.975 [info] <0.1790.0> accepting AMQP connection <0.1790.0> (10.0.0.23:42134 -> 10.0.0.41:5673) 2025-02-06 01:58:14.975 [info] <0.1793.0> accepting AMQP connection <0.1793.0> (10.0.0.38:48304 -> 10.0.0.41:5673) 2025-02-06 01:58:14.987 [info] <0.1754.0> connection <0.1754.0> (10.0.0.23:42114 -> 10.0.0.41:5673): user 'guest' authenticated and granted access to vhost '/' 2025-02-06 01:58:14.988 [info] <0.1793.0> connection <0.1793.0> (10.0.0.38:48304 -> 10.0.0.41:5673): user 'guest' authenticated and granted access to vhost '/' 2025-02-06 01:58:14.988 [info] <0.1784.0> connection <0.1784.0> (10.0.0.41:59308 -> 10.0.0.41:5673): user 'guest' authenticated and granted access to vhost '/' 2025-02-06 01:58:14.988 [info] <0.1773.0> connection <0.1773.0> (10.0.0.23:42132 -> 10.0.0.41:5673): user 'guest' authenticated and granted access to vhost '/' 2025-02-06 01:58:14.989 [info] <0.1806.0> accepting AMQP connection <0.1806.0> (10.0.0.41:59332 -> 10.0.0.41:5673) 2025-02-06 01:58:14.990 [info] <0.1817.0> accepting AMQP connection <0.1817.0> (10.0.0.38:48306 -> 10.0.0.41:5673) 2025-02-06 01:58:14.990 [info] <0.1767.0> connection <0.1767.0> (10.0.0.38:48294 -> 10.0.0.41:5673): user 'guest' authenticated and granted access to vhost '/' 2025-02-06 01:58:14.990 [info] <0.1787.0> connection <0.1787.0> (10.0.0.38:48298 -> 10.0.0.41:5673): user 'guest' authenticated and granted access to vhost '/' 2025-02-06 01:58:14.991 [info] <0.1835.0> accepting AMQP connection <0.1835.0> (10.0.0.41:59310 -> 10.0.0.41:5673) 2025-02-06 01:58:14.991 [info] <0.1841.0> accepting AMQP connection <0.1841.0> (10.0.0.23:42148 -> 10.0.0.41:5673) 2025-02-06 01:58:14.991 [info] <0.1850.0> accepting AMQP connection <0.1850.0> (10.0.0.38:48312 -> 10.0.0.41:5673) 2025-02-06 01:58:14.991 [info] <0.1770.0> connection <0.1770.0> (10.0.0.23:42120 -> 10.0.0.41:5673): user 'guest' authenticated and granted access to vhost '/' 2025-02-06 01:58:14.991 [info] <0.1790.0> connection <0.1790.0> (10.0.0.23:42134 -> 10.0.0.41:5673): user 'guest' authenticated and granted access to vhost '/' 2025-02-06 01:58:14.991 [info] <0.1779.0> connection <0.1779.0> (10.0.0.41:59302 -> 10.0.0.41:5673): user 'guest' authenticated and granted access to vhost '/' 2025-02-06 01:58:14.991 [info] <0.1776.0> connection <0.1776.0> (10.0.0.41:59300 -> 10.0.0.41:5673): user 'guest' authenticated and granted access to vhost '/' 2025-02-06 01:58:14.992 [info] <0.1761.0> connection <0.1761.0> (10.0.0.38:48286 -> 10.0.0.41:5673): user 'guest' authenticated and granted access to vhost '/' 2025-02-06 01:58:14.992 [info] <0.1764.0> connection <0.1764.0> (10.0.0.41:59288 -> 10.0.0.41:5673): user 'guest' authenticated and granted access to vhost '/' 2025-02-06 01:58:14.992 [info] <0.1870.0> accepting AMQP connection <0.1870.0> (10.0.0.41:59312 -> 10.0.0.41:5673) 2025-02-06 01:58:14.992 [info] <0.1874.0> accepting AMQP connection <0.1874.0> (10.0.0.41:59316 -> 10.0.0.41:5673) 2025-02-06 01:58:14.993 [info] <0.1886.0> accepting AMQP connection <0.1886.0> (10.0.0.41:59318 -> 10.0.0.41:5673) 2025-02-06 01:58:14.994 [info] <0.1892.0> accepting AMQP connection <0.1892.0> (10.0.0.23:42160 -> 10.0.0.41:5673) 2025-02-06 01:58:14.998 [info] <0.1817.0> connection <0.1817.0> (10.0.0.38:48306 -> 10.0.0.41:5673): user 'guest' authenticated and granted access to vhost '/' 2025-02-06 01:58:14.999 [info] <0.1874.0> connection <0.1874.0> (10.0.0.41:59316 -> 10.0.0.41:5673): user 'guest' authenticated and granted access to vhost '/' 2025-02-06 01:58:15.001 [info] <0.1870.0> connection <0.1870.0> (10.0.0.41:59312 -> 10.0.0.41:5673): user 'guest' authenticated and granted access to vhost '/' 2025-02-06 01:58:15.002 [info] <0.1841.0> connection <0.1841.0> (10.0.0.23:42148 -> 10.0.0.41:5673): user 'guest' authenticated and granted access to vhost '/' 2025-02-06 01:58:15.002 [info] <0.1850.0> connection <0.1850.0> (10.0.0.38:48312 -> 10.0.0.41:5673): user 'guest' authenticated and granted access to vhost '/' 2025-02-06 01:58:15.002 [info] <0.1806.0> connection <0.1806.0> (10.0.0.41:59332 -> 10.0.0.41:5673): user 'guest' authenticated and granted access to vhost '/' 2025-02-06 01:58:15.003 [info] <0.1835.0> connection <0.1835.0> (10.0.0.41:59310 -> 10.0.0.41:5673): user 'guest' authenticated and granted access to vhost '/' 2025-02-06 01:58:15.005 [info] <0.1886.0> connection <0.1886.0> (10.0.0.41:59318 -> 10.0.0.41:5673): user 'guest' authenticated and granted access to vhost '/' 2025-02-06 01:58:15.006 [info] <0.1892.0> connection <0.1892.0> (10.0.0.23:42160 -> 10.0.0.41:5673): user 'guest' authenticated and granted access to vhost '/' 2025-02-06 01:58:15.125 [info] <0.1986.0> accepting AMQP connection <0.1986.0> (10.0.0.41:59344 -> 10.0.0.41:5673) 2025-02-06 01:58:15.126 [info] <0.1989.0> accepting AMQP connection <0.1989.0> (10.0.0.41:59356 -> 10.0.0.41:5673) 2025-02-06 01:58:15.127 [info] <0.1983.0> accepting AMQP connection <0.1983.0> (10.0.0.23:42168 -> 10.0.0.41:5673) 2025-02-06 01:58:15.131 [info] <0.1986.0> connection <0.1986.0> (10.0.0.41:59344 -> 10.0.0.41:5673): user 'guest' authenticated and granted access to vhost '/' 2025-02-06 01:58:15.131 [info] <0.1989.0> connection <0.1989.0> (10.0.0.41:59356 -> 10.0.0.41:5673): user 'guest' authenticated and granted access to vhost '/' 2025-02-06 01:58:15.134 [info] <0.1983.0> connection <0.1983.0> (10.0.0.23:42168 -> 10.0.0.41:5673): user 'guest' authenticated and granted access to vhost '/' 2025-02-06 01:58:15.134 [info] <0.417.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-1' down 2025-02-06 01:58:15.143 [info] <0.417.0> Keeping contrail@cn-jenkins-deploy-platform-ansible-os-2696-1 listeners: the node is already back 2025-02-06 01:58:15.148 [info] <0.2035.0> accepting AMQP connection <0.2035.0> (10.0.0.38:48318 -> 10.0.0.41:5673) 2025-02-06 01:58:15.149 [info] <0.2006.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2696-2..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-3': <5876.1722.0> 2025-02-06 01:58:15.149 [info] <0.2035.0> connection <0.2035.0> (10.0.0.38:48318 -> 10.0.0.41:5673): user 'guest' authenticated and granted access to vhost '/' 2025-02-06 01:58:15.179 [info] <0.2006.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2696-2..dummy' in vhost '/': Synchronising: 0 messages to synchronise 2025-02-06 01:58:15.179 [info] <0.2006.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2696-2..dummy' in vhost '/': Synchronising: batch size: 5 2025-02-06 01:58:15.186 [info] <0.1542.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-1.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Slave saw deaths of mirrors 2025-02-06 01:58:15.186 [info] <0.544.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-1..tftp_queue' in vhost '/': Slave saw deaths of mirrors 2025-02-06 01:58:15.186 [info] <0.544.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-1..tftp_queue' in vhost '/': Promoting slave to master 2025-02-06 01:58:15.186 [info] <0.544.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-1..tftp_queue' in vhost '/': Synchronising: 0 messages to synchronise 2025-02-06 01:58:15.186 [info] <0.544.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-1..tftp_queue' in vhost '/': Synchronising: batch size: 5 2025-02-06 01:58:15.187 [info] <0.417.0> node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-1' down: connection_closed 2025-02-06 01:58:15.187 [info] <0.1216.0> Mirrored queue 'svc_monitor.cn-jenkins-deploy-platform-ansible-os-2696-2.' in vhost '/': Slave saw deaths of mirrors 2025-02-06 01:58:15.187 [info] <0.528.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-3..config_queue' in vhost '/': Slave saw deaths of mirrors 2025-02-06 01:58:15.187 [info] <0.2095.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2696-2..dummy' in vhost '/': Synchronising: all slaves already synced 2025-02-06 01:58:15.187 [info] <0.528.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-3..config_queue' in vhost '/': Promoting slave to master 2025-02-06 01:58:15.188 [info] <0.528.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-3..config_queue' in vhost '/': Synchronising: 0 messages to synchronise 2025-02-06 01:58:15.188 [info] <0.528.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-3..config_queue' in vhost '/': Synchronising: batch size: 5 2025-02-06 01:58:15.188 [info] <0.2022.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2696-1..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-3': <5876.1838.0> 2025-02-06 01:58:15.188 [info] <0.2046.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2696-3..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-3': <5876.1843.0> 2025-02-06 01:58:15.188 [info] <0.2009.0> Mirrored queue 'job_status_consumer.dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-3': <5876.1846.0> 2025-02-06 01:58:15.188 [info] <0.1496.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Slave saw deaths of mirrors 2025-02-06 01:58:15.189 [info] <0.1531.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-3.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Slave saw deaths of mirrors 2025-02-06 01:58:15.189 [info] <0.1650.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2696-1' in vhost '/': Slave saw deaths of mirrors 2025-02-06 01:58:15.189 [info] <0.1438.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-2..config_queue' in vhost '/': Slave saw deaths of mirrors 2025-02-06 01:58:15.190 [info] <0.1662.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2696-2' in vhost '/': Slave saw deaths of mirrors 2025-02-06 01:58:15.190 [info] <0.1427.0> Mirrored queue 'device_manager.cn-jenkins-deploy-platform-ansible-os-2696-2.' in vhost '/': Slave saw deaths of mirrors 2025-02-06 01:58:15.191 [info] <0.1578.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-3.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Slave saw deaths of mirrors 2025-02-06 01:58:15.191 [info] <0.2130.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-1..tftp_queue' in vhost '/': Synchronising: all slaves already synced 2025-02-06 01:58:15.191 [info] <0.1484.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2696-2.' in vhost '/': Slave saw deaths of mirrors 2025-02-06 01:58:15.192 [info] <0.2136.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-3..config_queue' in vhost '/': Synchronising: all slaves already synced 2025-02-06 01:58:15.192 [info] <0.1554.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-2.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2696-2.' in vhost '/': Slave saw deaths of mirrors 2025-02-06 01:58:15.192 [info] <0.508.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-1..config_queue' in vhost '/': Promoting slave to master 2025-02-06 01:58:15.192 [info] <0.508.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-1..config_queue' in vhost '/': Synchronising: 0 messages to synchronise 2025-02-06 01:58:16.189 [warning] lager_file_backend dropped 39 messages in the last second that exceeded the limit of 50 messages/sec 2025-02-06 01:58:45.213 [info] <0.417.0> node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-1' up 2025-02-06 01:59:11.926 [info] <0.2343.0> accepting AMQP connection <0.2343.0> (10.0.0.38:42192 -> 10.0.0.41:5673) 2025-02-06 01:59:11.928 [info] <0.2343.0> connection <0.2343.0> (10.0.0.38:42192 -> 10.0.0.41:5673): user 'guest' authenticated and granted access to vhost '/' 2025-02-06 01:59:11.939 [info] <0.2352.0> Mirrored queue 'schema_transformer.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-3': <5876.2079.0> 2025-02-06 01:59:11.943 [info] <0.2352.0> Mirrored queue 'schema_transformer.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Synchronising: 0 messages to synchronise 2025-02-06 01:59:11.943 [info] <0.2352.0> Mirrored queue 'schema_transformer.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Synchronising: batch size: 5 2025-02-06 01:59:11.944 [info] <0.2359.0> Mirrored queue 'schema_transformer.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Synchronising: all slaves already synced 2025-02-06 01:59:11.947 [info] <0.2363.0> accepting AMQP connection <0.2363.0> (10.0.0.38:42202 -> 10.0.0.41:5673) 2025-02-06 01:59:11.948 [info] <0.2363.0> connection <0.2363.0> (10.0.0.38:42202 -> 10.0.0.41:5673): user 'guest' authenticated and granted access to vhost '/' 2025-02-06 01:59:11.958 [info] <0.2343.0> closing AMQP connection <0.2343.0> (10.0.0.38:42192 -> 10.0.0.41:5673, vhost: '/', user: 'guest') 2025-02-06 01:59:11.958 [info] <0.2363.0> closing AMQP connection <0.2363.0> (10.0.0.38:42202 -> 10.0.0.41:5673, vhost: '/', user: 'guest') 2025-02-06 01:59:12.102 [info] <0.2378.0> accepting AMQP connection <0.2378.0> (10.0.0.38:42212 -> 10.0.0.41:5673) 2025-02-06 01:59:12.103 [info] <0.2378.0> connection <0.2378.0> (10.0.0.38:42212 -> 10.0.0.41:5673): user 'guest' authenticated and granted access to vhost '/' 2025-02-06 01:59:12.117 [info] <0.2387.0> Mirrored queue 'svc_monitor.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-3': <5876.2091.0> 2025-02-06 01:59:12.120 [info] <0.2387.0> Mirrored queue 'svc_monitor.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Synchronising: 0 messages to synchronise 2025-02-06 01:59:12.120 [info] <0.2387.0> Mirrored queue 'svc_monitor.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Synchronising: batch size: 5 2025-02-06 01:59:12.121 [info] <0.2394.0> Mirrored queue 'svc_monitor.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Synchronising: all slaves already synced 2025-02-06 01:59:12.136 [info] <0.2398.0> accepting AMQP connection <0.2398.0> (10.0.0.38:42218 -> 10.0.0.41:5673) 2025-02-06 01:59:12.137 [info] <0.2398.0> connection <0.2398.0> (10.0.0.38:42218 -> 10.0.0.41:5673): user 'guest' authenticated and granted access to vhost '/' 2025-02-06 01:59:12.145 [info] <0.2378.0> closing AMQP connection <0.2378.0> (10.0.0.38:42212 -> 10.0.0.41:5673, vhost: '/', user: 'guest') 2025-02-06 01:59:12.146 [info] <0.2398.0> closing AMQP connection <0.2398.0> (10.0.0.38:42218 -> 10.0.0.41:5673, vhost: '/', user: 'guest') 2025-02-06 01:59:12.771 [info] <0.2413.0> accepting AMQP connection <0.2413.0> (10.0.0.23:34064 -> 10.0.0.41:5673) 2025-02-06 01:59:12.773 [info] <0.2413.0> connection <0.2413.0> (10.0.0.23:34064 -> 10.0.0.41:5673): user 'guest' authenticated and granted access to vhost '/' 2025-02-06 01:59:12.786 [info] <0.2422.0> Mirrored queue 'schema_transformer.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-3': <5876.2103.0> 2025-02-06 01:59:12.788 [info] <0.2422.0> Mirrored queue 'schema_transformer.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Synchronising: 0 messages to synchronise 2025-02-06 01:59:12.788 [info] <0.2422.0> Mirrored queue 'schema_transformer.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Synchronising: batch size: 5 2025-02-06 01:59:12.789 [info] <0.2429.0> Mirrored queue 'schema_transformer.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Synchronising: all slaves already synced 2025-02-06 01:59:12.792 [info] <0.2433.0> accepting AMQP connection <0.2433.0> (10.0.0.23:34076 -> 10.0.0.41:5673) 2025-02-06 01:59:12.793 [info] <0.2433.0> connection <0.2433.0> (10.0.0.23:34076 -> 10.0.0.41:5673): user 'guest' authenticated and granted access to vhost '/' 2025-02-06 01:59:12.801 [info] <0.2413.0> closing AMQP connection <0.2413.0> (10.0.0.23:34064 -> 10.0.0.41:5673, vhost: '/', user: 'guest') 2025-02-06 01:59:12.802 [info] <0.2433.0> closing AMQP connection <0.2433.0> (10.0.0.23:34076 -> 10.0.0.41:5673, vhost: '/', user: 'guest') 2025-02-06 01:59:12.962 [info] <0.2448.0> accepting AMQP connection <0.2448.0> (10.0.0.23:34090 -> 10.0.0.41:5673) 2025-02-06 01:59:12.964 [info] <0.2448.0> connection <0.2448.0> (10.0.0.23:34090 -> 10.0.0.41:5673): user 'guest' authenticated and granted access to vhost '/' 2025-02-06 01:59:12.976 [info] <0.2457.0> Mirrored queue 'svc_monitor.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-3': <5876.2115.0> 2025-02-06 01:59:12.979 [info] <0.2457.0> Mirrored queue 'svc_monitor.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Synchronising: 0 messages to synchronise 2025-02-06 01:59:12.979 [info] <0.2457.0> Mirrored queue 'svc_monitor.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Synchronising: batch size: 5 2025-02-06 01:59:12.979 [info] <0.2464.0> Mirrored queue 'svc_monitor.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Synchronising: all slaves already synced 2025-02-06 01:59:12.993 [info] <0.2468.0> accepting AMQP connection <0.2468.0> (10.0.0.23:34094 -> 10.0.0.41:5673) 2025-02-06 01:59:12.994 [info] <0.2468.0> connection <0.2468.0> (10.0.0.23:34094 -> 10.0.0.41:5673): user 'guest' authenticated and granted access to vhost '/' 2025-02-06 01:59:13.003 [info] <0.2448.0> closing AMQP connection <0.2448.0> (10.0.0.23:34090 -> 10.0.0.41:5673, vhost: '/', user: 'guest') 2025-02-06 01:59:13.003 [info] <0.2468.0> closing AMQP connection <0.2468.0> (10.0.0.23:34094 -> 10.0.0.41:5673, vhost: '/', user: 'guest') 2025-02-06 01:59:21.356 [info] <0.496.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Synchronising: 0 messages to synchronise 2025-02-06 01:59:21.357 [info] <0.496.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Synchronising: batch size: 5 2025-02-06 01:59:21.403 [info] <0.2009.0> Mirrored queue 'job_status_consumer.dummy' in vhost '/': Synchronising: 0 messages to synchronise 2025-02-06 01:59:21.404 [info] <0.508.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-1..config_queue' in vhost '/': Synchronising: 0 messages to synchronise 2025-02-06 01:59:21.404 [info] <0.2009.0> Mirrored queue 'job_status_consumer.dummy' in vhost '/': Synchronising: batch size: 5 2025-02-06 01:59:21.404 [info] <0.508.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-1..config_queue' in vhost '/': Synchronising: batch size: 5 2025-02-06 01:59:21.407 [info] <0.2529.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Synchronising: all slaves already synced 2025-02-06 01:59:21.423 [info] <0.544.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-1..tftp_queue' in vhost '/': Synchronising: 0 messages to synchronise 2025-02-06 01:59:21.424 [info] <0.544.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-1..tftp_queue' in vhost '/': Synchronising: batch size: 5 2025-02-06 01:59:21.424 [info] <0.528.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-3..config_queue' in vhost '/': Synchronising: 0 messages to synchronise 2025-02-06 01:59:21.424 [info] <0.528.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-3..config_queue' in vhost '/': Synchronising: batch size: 5 2025-02-06 01:59:21.426 [info] <0.524.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Synchronising: 0 messages to synchronise 2025-02-06 01:59:21.426 [info] <0.524.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Synchronising: batch size: 5 2025-02-06 01:59:21.427 [info] <0.2022.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2696-1..dummy' in vhost '/': Synchronising: 0 messages to synchronise 2025-02-06 01:59:21.427 [info] <0.2022.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2696-1..dummy' in vhost '/': Synchronising: batch size: 5 2025-02-06 01:59:21.438 [info] <0.568.0> Mirrored queue 'device_manager.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Synchronising: 584 messages to synchronise 2025-02-06 01:59:21.438 [info] <0.568.0> Mirrored queue 'device_manager.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Synchronising: batch size: 5 2025-02-06 01:59:21.439 [info] <0.2006.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2696-2..dummy' in vhost '/': Synchronising: 0 messages to synchronise 2025-02-06 01:59:21.439 [info] <0.2006.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2696-2..dummy' in vhost '/': Synchronising: batch size: 5 2025-02-06 01:59:21.439 [info] <0.2538.0> Mirrored queue 'device_manager.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Synchronising: mirrors ['contrail@cn-jenkins-deploy-platform-ansible-os-2696-1'] to sync 2025-02-06 01:59:21.441 [info] <0.568.0> Mirrored queue 'device_manager.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Synchronising: complete 2025-02-06 01:59:21.446 [info] <0.572.0> Mirrored queue 'device_manager_ztp.ztp_queue' in vhost '/': Synchronising: 0 messages to synchronise 2025-02-06 01:59:21.446 [info] <0.572.0> Mirrored queue 'device_manager_ztp.ztp_queue' in vhost '/': Synchronising: batch size: 5 2025-02-06 01:59:21.448 [info] <0.2532.0> Mirrored queue 'job_status_consumer.dummy' in vhost '/': Synchronising: all slaves already synced 2025-02-06 01:59:21.449 [info] <0.2533.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-1..config_queue' in vhost '/': Synchronising: all slaves already synced 2025-02-06 01:59:21.449 [info] <0.2046.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2696-3..dummy' in vhost '/': Synchronising: 0 messages to synchronise 2025-02-06 01:59:21.450 [info] <0.2046.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2696-3..dummy' in vhost '/': Synchronising: batch size: 5 2025-02-06 01:59:21.459 [info] <0.592.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-3..tftp_queue' in vhost '/': Synchronising: 0 messages to synchronise 2025-02-06 01:59:21.459 [info] <0.592.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-3..tftp_queue' in vhost '/': Synchronising: batch size: 5 2025-02-06 01:59:21.461 [info] <0.2534.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-1..tftp_queue' in vhost '/': Synchronising: all slaves already synced 2025-02-06 01:59:21.468 [info] <0.2535.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-3..config_queue' in vhost '/': Synchronising: all slaves already synced 2025-02-06 01:59:21.469 [info] <0.2537.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2696-1..dummy' in vhost '/': Synchronising: all slaves already synced 2025-02-06 01:59:21.469 [info] <0.2536.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Synchronising: all slaves already synced 2025-02-06 01:59:21.469 [info] <0.536.0> Mirrored queue 'job_request_consumer' in vhost '/': Synchronising: 0 messages to synchronise 2025-02-06 01:59:21.470 [info] <0.536.0> Mirrored queue 'job_request_consumer' in vhost '/': Synchronising: batch size: 5 2025-02-06 01:59:21.470 [info] <0.2543.0> Mirrored queue 'job_request_consumer' in vhost '/': Synchronising: all slaves already synced 2025-02-06 01:59:21.471 [info] <0.2539.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2696-2..dummy' in vhost '/': Synchronising: all slaves already synced 2025-02-06 01:59:21.473 [info] <0.2540.0> Mirrored queue 'device_manager_ztp.ztp_queue' in vhost '/': Synchronising: all slaves already synced 2025-02-06 01:59:21.476 [info] <0.2542.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-3..tftp_queue' in vhost '/': Synchronising: all slaves already synced 2025-02-06 01:59:21.477 [info] <0.2541.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2696-3..dummy' in vhost '/': Synchronising: all slaves already synced 2025-02-06 01:59:21.486 [info] <0.417.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-1' up 2025-02-06 01:59:43.236 [info] <0.2692.0> RabbitMQ is asked to stop... 2025-02-06 01:59:43.310 [info] <0.2692.0> Stopping RabbitMQ applications and their dependencies in the following order: rabbitmq_management amqp_client rabbitmq_web_dispatch cowboy cowlib rabbitmq_management_agent rabbit mnesia rabbit_common sysmon_handler os_mon 2025-02-06 01:59:43.310 [info] <0.2692.0> Stopping application 'rabbitmq_management' 2025-02-06 01:59:43.313 [warning] <0.789.0> RabbitMQ HTTP listener registry could not find context rabbitmq_management_tls 2025-02-06 01:59:43.314 [info] <0.2692.0> Stopping application 'amqp_client' 2025-02-06 01:59:43.314 [info] <0.43.0> Application rabbitmq_management exited with reason: stopped 2025-02-06 01:59:43.315 [info] <0.2692.0> Stopping application 'rabbitmq_web_dispatch' 2025-02-06 01:59:43.315 [info] <0.43.0> Application amqp_client exited with reason: stopped 2025-02-06 01:59:43.317 [info] <0.43.0> Application rabbitmq_web_dispatch exited with reason: stopped 2025-02-06 01:59:43.317 [info] <0.2692.0> Stopping application 'cowboy' 2025-02-06 01:59:43.318 [info] <0.2692.0> Stopping application 'cowlib' 2025-02-06 01:59:43.318 [info] <0.43.0> Application cowboy exited with reason: stopped 2025-02-06 01:59:43.318 [info] <0.2692.0> Stopping application 'rabbitmq_management_agent' 2025-02-06 01:59:43.318 [info] <0.43.0> Application cowlib exited with reason: stopped 2025-02-06 01:59:43.319 [info] <0.43.0> Application rabbitmq_management_agent exited with reason: stopped 2025-02-06 01:59:43.319 [info] <0.2692.0> Stopping application 'rabbit' 2025-02-06 01:59:43.319 [info] <0.309.0> Will unregister with peer discovery backend rabbit_peer_discovery_classic_config 2025-02-06 01:59:43.319 [info] <0.740.0> stopped TCP listener on [::]:5673 2025-02-06 01:59:43.320 [error] <0.1986.0> Error on AMQP connection <0.1986.0> (10.0.0.41:59344 -> 10.0.0.41:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-02-06 01:59:43.320 [error] <0.1989.0> Error on AMQP connection <0.1989.0> (10.0.0.41:59356 -> 10.0.0.41:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-02-06 01:59:43.320 [error] <0.1754.0> Error on AMQP connection <0.1754.0> (10.0.0.23:42114 -> 10.0.0.41:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-02-06 01:59:43.320 [error] <0.1779.0> Error on AMQP connection <0.1779.0> (10.0.0.41:59302 -> 10.0.0.41:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-02-06 01:59:43.320 [error] <0.1983.0> Error on AMQP connection <0.1983.0> (10.0.0.23:42168 -> 10.0.0.41:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-02-06 01:59:43.320 [error] <0.1776.0> Error on AMQP connection <0.1776.0> (10.0.0.41:59300 -> 10.0.0.41:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-02-06 01:59:43.320 [error] <0.1745.0> Error on AMQP connection <0.1745.0> (10.0.0.23:42104 -> 10.0.0.41:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-02-06 01:59:43.320 [error] <0.1841.0> Error on AMQP connection <0.1841.0> (10.0.0.23:42148 -> 10.0.0.41:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-02-06 01:59:43.320 [error] <0.1817.0> Error on AMQP connection <0.1817.0> (10.0.0.38:48306 -> 10.0.0.41:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-02-06 01:59:43.321 [error] <0.1764.0> Error on AMQP connection <0.1764.0> (10.0.0.41:59288 -> 10.0.0.41:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-02-06 01:59:43.321 [error] <0.1892.0> Error on AMQP connection <0.1892.0> (10.0.0.23:42160 -> 10.0.0.41:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-02-06 01:59:43.321 [error] <0.1835.0> Error on AMQP connection <0.1835.0> (10.0.0.41:59310 -> 10.0.0.41:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-02-06 01:59:43.321 [error] <0.1806.0> Error on AMQP connection <0.1806.0> (10.0.0.41:59332 -> 10.0.0.41:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-02-06 01:59:43.321 [error] <0.1787.0> Error on AMQP connection <0.1787.0> (10.0.0.38:48298 -> 10.0.0.41:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-02-06 01:59:43.321 [error] <0.1767.0> Error on AMQP connection <0.1767.0> (10.0.0.38:48294 -> 10.0.0.41:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-02-06 01:59:43.321 [error] <0.1737.0> Error on AMQP connection <0.1737.0> (10.0.0.41:59286 -> 10.0.0.41:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-02-06 01:59:43.321 [error] <0.1773.0> Error on AMQP connection <0.1773.0> (10.0.0.23:42132 -> 10.0.0.41:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-02-06 01:59:43.321 [error] <0.1850.0> Error on AMQP connection <0.1850.0> (10.0.0.38:48312 -> 10.0.0.41:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-02-06 01:59:43.321 [error] <0.1870.0> Error on AMQP connection <0.1870.0> (10.0.0.41:59312 -> 10.0.0.41:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-02-06 01:59:43.321 [error] <0.1770.0> Error on AMQP connection <0.1770.0> (10.0.0.23:42120 -> 10.0.0.41:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-02-06 01:59:43.321 [error] <0.1793.0> Error on AMQP connection <0.1793.0> (10.0.0.38:48304 -> 10.0.0.41:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-02-06 01:59:43.321 [error] <0.1784.0> Error on AMQP connection <0.1784.0> (10.0.0.41:59308 -> 10.0.0.41:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-02-06 01:59:43.321 [error] <0.1886.0> Error on AMQP connection <0.1886.0> (10.0.0.41:59318 -> 10.0.0.41:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-02-06 01:59:43.321 [error] <0.1758.0> Error on AMQP connection <0.1758.0> (10.0.0.38:48284 -> 10.0.0.41:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-02-06 01:59:43.321 [error] <0.1874.0> Error on AMQP connection <0.1874.0> (10.0.0.41:59316 -> 10.0.0.41:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-02-06 01:59:43.321 [error] <0.2035.0> Error on AMQP connection <0.2035.0> (10.0.0.38:48318 -> 10.0.0.41:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-02-06 01:59:43.321 [error] <0.1761.0> Error on AMQP connection <0.1761.0> (10.0.0.38:48286 -> 10.0.0.41:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-02-06 01:59:43.321 [error] <0.1790.0> Error on AMQP connection <0.1790.0> (10.0.0.23:42134 -> 10.0.0.41:5673, vhost: '/', user: 'guest', state: running), channel 0: operation none caused a connection exception connection_forced: "broker forced connection closure with reason 'shutdown'" 2025-02-06 01:59:43.329 [error] <0.1822.0> Supervisor {<0.1822.0>,rabbit_channel_sup_sup} had child channel_sup started with rabbit_channel_sup:start_link() at undefined exit with reason shutdown in context shutdown_error 2025-02-06 01:59:43.330 [error] <0.1811.0> Supervisor {<0.1811.0>,rabbit_channel_sup_sup} had child channel_sup started with rabbit_channel_sup:start_link() at undefined exit with reason shutdown in context shutdown_error 2025-02-06 01:59:43.330 [error] <0.1856.0> Supervisor {<0.1856.0>,rabbit_channel_sup_sup} had child channel_sup started with rabbit_channel_sup:start_link() at undefined exit with reason shutdown in context shutdown_error 2025-02-06 01:59:43.331 [error] <0.1834.0> Supervisor {<0.1834.0>,rabbit_channel_sup_sup} had child channel_sup started with rabbit_channel_sup:start_link() at undefined exit with reason shutdown in context shutdown_error 2025-02-06 02:00:14.251 [info] <0.8.0> Log file opened with Lager 2025-02-06 02:00:16.909 [info] <0.8.0> Feature flags: list of feature flags found: 2025-02-06 02:00:16.909 [info] <0.8.0> Feature flags: feature flag states written to disk: yes 2025-02-06 02:00:16.931 [info] <0.249.0> Starting RabbitMQ 3.7.28 on Erlang 21.3.8.21 Copyright (c) 2007-2020 Pivotal Software, Inc. Licensed under the MPL. See https://www.rabbitmq.com/ 2025-02-06 02:00:16.932 [info] <0.249.0> node : contrail@cn-jenkins-deploy-platform-ansible-os-2696-2 home dir : /var/lib/rabbitmq config file(s) : /etc/rabbitmq/rabbitmq.config cookie hash : esZsDxSN6VGbi9JkMSxNZA== log(s) : /var/log/contrail/config-database-rabbitmq/rabbitmq.log : /var/log/rabbitmq/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2_upgrade.log database dir : /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2 2025-02-06 02:00:16.944 [info] <0.249.0> Running boot step pre_boot defined by app rabbit 2025-02-06 02:00:16.944 [info] <0.249.0> Running boot step rabbit_core_metrics defined by app rabbit 2025-02-06 02:00:16.945 [info] <0.249.0> Running boot step rabbit_alarm defined by app rabbit 2025-02-06 02:00:16.954 [info] <0.255.0> Memory high watermark set to 12637 MiB (13251077734 bytes) of 15796 MiB (16563847168 bytes) total 2025-02-06 02:00:16.959 [info] <0.257.0> Enabling free disk space monitoring 2025-02-06 02:00:16.959 [info] <0.257.0> Disk free limit set to 50MB 2025-02-06 02:00:16.966 [info] <0.249.0> Running boot step code_server_cache defined by app rabbit 2025-02-06 02:00:16.966 [info] <0.249.0> Running boot step file_handle_cache defined by app rabbit 2025-02-06 02:00:16.966 [info] <0.260.0> Limiting to approx 65436 file handles (58890 sockets) 2025-02-06 02:00:16.966 [info] <0.261.0> FHC read buffering: OFF 2025-02-06 02:00:16.966 [info] <0.261.0> FHC write buffering: ON 2025-02-06 02:00:16.968 [info] <0.249.0> Running boot step worker_pool defined by app rabbit 2025-02-06 02:00:16.968 [info] <0.250.0> Will use 4 processes for default worker pool 2025-02-06 02:00:16.968 [info] <0.250.0> Starting worker pool 'worker_pool' with 4 processes in it 2025-02-06 02:00:16.968 [info] <0.249.0> Running boot step database defined by app rabbit 2025-02-06 02:00:16.969 [info] <0.249.0> Node database directory at /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2 is empty. Assuming we need to join an existing cluster or initialise from scratch... 2025-02-06 02:00:16.969 [info] <0.249.0> Configured peer discovery backend: rabbit_peer_discovery_classic_config 2025-02-06 02:00:16.969 [info] <0.249.0> Will try to lock with peer discovery backend rabbit_peer_discovery_classic_config 2025-02-06 02:00:16.969 [info] <0.249.0> Peer discovery backend does not support locking, falling back to randomized delay 2025-02-06 02:00:16.969 [info] <0.249.0> Peer discovery backend rabbit_peer_discovery_classic_config supports registration. 2025-02-06 02:00:16.970 [info] <0.249.0> Will wait for 5000 milliseconds before proceeding with registration... 2025-02-06 02:00:21.971 [info] <0.249.0> All discovered existing cluster peers: contrail@cn-jenkins-deploy-platform-ansible-os-2696-1, contrail@cn-jenkins-deploy-platform-ansible-os-2696-2, contrail@cn-jenkins-deploy-platform-ansible-os-2696-3 2025-02-06 02:00:21.971 [info] <0.249.0> Peer nodes we can cluster with: contrail@cn-jenkins-deploy-platform-ansible-os-2696-1, contrail@cn-jenkins-deploy-platform-ansible-os-2696-3 2025-02-06 02:00:21.975 [info] <0.249.0> Node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-1' selected for auto-clustering 2025-02-06 02:00:22.013 [info] <0.249.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-02-06 02:00:22.201 [info] <0.249.0> Successfully synced tables from a peer 2025-02-06 02:00:22.202 [info] <0.249.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-02-06 02:00:22.203 [info] <0.249.0> Successfully synced tables from a peer 2025-02-06 02:00:22.417 [info] <0.249.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-02-06 02:00:22.417 [info] <0.249.0> Successfully synced tables from a peer 2025-02-06 02:00:22.418 [info] <0.249.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2696-2' 2025-02-06 02:00:22.418 [info] <0.249.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2696-2' 2025-02-06 02:00:22.419 [info] <0.249.0> Will register with peer discovery backend rabbit_peer_discovery_classic_config 2025-02-06 02:00:22.419 [info] <0.249.0> Running boot step database_sync defined by app rabbit 2025-02-06 02:00:22.419 [info] <0.249.0> Running boot step feature_flags defined by app rabbit 2025-02-06 02:00:22.424 [info] <0.249.0> Running boot step codec_correctness_check defined by app rabbit 2025-02-06 02:00:22.424 [info] <0.249.0> Running boot step external_infrastructure defined by app rabbit 2025-02-06 02:00:22.424 [info] <0.249.0> Running boot step rabbit_registry defined by app rabbit 2025-02-06 02:00:22.424 [info] <0.249.0> Running boot step rabbit_auth_mechanism_cr_demo defined by app rabbit 2025-02-06 02:00:22.424 [info] <0.249.0> Running boot step rabbit_queue_location_random defined by app rabbit 2025-02-06 02:00:22.424 [info] <0.249.0> Running boot step rabbit_event defined by app rabbit 2025-02-06 02:00:22.424 [info] <0.249.0> Running boot step rabbit_auth_mechanism_amqplain defined by app rabbit 2025-02-06 02:00:22.424 [info] <0.249.0> Running boot step rabbit_auth_mechanism_plain defined by app rabbit 2025-02-06 02:00:22.424 [info] <0.249.0> Running boot step rabbit_exchange_type_direct defined by app rabbit 2025-02-06 02:00:22.425 [info] <0.249.0> Running boot step rabbit_exchange_type_fanout defined by app rabbit 2025-02-06 02:00:22.425 [info] <0.249.0> Running boot step rabbit_exchange_type_headers defined by app rabbit 2025-02-06 02:00:22.425 [info] <0.249.0> Running boot step rabbit_exchange_type_topic defined by app rabbit 2025-02-06 02:00:22.425 [info] <0.249.0> Running boot step rabbit_mirror_queue_mode_all defined by app rabbit 2025-02-06 02:00:22.425 [info] <0.249.0> Running boot step rabbit_mirror_queue_mode_exactly defined by app rabbit 2025-02-06 02:00:22.425 [info] <0.249.0> Running boot step rabbit_mirror_queue_mode_nodes defined by app rabbit 2025-02-06 02:00:22.425 [info] <0.249.0> Running boot step rabbit_priority_queue defined by app rabbit 2025-02-06 02:00:22.425 [info] <0.249.0> Priority queues enabled, real BQ is rabbit_variable_queue 2025-02-06 02:00:22.425 [info] <0.249.0> Running boot step rabbit_queue_location_client_local defined by app rabbit 2025-02-06 02:00:22.425 [info] <0.249.0> Running boot step rabbit_queue_location_min_masters defined by app rabbit 2025-02-06 02:00:22.425 [info] <0.249.0> Running boot step kernel_ready defined by app rabbit 2025-02-06 02:00:22.425 [info] <0.249.0> Running boot step rabbit_sysmon_minder defined by app rabbit 2025-02-06 02:00:22.425 [info] <0.249.0> Running boot step rabbit_epmd_monitor defined by app rabbit 2025-02-06 02:00:22.469 [info] <0.383.0> epmd monitor knows us, inter-node communication (distribution) port: 25673 2025-02-06 02:00:22.469 [info] <0.249.0> Running boot step guid_generator defined by app rabbit 2025-02-06 02:00:22.496 [info] <0.249.0> Running boot step rabbit_node_monitor defined by app rabbit 2025-02-06 02:00:22.497 [info] <0.389.0> Starting rabbit_node_monitor 2025-02-06 02:00:22.500 [info] <0.249.0> Running boot step delegate_sup defined by app rabbit 2025-02-06 02:00:22.502 [info] <0.249.0> Running boot step rabbit_memory_monitor defined by app rabbit 2025-02-06 02:00:22.505 [info] <0.249.0> Running boot step core_initialized defined by app rabbit 2025-02-06 02:00:22.509 [info] <0.249.0> Running boot step upgrade_queues defined by app rabbit 2025-02-06 02:00:22.573 [info] <0.249.0> message_store upgrades: 1 to apply 2025-02-06 02:00:22.575 [info] <0.249.0> message_store upgrades: Applying rabbit_variable_queue:move_messages_to_vhost_store 2025-02-06 02:00:22.575 [info] <0.249.0> message_store upgrades: No durable queues found. Skipping message store migration 2025-02-06 02:00:22.575 [info] <0.249.0> message_store upgrades: Removing the old message store data 2025-02-06 02:00:22.582 [info] <0.249.0> message_store upgrades: All upgrades applied successfully 2025-02-06 02:00:22.654 [info] <0.249.0> Running boot step rabbit_connection_tracking defined by app rabbit 2025-02-06 02:00:22.655 [info] <0.249.0> Running boot step rabbit_connection_tracking_handler defined by app rabbit 2025-02-06 02:00:22.655 [info] <0.249.0> Running boot step rabbit_exchange_parameters defined by app rabbit 2025-02-06 02:00:22.655 [info] <0.249.0> Running boot step rabbit_mirror_queue_misc defined by app rabbit 2025-02-06 02:00:22.655 [info] <0.249.0> Running boot step rabbit_policies defined by app rabbit 2025-02-06 02:00:22.659 [info] <0.249.0> Running boot step rabbit_policy defined by app rabbit 2025-02-06 02:00:22.659 [info] <0.249.0> Running boot step rabbit_queue_location_validator defined by app rabbit 2025-02-06 02:00:22.660 [info] <0.249.0> Running boot step rabbit_vhost_limit defined by app rabbit 2025-02-06 02:00:22.660 [info] <0.249.0> Running boot step rabbit_mgmt_reset_handler defined by app rabbitmq_management 2025-02-06 02:00:22.660 [info] <0.249.0> Running boot step rabbit_mgmt_db_handler defined by app rabbitmq_management_agent 2025-02-06 02:00:22.660 [warning] <0.249.0> force_fine_statistics set to true; ignored. Replaced by {rates_mode, none} in the rabbitmq_management application. 2025-02-06 02:00:22.660 [info] <0.249.0> Management plugin: using rates mode 'basic' 2025-02-06 02:00:22.664 [info] <0.249.0> Running boot step recovery defined by app rabbit 2025-02-06 02:00:22.665 [info] <0.426.0> Making sure data directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists 2025-02-06 02:00:22.684 [info] <0.426.0> Starting message stores for vhost '/' 2025-02-06 02:00:22.685 [info] <0.430.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index 2025-02-06 02:00:22.689 [info] <0.426.0> Started message store of type transient for vhost '/' 2025-02-06 02:00:22.689 [info] <0.433.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index 2025-02-06 02:00:22.692 [warning] <0.433.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": rebuilding indices from scratch 2025-02-06 02:00:22.700 [info] <0.426.0> Started message store of type persistent for vhost '/' 2025-02-06 02:00:22.702 [info] <0.426.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-1.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.452.0> 2025-02-06 02:00:22.703 [info] <0.426.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-2.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2696-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.456.0> 2025-02-06 02:00:22.703 [info] <0.426.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2696-1' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.460.0> 2025-02-06 02:00:22.704 [info] <0.426.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-2.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2696-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.464.0> 2025-02-06 02:00:22.704 [info] <0.426.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.468.0> 2025-02-06 02:00:22.704 [info] <0.426.0> Mirrored queue 'job_status_consumer.dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.472.0> 2025-02-06 02:00:22.704 [info] <0.426.0> Mirrored queue 'device_manager.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.476.0> 2025-02-06 02:00:22.705 [info] <0.426.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2696-2' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.480.0> 2025-02-06 02:00:22.705 [info] <0.426.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-2..tftp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.486.0> 2025-02-06 02:00:22.705 [info] <0.426.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-1..config_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.490.0> 2025-02-06 02:00:22.706 [info] <0.426.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-3.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.495.0> 2025-02-06 02:00:22.707 [info] <0.426.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2696-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.498.0> 2025-02-06 02:00:22.707 [info] <0.426.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.504.0> 2025-02-06 02:00:22.707 [info] <0.426.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.508.0> 2025-02-06 02:00:22.708 [info] <0.426.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-3..config_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.512.0> 2025-02-06 02:00:22.708 [info] <0.426.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2696-1..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.516.0> 2025-02-06 02:00:22.708 [info] <0.426.0> Mirrored queue 'job_request_consumer' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.520.0> 2025-02-06 02:00:22.708 [info] <0.426.0> Mirrored queue 'schema_transformer.cn-jenkins-deploy-platform-ansible-os-2696-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.524.0> 2025-02-06 02:00:22.708 [info] <0.426.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2696-3.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.528.0> 2025-02-06 02:00:22.709 [info] <0.426.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-1..tftp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.532.0> 2025-02-06 02:00:22.709 [info] <0.426.0> Mirrored queue 'svc_monitor.cn-jenkins-deploy-platform-ansible-os-2696-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.536.0> 2025-02-06 02:00:22.709 [info] <0.426.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2696-1.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.540.0> 2025-02-06 02:00:22.709 [info] <0.426.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2696-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.544.0> 2025-02-06 02:00:22.709 [info] <0.426.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2696-3' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.548.0> 2025-02-06 02:00:22.709 [info] <0.426.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2696-2' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.552.0> 2025-02-06 02:00:22.710 [info] <0.426.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2696-2.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.556.0> 2025-02-06 02:00:22.710 [info] <0.426.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2696-2..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.560.0> 2025-02-06 02:00:22.710 [info] <0.426.0> Mirrored queue 'device_manager.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.564.0> 2025-02-06 02:00:22.710 [info] <0.426.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-3.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.568.0> 2025-02-06 02:00:22.710 [info] <0.426.0> Mirrored queue 'device_manager_ztp.ztp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.572.0> 2025-02-06 02:00:22.710 [info] <0.426.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-1.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.576.0> 2025-02-06 02:00:22.711 [info] <0.426.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-2.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2696-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.580.0> 2025-02-06 02:00:22.711 [info] <0.426.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-2..config_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.584.0> 2025-02-06 02:00:22.711 [info] <0.426.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2696-1' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.588.0> 2025-02-06 02:00:22.711 [info] <0.426.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-3..tftp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.592.0> 2025-02-06 02:00:22.711 [info] <0.426.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2696-3' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.596.0> 2025-02-06 02:00:22.712 [info] <0.426.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-1.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.600.0> 2025-02-06 02:00:22.712 [info] <0.426.0> Mirrored queue 'device_manager.cn-jenkins-deploy-platform-ansible-os-2696-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.604.0> 2025-02-06 02:00:22.712 [info] <0.426.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.608.0> 2025-02-06 02:00:22.712 [info] <0.426.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-3.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.612.0> 2025-02-06 02:00:22.712 [info] <0.426.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2696-3..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.616.0> 2025-02-06 02:00:22.715 [info] <0.249.0> Running boot step load_definitions defined by app rabbitmq_management 2025-02-06 02:00:22.716 [info] <0.249.0> Applying definitions from /etc/rabbitmq/definitions.json 2025-02-06 02:00:22.716 [info] <0.249.0> Asked to import definitions. Acting user: rmq-internal 2025-02-06 02:00:22.716 [info] <0.249.0> Importing users... 2025-02-06 02:00:22.983 [info] <0.249.0> Setting user tags for user 'guest' to [administrator] 2025-02-06 02:00:23.010 [info] <0.249.0> Importing vhosts... 2025-02-06 02:00:23.010 [info] <0.249.0> Importing user permissions... 2025-02-06 02:00:23.010 [info] <0.249.0> Setting permissions for 'guest' in '/' to '.*', '.*', '.*' 2025-02-06 02:00:23.087 [info] <0.249.0> Importing topic permissions... 2025-02-06 02:00:23.087 [info] <0.249.0> Importing parameters... 2025-02-06 02:00:23.087 [info] <0.249.0> Importing global parameters... 2025-02-06 02:00:23.087 [info] <0.249.0> Importing policies... 2025-02-06 02:00:23.093 [info] <0.249.0> Importing queues... 2025-02-06 02:00:23.093 [info] <0.249.0> Importing exchanges... 2025-02-06 02:00:23.093 [info] <0.249.0> Importing bindings... 2025-02-06 02:00:23.093 [info] <0.249.0> Running boot step empty_db_check defined by app rabbit 2025-02-06 02:00:23.093 [info] <0.249.0> Running boot step rabbit_looking_glass defined by app rabbit 2025-02-06 02:00:23.093 [info] <0.249.0> Running boot step rabbit_core_metrics_gc defined by app rabbit 2025-02-06 02:00:23.094 [info] <0.249.0> Running boot step background_gc defined by app rabbit 2025-02-06 02:00:23.094 [info] <0.249.0> Running boot step connection_tracking defined by app rabbit 2025-02-06 02:00:23.094 [info] <0.249.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2696-2' 2025-02-06 02:00:23.095 [info] <0.249.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2696-2' 2025-02-06 02:00:23.095 [info] <0.249.0> Running boot step routing_ready defined by app rabbit 2025-02-06 02:00:23.095 [info] <0.249.0> Running boot step pre_flight defined by app rabbit 2025-02-06 02:00:23.095 [info] <0.249.0> Running boot step notify_cluster defined by app rabbit 2025-02-06 02:00:23.095 [info] <0.249.0> Running boot step networking defined by app rabbit 2025-02-06 02:00:23.096 [info] <0.389.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-3' up 2025-02-06 02:00:23.096 [info] <0.772.0> started TCP listener on [::]:5673 2025-02-06 02:00:23.097 [info] <0.249.0> Running boot step cluster_name defined by app rabbit 2025-02-06 02:00:23.097 [info] <0.249.0> Running boot step direct_client defined by app rabbit 2025-02-06 02:00:23.097 [info] <0.249.0> Running boot step os_signal_handler defined by app rabbit 2025-02-06 02:00:23.097 [info] <0.774.0> Swapping OS signal event handler (erl_signal_server) for our own 2025-02-06 02:00:23.213 [info] <0.389.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-1' up 2025-02-06 02:00:23.267 [info] <0.908.0> Management plugin: HTTP (non-TLS) listener started on port 15673 2025-02-06 02:00:23.267 [info] <0.1014.0> Statistics database started. 2025-02-06 02:00:23.267 [info] <0.1013.0> Starting worker pool 'management_worker_pool' with 3 processes in it 2025-02-06 02:00:23.268 [notice] <0.106.0> Changed loghwm of /var/log/contrail/config-database-rabbitmq/rabbitmq.log to 50 2025-02-06 02:00:23.422 [info] <0.8.0> Server startup complete; 3 plugins started. * rabbitmq_management * rabbitmq_management_agent * rabbitmq_web_dispatch 2025-02-06 02:00:23.837 [info] <0.1027.0> RabbitMQ is asked to stop... 2025-02-06 02:00:23.899 [info] <0.1027.0> Stopping RabbitMQ applications and their dependencies in the following order: rabbitmq_management amqp_client rabbitmq_web_dispatch cowboy cowlib rabbitmq_management_agent rabbit mnesia rabbit_common sysmon_handler os_mon 2025-02-06 02:00:23.899 [info] <0.1027.0> Stopping application 'rabbitmq_management' 2025-02-06 02:00:23.901 [warning] <0.780.0> RabbitMQ HTTP listener registry could not find context rabbitmq_management_tls 2025-02-06 02:00:23.903 [info] <0.43.0> Application rabbitmq_management exited with reason: stopped 2025-02-06 02:00:23.903 [info] <0.1027.0> Stopping application 'amqp_client' 2025-02-06 02:00:23.905 [info] <0.43.0> Application amqp_client exited with reason: stopped 2025-02-06 02:00:23.905 [info] <0.1027.0> Stopping application 'rabbitmq_web_dispatch' 2025-02-06 02:00:23.906 [info] <0.1027.0> Stopping application 'cowboy' 2025-02-06 02:00:23.906 [info] <0.43.0> Application rabbitmq_web_dispatch exited with reason: stopped 2025-02-06 02:00:23.907 [info] <0.1027.0> Stopping application 'cowlib' 2025-02-06 02:00:23.907 [info] <0.43.0> Application cowboy exited with reason: stopped 2025-02-06 02:00:23.907 [info] <0.1027.0> Stopping application 'rabbitmq_management_agent' 2025-02-06 02:00:23.907 [info] <0.43.0> Application cowlib exited with reason: stopped 2025-02-06 02:00:23.908 [info] <0.1027.0> Stopping application 'rabbit' 2025-02-06 02:00:23.908 [info] <0.43.0> Application rabbitmq_management_agent exited with reason: stopped 2025-02-06 02:00:23.908 [info] <0.249.0> Will unregister with peer discovery backend rabbit_peer_discovery_classic_config 2025-02-06 02:00:23.908 [info] <0.772.0> stopped TCP listener on [::]:5673 2025-02-06 02:00:23.909 [info] <0.414.0> Closing all connections in vhost '/' on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2' because the vhost is stopping 2025-02-06 02:00:23.910 [info] <0.433.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' 2025-02-06 02:00:23.916 [info] <0.433.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent' is stopped 2025-02-06 02:00:23.917 [info] <0.430.0> Stopping message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' 2025-02-06 02:00:23.920 [info] <0.430.0> Message store for directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L/msg_store_transient' is stopped 2025-02-06 02:00:23.926 [info] <0.1027.0> Stopping application 'mnesia' 2025-02-06 02:00:23.926 [info] <0.43.0> Application rabbit exited with reason: stopped 2025-02-06 02:00:23.928 [info] <0.1027.0> Stopping application 'rabbit_common' 2025-02-06 02:00:23.928 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-02-06 02:00:23.928 [info] <0.1027.0> Stopping application 'sysmon_handler' 2025-02-06 02:00:23.928 [info] <0.43.0> Application rabbit_common exited with reason: stopped 2025-02-06 02:00:23.929 [info] <0.1027.0> Stopping application 'os_mon' 2025-02-06 02:00:23.929 [info] <0.43.0> Application sysmon_handler exited with reason: stopped 2025-02-06 02:00:23.931 [info] <0.1027.0> Successfully stopped RabbitMQ and its dependencies 2025-02-06 02:00:23.931 [info] <0.43.0> Application os_mon exited with reason: stopped 2025-02-06 02:00:23.931 [info] <0.1027.0> Halting Erlang VM with the following applications: ranch ssl public_key asn1 observer_cli tools credentials_obfuscation crypto stdout_formatter recon inets jsx xmerl lager goldrush compiler syntax_tools sasl stdlib kernel 2025-02-06 02:00:27.884 [info] <0.8.0> Log file opened with Lager 2025-02-06 02:00:29.880 [info] <0.8.0> Feature flags: list of feature flags found: 2025-02-06 02:00:29.880 [info] <0.8.0> Feature flags: feature flag states written to disk: yes 2025-02-06 02:00:29.960 [info] <0.43.0> Application mnesia exited with reason: stopped 2025-02-06 02:00:30.022 [info] <0.302.0> Starting RabbitMQ 3.7.28 on Erlang 21.3.8.21 Copyright (c) 2007-2020 Pivotal Software, Inc. Licensed under the MPL. See https://www.rabbitmq.com/ 2025-02-06 02:00:30.022 [info] <0.302.0> node : contrail@cn-jenkins-deploy-platform-ansible-os-2696-2 home dir : /var/lib/rabbitmq config file(s) : /etc/rabbitmq/rabbitmq.config cookie hash : esZsDxSN6VGbi9JkMSxNZA== log(s) : /var/log/contrail/config-database-rabbitmq/rabbitmq.log : /var/log/rabbitmq/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2_upgrade.log database dir : /var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2 2025-02-06 02:00:30.034 [info] <0.302.0> Running boot step pre_boot defined by app rabbit 2025-02-06 02:00:30.034 [info] <0.302.0> Running boot step rabbit_core_metrics defined by app rabbit 2025-02-06 02:00:30.035 [info] <0.302.0> Running boot step rabbit_alarm defined by app rabbit 2025-02-06 02:00:30.042 [info] <0.308.0> Memory high watermark set to 12637 MiB (13251077734 bytes) of 15796 MiB (16563847168 bytes) total 2025-02-06 02:00:30.055 [info] <0.317.0> Enabling free disk space monitoring 2025-02-06 02:00:30.055 [info] <0.317.0> Disk free limit set to 50MB 2025-02-06 02:00:30.066 [info] <0.302.0> Running boot step code_server_cache defined by app rabbit 2025-02-06 02:00:30.066 [info] <0.302.0> Running boot step file_handle_cache defined by app rabbit 2025-02-06 02:00:30.066 [info] <0.331.0> Limiting to approx 65436 file handles (58890 sockets) 2025-02-06 02:00:30.066 [info] <0.332.0> FHC read buffering: OFF 2025-02-06 02:00:30.067 [info] <0.332.0> FHC write buffering: ON 2025-02-06 02:00:30.067 [info] <0.302.0> Running boot step worker_pool defined by app rabbit 2025-02-06 02:00:30.067 [info] <0.303.0> Will use 4 processes for default worker pool 2025-02-06 02:00:30.067 [info] <0.303.0> Starting worker pool 'worker_pool' with 4 processes in it 2025-02-06 02:00:30.067 [info] <0.302.0> Running boot step database defined by app rabbit 2025-02-06 02:00:30.099 [info] <0.302.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-02-06 02:00:30.116 [info] <0.302.0> Successfully synced tables from a peer 2025-02-06 02:00:30.117 [info] <0.302.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-02-06 02:00:30.117 [info] <0.302.0> Successfully synced tables from a peer 2025-02-06 02:00:30.151 [info] <0.302.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-02-06 02:00:30.151 [info] <0.302.0> Successfully synced tables from a peer 2025-02-06 02:00:30.151 [info] <0.302.0> Will register with peer discovery backend rabbit_peer_discovery_classic_config 2025-02-06 02:00:30.188 [info] <0.302.0> Running boot step database_sync defined by app rabbit 2025-02-06 02:00:30.188 [info] <0.302.0> Running boot step feature_flags defined by app rabbit 2025-02-06 02:00:30.189 [info] <0.302.0> Running boot step codec_correctness_check defined by app rabbit 2025-02-06 02:00:30.189 [info] <0.302.0> Running boot step external_infrastructure defined by app rabbit 2025-02-06 02:00:30.189 [info] <0.302.0> Running boot step rabbit_registry defined by app rabbit 2025-02-06 02:00:30.189 [info] <0.302.0> Running boot step rabbit_auth_mechanism_cr_demo defined by app rabbit 2025-02-06 02:00:30.189 [info] <0.302.0> Running boot step rabbit_queue_location_random defined by app rabbit 2025-02-06 02:00:30.189 [info] <0.302.0> Running boot step rabbit_event defined by app rabbit 2025-02-06 02:00:30.189 [info] <0.302.0> Running boot step rabbit_auth_mechanism_amqplain defined by app rabbit 2025-02-06 02:00:30.189 [info] <0.302.0> Running boot step rabbit_auth_mechanism_plain defined by app rabbit 2025-02-06 02:00:30.189 [info] <0.302.0> Running boot step rabbit_exchange_type_direct defined by app rabbit 2025-02-06 02:00:30.189 [info] <0.302.0> Running boot step rabbit_exchange_type_fanout defined by app rabbit 2025-02-06 02:00:30.189 [info] <0.302.0> Running boot step rabbit_exchange_type_headers defined by app rabbit 2025-02-06 02:00:30.189 [info] <0.302.0> Running boot step rabbit_exchange_type_topic defined by app rabbit 2025-02-06 02:00:30.189 [info] <0.302.0> Running boot step rabbit_mirror_queue_mode_all defined by app rabbit 2025-02-06 02:00:30.190 [info] <0.302.0> Running boot step rabbit_mirror_queue_mode_exactly defined by app rabbit 2025-02-06 02:00:30.190 [info] <0.302.0> Running boot step rabbit_mirror_queue_mode_nodes defined by app rabbit 2025-02-06 02:00:30.190 [info] <0.302.0> Running boot step rabbit_priority_queue defined by app rabbit 2025-02-06 02:00:30.190 [info] <0.302.0> Priority queues enabled, real BQ is rabbit_variable_queue 2025-02-06 02:00:30.190 [info] <0.302.0> Running boot step rabbit_queue_location_client_local defined by app rabbit 2025-02-06 02:00:30.190 [info] <0.302.0> Running boot step rabbit_queue_location_min_masters defined by app rabbit 2025-02-06 02:00:30.190 [info] <0.302.0> Running boot step kernel_ready defined by app rabbit 2025-02-06 02:00:30.190 [info] <0.302.0> Running boot step rabbit_sysmon_minder defined by app rabbit 2025-02-06 02:00:30.190 [info] <0.302.0> Running boot step rabbit_epmd_monitor defined by app rabbit 2025-02-06 02:00:30.191 [info] <0.414.0> epmd monitor knows us, inter-node communication (distribution) port: 25673 2025-02-06 02:00:30.191 [info] <0.302.0> Running boot step guid_generator defined by app rabbit 2025-02-06 02:00:30.198 [info] <0.302.0> Running boot step rabbit_node_monitor defined by app rabbit 2025-02-06 02:00:30.208 [info] <0.418.0> Starting rabbit_node_monitor 2025-02-06 02:00:30.209 [info] <0.302.0> Running boot step delegate_sup defined by app rabbit 2025-02-06 02:00:30.209 [info] <0.302.0> Running boot step rabbit_memory_monitor defined by app rabbit 2025-02-06 02:00:30.209 [info] <0.302.0> Running boot step core_initialized defined by app rabbit 2025-02-06 02:00:30.209 [info] <0.302.0> Running boot step upgrade_queues defined by app rabbit 2025-02-06 02:00:30.229 [info] <0.302.0> Running boot step rabbit_connection_tracking defined by app rabbit 2025-02-06 02:00:30.229 [info] <0.302.0> Running boot step rabbit_connection_tracking_handler defined by app rabbit 2025-02-06 02:00:30.229 [info] <0.302.0> Running boot step rabbit_exchange_parameters defined by app rabbit 2025-02-06 02:00:30.229 [info] <0.302.0> Running boot step rabbit_mirror_queue_misc defined by app rabbit 2025-02-06 02:00:30.229 [info] <0.302.0> Running boot step rabbit_policies defined by app rabbit 2025-02-06 02:00:30.230 [info] <0.302.0> Running boot step rabbit_policy defined by app rabbit 2025-02-06 02:00:30.230 [info] <0.302.0> Running boot step rabbit_queue_location_validator defined by app rabbit 2025-02-06 02:00:30.230 [info] <0.302.0> Running boot step rabbit_vhost_limit defined by app rabbit 2025-02-06 02:00:30.230 [info] <0.302.0> Running boot step rabbit_mgmt_reset_handler defined by app rabbitmq_management 2025-02-06 02:00:30.230 [info] <0.302.0> Running boot step rabbit_mgmt_db_handler defined by app rabbitmq_management_agent 2025-02-06 02:00:30.230 [warning] <0.302.0> force_fine_statistics set to true; ignored. Replaced by {rates_mode, none} in the rabbitmq_management application. 2025-02-06 02:00:30.230 [info] <0.302.0> Management plugin: using rates mode 'basic' 2025-02-06 02:00:30.232 [info] <0.302.0> Running boot step recovery defined by app rabbit 2025-02-06 02:00:30.233 [info] <0.455.0> Making sure data directory '/var/lib/rabbitmq/mnesia/contrail@cn-jenkins-deploy-platform-ansible-os-2696-2/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists 2025-02-06 02:00:30.236 [info] <0.455.0> Starting message stores for vhost '/' 2025-02-06 02:00:30.236 [info] <0.459.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index 2025-02-06 02:00:30.237 [info] <0.455.0> Started message store of type transient for vhost '/' 2025-02-06 02:00:30.237 [info] <0.462.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index 2025-02-06 02:00:30.238 [info] <0.455.0> Started message store of type persistent for vhost '/' 2025-02-06 02:00:30.240 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-1.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.481.0> 2025-02-06 02:00:30.240 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-2.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2696-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.485.0> 2025-02-06 02:00:30.240 [info] <0.455.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2696-1' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.489.0> 2025-02-06 02:00:30.240 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-2.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2696-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.493.0> 2025-02-06 02:00:30.241 [info] <0.455.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.497.0> 2025-02-06 02:00:30.241 [info] <0.455.0> Mirrored queue 'job_status_consumer.dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.501.0> 2025-02-06 02:00:30.241 [info] <0.455.0> Mirrored queue 'device_manager.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.505.0> 2025-02-06 02:00:30.241 [info] <0.455.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2696-2' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.509.0> 2025-02-06 02:00:30.242 [info] <0.455.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-2..tftp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.514.0> 2025-02-06 02:00:30.242 [info] <0.455.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-1..config_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.518.0> 2025-02-06 02:00:30.242 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-3.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.522.0> 2025-02-06 02:00:30.243 [info] <0.455.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2696-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.528.0> 2025-02-06 02:00:30.243 [info] <0.455.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.533.0> 2025-02-06 02:00:30.244 [info] <0.455.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.538.0> 2025-02-06 02:00:30.244 [info] <0.455.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-3..config_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.544.0> 2025-02-06 02:00:30.244 [info] <0.455.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2696-1..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.550.0> 2025-02-06 02:00:30.245 [info] <0.455.0> Mirrored queue 'job_request_consumer' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.554.0> 2025-02-06 02:00:30.245 [info] <0.455.0> Mirrored queue 'schema_transformer.cn-jenkins-deploy-platform-ansible-os-2696-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.559.0> 2025-02-06 02:00:30.245 [info] <0.455.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2696-3.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.564.0> 2025-02-06 02:00:30.246 [info] <0.455.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-1..tftp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.567.0> 2025-02-06 02:00:30.246 [info] <0.455.0> Mirrored queue 'svc_monitor.cn-jenkins-deploy-platform-ansible-os-2696-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.573.0> 2025-02-06 02:00:30.246 [info] <0.455.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2696-1.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.576.0> 2025-02-06 02:00:30.246 [info] <0.455.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2696-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.580.0> 2025-02-06 02:00:30.247 [info] <0.455.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2696-3' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.584.0> 2025-02-06 02:00:30.247 [info] <0.455.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2696-2' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.590.0> 2025-02-06 02:00:30.248 [info] <0.455.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2696-2.-8082-0' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.594.0> 2025-02-06 02:00:30.248 [info] <0.455.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2696-2..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.601.0> 2025-02-06 02:00:30.248 [info] <0.455.0> Mirrored queue 'device_manager.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.606.0> 2025-02-06 02:00:30.249 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-3.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.611.0> 2025-02-06 02:00:30.249 [info] <0.455.0> Mirrored queue 'device_manager_ztp.ztp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.616.0> 2025-02-06 02:00:30.250 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-1.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.620.0> 2025-02-06 02:00:30.250 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-2.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2696-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.624.0> 2025-02-06 02:00:30.251 [info] <0.455.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-2..config_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.629.0> 2025-02-06 02:00:30.251 [info] <0.455.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2696-1' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.634.0> 2025-02-06 02:00:30.251 [info] <0.455.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-3..tftp_queue' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.638.0> 2025-02-06 02:00:30.251 [info] <0.455.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2696-3' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.642.0> 2025-02-06 02:00:30.252 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-1.:contrail-topology:0.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.646.0> 2025-02-06 02:00:30.252 [info] <0.455.0> Mirrored queue 'device_manager.cn-jenkins-deploy-platform-ansible-os-2696-2.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.650.0> 2025-02-06 02:00:30.252 [info] <0.455.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.654.0> 2025-02-06 02:00:30.252 [info] <0.455.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-3.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.659.0> 2025-02-06 02:00:30.253 [info] <0.455.0> Mirrored queue 'amqp_request.cn-jenkins-deploy-platform-ansible-os-2696-3..dummy' in vhost '/': Adding mirror on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-2': <0.663.0> 2025-02-06 02:00:30.253 [info] <0.302.0> Running boot step load_definitions defined by app rabbitmq_management 2025-02-06 02:00:30.253 [info] <0.302.0> Applying definitions from /etc/rabbitmq/definitions.json 2025-02-06 02:00:30.253 [info] <0.302.0> Asked to import definitions. Acting user: rmq-internal 2025-02-06 02:00:30.253 [info] <0.302.0> Importing users... 2025-02-06 02:00:30.269 [info] <0.302.0> Setting user tags for user 'guest' to [administrator] 2025-02-06 02:00:30.320 [info] <0.302.0> Importing vhosts... 2025-02-06 02:00:30.320 [info] <0.302.0> Importing user permissions... 2025-02-06 02:00:30.320 [info] <0.302.0> Setting permissions for 'guest' in '/' to '.*', '.*', '.*' 2025-02-06 02:00:30.323 [info] <0.302.0> Importing topic permissions... 2025-02-06 02:00:30.323 [info] <0.302.0> Importing parameters... 2025-02-06 02:00:30.323 [info] <0.302.0> Importing global parameters... 2025-02-06 02:00:30.323 [info] <0.302.0> Importing policies... 2025-02-06 02:00:30.326 [info] <0.302.0> Importing queues... 2025-02-06 02:00:30.326 [info] <0.302.0> Importing exchanges... 2025-02-06 02:00:30.326 [info] <0.302.0> Importing bindings... 2025-02-06 02:00:30.326 [info] <0.302.0> Running boot step empty_db_check defined by app rabbit 2025-02-06 02:00:30.326 [info] <0.302.0> Running boot step rabbit_looking_glass defined by app rabbit 2025-02-06 02:00:30.326 [info] <0.302.0> Running boot step rabbit_core_metrics_gc defined by app rabbit 2025-02-06 02:00:30.326 [info] <0.302.0> Running boot step background_gc defined by app rabbit 2025-02-06 02:00:30.326 [info] <0.302.0> Running boot step connection_tracking defined by app rabbit 2025-02-06 02:00:30.327 [info] <0.302.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2696-2' 2025-02-06 02:00:30.327 [info] <0.302.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_contrail@cn-jenkins-deploy-platform-ansible-os-2696-2' 2025-02-06 02:00:30.327 [info] <0.302.0> Running boot step routing_ready defined by app rabbit 2025-02-06 02:00:30.327 [info] <0.302.0> Running boot step pre_flight defined by app rabbit 2025-02-06 02:00:30.327 [info] <0.302.0> Running boot step notify_cluster defined by app rabbit 2025-02-06 02:00:30.327 [info] <0.302.0> Running boot step networking defined by app rabbit 2025-02-06 02:00:30.328 [info] <0.418.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-1' up 2025-02-06 02:00:30.329 [info] <0.790.0> started TCP listener on [::]:5673 2025-02-06 02:00:30.329 [info] <0.302.0> Running boot step cluster_name defined by app rabbit 2025-02-06 02:00:30.329 [info] <0.302.0> Running boot step direct_client defined by app rabbit 2025-02-06 02:00:30.329 [info] <0.302.0> Running boot step os_signal_handler defined by app rabbit 2025-02-06 02:00:30.329 [info] <0.792.0> Swapping OS signal event handler (erl_signal_server) for our own 2025-02-06 02:00:30.344 [info] <0.418.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-3' up 2025-02-06 02:00:30.405 [info] <0.926.0> Management plugin: HTTP (non-TLS) listener started on port 15673 2025-02-06 02:00:30.405 [info] <0.1032.0> Statistics database started. 2025-02-06 02:00:30.405 [info] <0.1031.0> Starting worker pool 'management_worker_pool' with 3 processes in it 2025-02-06 02:00:30.406 [notice] <0.106.0> Changed loghwm of /var/log/contrail/config-database-rabbitmq/rabbitmq.log to 50 2025-02-06 02:00:30.547 [info] <0.8.0> Server startup complete; 3 plugins started. * rabbitmq_management * rabbitmq_management_agent * rabbitmq_web_dispatch 2025-02-06 02:01:11.126 [info] <0.654.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:01:11.130 [info] <0.634.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2696-1' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:01:11.130 [info] <0.594.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2696-2.-8082-0' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:01:11.130 [info] <0.481.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-1.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:01:11.140 [info] <0.584.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2696-3' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:01:11.141 [info] <0.418.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-3' down 2025-02-06 02:01:11.151 [info] <0.418.0> Keeping contrail@cn-jenkins-deploy-platform-ansible-os-2696-3 listeners: the node is already back 2025-02-06 02:01:11.166 [info] <0.580.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2696-2.' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:01:11.169 [info] <0.573.0> Mirrored queue 'svc_monitor.cn-jenkins-deploy-platform-ansible-os-2696-2.' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:01:11.170 [info] <0.522.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-3.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:01:11.173 [info] <0.497.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:01:11.173 [info] <0.418.0> node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-3' down: connection_closed 2025-02-06 02:01:11.174 [info] <0.564.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2696-3.-8082-0' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:01:11.175 [info] <0.650.0> Mirrored queue 'device_manager.cn-jenkins-deploy-platform-ansible-os-2696-2.' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:01:11.218 [info] <0.620.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-1.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:01:11.250 [info] <0.629.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-2..config_queue' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:01:11.250 [info] <0.616.0> Mirrored queue 'device_manager_ztp.ztp_queue' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:01:11.252 [info] <0.518.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-1..config_queue' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:01:11.253 [info] <0.567.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-1..tftp_queue' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:01:11.261 [info] <0.505.0> Mirrored queue 'device_manager.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:01:11.277 [info] <0.544.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-3..config_queue' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:01:42.259 [info] <0.418.0> node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-3' up 2025-02-06 02:02:21.247 [info] <0.418.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-3' up 2025-02-06 02:02:24.560 [info] <0.418.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-3' down 2025-02-06 02:02:24.570 [info] <0.418.0> Keeping contrail@cn-jenkins-deploy-platform-ansible-os-2696-3 listeners: the node is already back 2025-02-06 02:02:24.660 [info] <0.493.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-2.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2696-2.' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:02:24.661 [info] <0.634.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2696-1' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:02:24.674 [info] <0.573.0> Mirrored queue 'svc_monitor.cn-jenkins-deploy-platform-ansible-os-2696-2.' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:02:24.689 [info] <0.629.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-2..config_queue' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:02:24.689 [info] <0.638.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-3..tftp_queue' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:02:24.689 [info] <0.514.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-2..tftp_queue' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:02:24.691 [info] <0.567.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-1..tftp_queue' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:02:24.693 [info] <0.624.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-2.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2696-2.' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:02:24.694 [info] <0.594.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2696-2.-8082-0' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:02:24.694 [info] <0.576.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2696-1.-8082-0' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:02:24.695 [info] <0.564.0> Mirrored queue 'vnc_config.cn-jenkins-deploy-platform-ansible-os-2696-3.-8082-0' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:02:24.699 [info] <0.509.0> Mirrored queue 'contrail-control.cn-jenkins-deploy-platform-ansible-os-2696-2' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:02:24.701 [info] <0.518.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-1..config_queue' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:02:24.701 [info] <0.590.0> Mirrored queue 'contrail-dns.cn-jenkins-deploy-platform-ansible-os-2696-2' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:02:24.704 [info] <0.538.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:02:24.728 [info] <0.620.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-1.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2696-1.' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:02:24.728 [info] <0.544.0> Mirrored queue 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-2696-3..config_queue' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:02:24.729 [info] <0.606.0> Mirrored queue 'device_manager.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:02:24.729 [info] <0.533.0> Mirrored queue 'contrail-collector.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:02:24.729 [info] <0.580.0> Mirrored queue 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-2696-2.' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:02:24.730 [info] <0.522.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-3.:contrail-alarm-gen:0.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:02:24.730 [info] <0.559.0> Mirrored queue 'schema_transformer.cn-jenkins-deploy-platform-ansible-os-2696-2.' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:02:24.731 [info] <0.659.0> Mirrored queue 'cn-jenkins-deploy-platform-ansible-os-2696-3.:contrail-snmp-collector:0.cn-jenkins-deploy-platform-ansible-os-2696-3.' in vhost '/': Slave saw deaths of mirrors 2025-02-06 02:02:31.804 [info] <0.418.0> node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-3' down: connection_closed 2025-02-06 02:02:32.497 [info] <0.418.0> node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-3' up 2025-02-06 02:02:34.825 [info] <0.418.0> node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-3' down: connection_closed 2025-02-06 02:02:40.690 [info] <0.418.0> node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-3' up 2025-02-06 02:02:41.249 [info] <0.418.0> rabbit on node 'contrail@cn-jenkins-deploy-platform-ansible-os-2696-3' up