02/03/2025 00:44:57.732 7f39e3235df0 [contrail-config-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 02/03/2025 00:44:57.732 7f39e3235df0 [contrail-config-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 02/03/2025 00:44:57.732 7f39e3235df0 [contrail-config-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 02/03/2025 00:44:57.739 7f39dd8a4648 [contrail-config-database-nodemgr] [ERROR]: Cannot write http_port 8112 to /tmp/contrail-config-database-nodemgr.0.http_port 02/03/2025 00:44:57.740 7f39dd8a4648 [contrail-config-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8112 02/03/2025 00:44:57.742 7f39dd8a4e48 [contrail-config-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 02/03/2025 00:44:57.744 7f39dd8a4e48 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 02/03/2025 00:44:57.746 7f39dd8a4e48 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/03/2025 00:44:57.746 7f39dd8a4e48 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 02/03/2025 00:44:57.746 7f39dd8a4e48 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:44:57.746 7f39dd8a4e48 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/03/2025 00:44:57.746 7f39dd8a4e48 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:44:57.746 7f39dd8a4e48 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:44:57.746 7f39dd8a4e48 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 02/03/2025 00:44:57.746 7f39dd8a4e48 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:44:57.876 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 38485365180288942233771732495250609784093498227511320804409121808061005781669, 'start': '1738543494000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'zookeeper', 'group': 'zookeeper', 'pid': 54947814440601983878499896483126128320898594542197243424834601089954080386149, 'start': '1738543494000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-config-database-nodemgr', 'group': 'contrail-config-database-nodemgr', 'pid': 99729950964623532725485536865583318687546182852967395215796970884768674824631, 'start': '1738543494000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 02/03/2025 00:44:58.069 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:44:58.070 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. build_info = {"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-01-30 01:14:40.226098", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7573-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 4 num_core_per_socket = 4 num_thread_per_core = 1 >> running_package_version = nightly.r24.1.centos-nightly_r24.1_centos.el7 installed_package_version = nightly.r24.1.centos-nightly_r24.1_centos.el7 >> 02/03/2025 00:44:58.070 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:44:58.071 7f39e3235df0 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2672-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1738543498071278, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2672-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 02/03/2025 00:44:58.071 7f39e3235df0 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2672-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1738543498071803, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2672-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738543494000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738543494000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738543494000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-01-30 01:14:40.226098", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7573-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 02/03/2025 00:44:58.087 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:44:58.087 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/03/2025 00:44:58.087 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:44:58.087 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:44:58.087 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1738543494000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = zookeeper process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1738543494000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1738543494000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-01-30 01:14:40.226098", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7573-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} >> 02/03/2025 00:44:58.087 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:44:58.557 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/03/2025 00:44:59.156 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 02/03/2025 00:44:59.158 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:44:59.158 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. database_usage = [ << disk_space_used_1k = 11652068 disk_space_available_1k = 69438416 config_db_size_1k = 4 >>, ] >> 02/03/2025 00:44:59.158 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:44:59.619 7f39e3235df0 [contrail-config-database-nodemgr] [ERROR]: Failed to connect to database by CQL: Result: 1 02/03/2025 00:44:59.620 7f39e3235df0 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2672-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1738543499619986, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2672-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 02/03/2025 00:44:59.621 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:44:59.621 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/03/2025 00:44:59.621 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:01.745 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:45:01.753 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:45:01.753 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:45:01.753 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:45:01.753 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:45:01.753 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:45:01.754 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:45:01.754 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:45:01.754 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:01.755 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:45:01.755 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:01.755 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:01.755 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:45:01.755 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:02.810 7f39e3235df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/03/2025 00:45:04.932 7f39e3235df0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr config_db_uuid'. Error is: Result: 1 02/03/2025 00:45:05.753 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:45:05.755 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:45:05.755 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:45:05.755 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:45:05.756 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:45:05.756 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:45:05.756 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:45:05.757 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:45:05.757 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:05.757 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:45:05.757 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:05.758 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:05.758 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:45:05.758 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:06.251 7f39e3235df0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr to_bgp_keyspace'. Error is: Result: 1 02/03/2025 00:45:08.678 7f39e3235df0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr svc_monitor_keyspace'. Error is: Result: 1 02/03/2025 00:45:09.756 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:45:09.757 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:45:09.758 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:45:09.758 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:45:09.758 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:45:09.758 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:45:09.758 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:45:09.758 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:45:09.758 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:09.758 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:45:09.759 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:09.759 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:09.759 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:45:09.759 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:10.674 7f39e3235df0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr DISCOVERY_SERVER'. Error is: Result: 1 02/03/2025 00:45:11.950 7f39e3235df0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr useragent'. Error is: Result: 1 02/03/2025 00:45:13.353 7f39e3235df0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr dm_keyspace'. Error is: Result: 1 02/03/2025 00:45:13.758 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:45:13.760 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:45:13.760 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:45:13.760 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:45:13.760 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:45:13.760 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:45:13.760 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:45:13.761 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:45:13.761 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:13.761 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:45:13.761 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:13.761 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:13.761 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:45:13.761 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:17.761 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:45:17.762 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:45:17.763 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:45:17.763 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:45:17.763 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:45:17.763 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:45:17.763 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:45:17.764 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:45:17.764 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:17.764 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:45:17.764 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:17.764 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:17.764 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:45:17.765 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:18.377 7f39e3235df0 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2672-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1738543518377647, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2672-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738543494000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738543494000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738543494000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-01-30 01:14:40.226098", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7573-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 02/03/2025 00:45:18.378 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2672-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12004964, partition_space_available_1k=69085520, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1235872, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58124, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51040, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=4566444, free=235996, buffers=543160, cached=10830036, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.95, five_min_avg=2.17, fifteen_min_avg=1.32, cpu_share=0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/03/2025 00:45:18.378 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:18.378 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1738543494000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = zookeeper process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1738543494000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1738543494000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-01-30 01:14:40.226098", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7573-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} >> 02/03/2025 00:45:18.378 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:18.378 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:18.378 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 12004964 partition_space_available_1k = 69085520 percentage_partition_space_used = 15 >> } all_core_file_list = [ ] process_mem_cpu_usage = { cassandra : << mem_virt = 1235872 cpu_share = 0 mem_res = 0 >>zookeeper : << mem_virt = 58124 cpu_share = 0 mem_res = 0 >>contrail-config-database-nodemgr : << mem_virt = 51040 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175636 used = 4566444 free = 235996 buffers = 543160 cached = 10830036 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 3.95 five_min_avg = 2.17 fifteen_min_avg = 1.32 cpu_share = 0 node_type = config-database-node >> >> 02/03/2025 00:45:18.379 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:21.765 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:45:21.767 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:45:21.767 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:45:21.767 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:45:21.767 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:45:21.767 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:45:21.767 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:45:21.768 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:45:21.768 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:21.768 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:45:21.768 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:21.769 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:21.769 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:45:21.769 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:25.769 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:45:25.770 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:45:25.770 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:45:25.770 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:45:25.771 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:45:25.771 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:45:25.771 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:45:25.771 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:45:25.771 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:25.771 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:45:25.771 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:25.771 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:25.772 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:45:25.772 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:29.771 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:45:29.772 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:45:29.772 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:45:29.772 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:45:29.772 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:45:29.772 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:45:29.772 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:45:29.773 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:45:29.773 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:29.773 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:45:29.773 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:29.773 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:29.773 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:45:29.773 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:33.773 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:45:33.774 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:45:33.775 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:45:33.775 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:45:33.775 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:45:33.775 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:45:33.775 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:45:33.775 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:45:33.775 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:33.775 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:45:33.776 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:33.776 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:33.776 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:45:33.776 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:37.777 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:45:37.778 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:45:37.778 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:45:37.779 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:45:37.779 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:45:37.779 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:45:37.779 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:45:37.779 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:45:37.779 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:37.780 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:45:37.780 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:37.780 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:37.780 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:45:37.781 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:41.781 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:45:41.782 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:45:41.782 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:45:41.782 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:45:41.783 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:45:41.783 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:45:41.783 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:45:41.783 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:45:41.783 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:41.783 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:45:41.783 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:41.783 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:41.784 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:45:41.784 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:45.783 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:45:45.785 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:45:45.785 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:45:45.785 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:45:45.785 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:45:45.785 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:45:45.785 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:45:45.786 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:45:45.786 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:45.786 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:45:45.786 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:45.787 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:45.787 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:45:45.787 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:49.785 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:45:49.786 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:45:49.786 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:45:49.787 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:45:49.787 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:45:49.787 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:45:49.787 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:45:49.787 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:45:49.788 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:49.788 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:45:49.788 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:49.788 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:49.789 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:45:49.789 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:53.789 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:45:53.790 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:45:53.790 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:45:53.790 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:45:53.790 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:45:53.791 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:45:53.791 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:45:53.791 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:45:53.791 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:53.791 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:45:53.791 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:53.791 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:53.792 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:45:53.792 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:57.791 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:45:57.792 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:45:57.792 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:45:57.792 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:45:57.792 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:45:57.793 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:45:57.793 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:45:57.793 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:45:57.793 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:57.793 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:45:57.793 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:57.793 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:57.794 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:45:57.794 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:58.341 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/03/2025 00:45:58.743 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:58.743 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. database_usage = [ << disk_space_used_1k = 12926020 disk_space_available_1k = 68164464 config_db_size_1k = 1576 >>, ] >> 02/03/2025 00:45:58.744 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:59.446 7f39e3235df0 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2672-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1738543559446813, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2672-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.30:8086'], status='Down', description='Connect to Idle on EvTcpConnectFail')]), flag_infos=None, description='Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 02/03/2025 00:45:59.448 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:45:59.449 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:45:59.449 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:00.572 7f39e3235df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/03/2025 00:46:01.793 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:46:01.795 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:46:01.795 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:46:01.795 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:01.795 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:46:01.795 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:01.795 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:46:01.796 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:46:01.796 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:01.796 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:46:01.796 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:01.796 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:01.796 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:46:01.796 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:05.593 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2672-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12926412, partition_space_available_1k=68164072, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1774388, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57948, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51104, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=5099168, free=279228, buffers=603236, cached=10194004, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.6, five_min_avg=2.34, fifteen_min_avg=1.42, cpu_share=17.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/03/2025 00:46:05.594 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:05.594 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 12926412 partition_space_available_1k = 68164072 percentage_partition_space_used = 16 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1774388 cpu_share = 0 mem_res = 0 >>zookeeper : << mem_virt = 57948 cpu_share = 0 mem_res = 0 >>contrail-config-database-nodemgr : << mem_virt = 51104 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175636 used = 5099168 free = 279228 buffers = 603236 cached = 10194004 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 3.6 five_min_avg = 2.34 fifteen_min_avg = 1.42 cpu_share = 17.5 node_type = config-database-node >> >> 02/03/2025 00:46:05.594 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:05.796 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:46:05.797 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:46:05.797 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:46:05.798 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:05.798 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:46:05.798 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:05.798 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:46:05.799 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:46:05.799 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:05.800 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:46:05.800 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:05.800 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:05.801 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:46:05.801 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:09.801 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:46:09.802 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:46:09.802 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:46:09.802 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:09.803 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:46:09.803 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:09.803 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:46:09.803 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:46:09.803 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:09.803 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:46:09.803 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:09.803 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:09.804 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:46:09.804 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:13.805 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:46:13.806 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:46:13.806 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:46:13.806 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:13.806 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:46:13.807 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:13.807 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:46:13.807 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:46:13.807 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:13.807 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:46:13.808 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:13.808 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:13.808 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:46:13.808 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:17.810 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:46:17.811 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:46:17.811 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:46:17.811 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:17.811 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:46:17.811 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:17.811 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:46:17.812 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:46:17.812 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:17.812 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:46:17.812 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:17.812 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:17.812 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:46:17.812 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:21.814 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:46:21.816 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:46:21.816 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:46:21.816 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:21.816 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:46:21.816 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:21.817 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:46:21.817 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:46:21.817 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:21.817 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:46:21.817 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:21.817 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:21.817 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:46:21.818 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:25.818 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:46:25.820 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:46:25.820 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:46:25.820 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:25.820 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:46:25.820 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:25.821 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:46:25.821 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:46:25.821 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:25.821 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:46:25.821 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:25.822 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:25.822 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:46:25.822 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:29.820 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:46:29.821 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:46:29.821 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:46:29.821 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:29.821 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:46:29.822 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:29.822 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:46:29.822 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:46:29.822 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:29.822 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:46:29.822 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:29.823 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:29.823 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:46:29.823 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:33.822 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:46:33.824 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:46:33.824 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:46:33.824 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:33.824 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:46:33.825 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:33.825 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:46:33.825 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:46:33.825 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:33.826 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:46:33.826 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:33.826 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:33.826 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:46:33.826 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:37.826 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:46:37.828 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:46:37.828 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:46:37.828 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:37.828 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:46:37.828 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:37.828 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:46:37.829 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:46:37.829 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:37.829 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:46:37.829 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:37.829 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:37.829 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:46:37.829 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:41.832 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:46:41.833 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:46:41.833 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:46:41.834 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:41.834 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:46:41.834 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:41.834 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:46:41.834 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:46:41.834 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:41.834 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:46:41.835 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:41.835 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:41.835 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:46:41.835 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:45.834 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:46:45.835 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:46:45.836 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:46:45.836 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:45.836 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:46:45.836 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:45.836 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:46:45.836 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:46:45.836 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:45.837 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:46:45.837 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:45.837 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:45.837 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:46:45.837 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:49.837 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:46:49.838 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:46:49.838 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:46:49.838 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:49.838 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:46:49.839 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:49.839 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:46:49.839 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:46:49.839 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:49.839 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:46:49.839 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:49.839 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:49.839 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:46:49.839 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:53.841 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:46:53.842 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:46:53.843 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:46:53.843 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:53.843 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:46:53.843 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:53.843 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:46:53.843 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:46:53.844 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:53.844 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:46:53.844 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:53.844 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:53.844 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:46:53.844 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:57.845 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:46:57.846 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:46:57.846 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:46:57.846 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:57.847 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:46:57.847 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:46:57.847 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:46:57.847 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:46:57.847 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:57.847 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:46:57.848 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:57.848 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:57.848 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:46:57.848 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:58.712 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/03/2025 00:46:59.449 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:46:59.456 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. database_usage = [ << disk_space_used_1k = 13225736 disk_space_available_1k = 67864748 config_db_size_1k = 1832 >>, ] >> 02/03/2025 00:46:59.457 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:01.847 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:47:01.849 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:47:01.849 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:47:01.849 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:01.850 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:47:01.852 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:01.852 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:47:01.853 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:47:01.854 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:01.854 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:47:01.854 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:01.855 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:01.855 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:47:01.855 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:04.158 7f39e3235df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/03/2025 00:47:05.850 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:47:05.853 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:47:05.854 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:47:05.854 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:05.854 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:47:05.854 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:05.854 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:47:05.855 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:47:05.855 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:05.855 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:47:05.855 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:05.855 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:05.855 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:47:05.856 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:09.182 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2672-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13225920, partition_space_available_1k=67864564, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1987460, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=77820, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51092, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=5715608, free=366456, buffers=630160, cached=9463412, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.06, five_min_avg=3.18, fifteen_min_avg=1.77, cpu_share=20.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/03/2025 00:47:09.183 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:09.183 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 13225920 partition_space_available_1k = 67864564 percentage_partition_space_used = 16 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1987460 cpu_share = 0 mem_res = 0 >>zookeeper : << mem_virt = 77820 cpu_share = 0 mem_res = 0 >>contrail-config-database-nodemgr : << mem_virt = 51092 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175636 used = 5715608 free = 366456 buffers = 630160 cached = 9463412 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 6.06 five_min_avg = 3.18 fifteen_min_avg = 1.77 cpu_share = 20.0 node_type = config-database-node >> >> 02/03/2025 00:47:09.183 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:09.854 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:47:09.856 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:47:09.856 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:47:09.856 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:09.856 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:47:09.856 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:09.857 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:47:09.857 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:47:09.857 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:09.857 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:47:09.857 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:09.858 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:09.858 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:47:09.858 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:13.858 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:47:13.860 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:47:13.860 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:47:13.860 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:13.860 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:47:13.860 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:13.860 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:47:13.861 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:47:13.861 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:13.861 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:47:13.861 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:13.861 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:13.861 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:47:13.861 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:17.861 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:47:17.862 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:47:17.862 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:47:17.862 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:17.862 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:47:17.863 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:17.863 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:47:17.863 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:47:17.863 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:17.863 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:47:17.863 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:17.863 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:17.863 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:47:17.863 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:21.865 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:47:21.868 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:47:21.868 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:47:21.869 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:21.869 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:47:21.869 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:21.869 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:47:21.869 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:47:21.869 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:21.869 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:47:21.869 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:21.869 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:21.870 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:47:21.870 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:25.869 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:47:25.870 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:47:25.870 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:47:25.871 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:25.871 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:47:25.871 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:25.871 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:47:25.871 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:47:25.871 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:25.871 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:47:25.872 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:25.872 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:25.872 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:47:25.872 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:29.873 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:47:29.874 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:47:29.874 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:47:29.874 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:29.874 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:47:29.875 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:29.875 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:47:29.875 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:47:29.875 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:29.876 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:47:29.876 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:29.876 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:29.876 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:47:29.876 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:33.877 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:47:33.878 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:47:33.878 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:47:33.878 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:33.879 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:47:33.879 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:33.880 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:47:33.880 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:47:33.880 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:33.881 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:47:33.881 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:33.884 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:33.884 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:47:33.885 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:37.881 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:47:37.882 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:47:37.882 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:47:37.882 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:37.883 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:47:37.883 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:37.883 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:47:37.883 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:47:37.884 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:37.884 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:47:37.884 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:37.884 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:37.885 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:47:37.885 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:41.885 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:47:41.886 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:47:41.886 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:47:41.886 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:41.887 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:47:41.887 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:41.887 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:47:41.888 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:47:41.888 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:41.888 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:47:41.889 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:41.889 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:41.889 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:47:41.889 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:45.886 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:47:45.887 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:47:45.888 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:47:45.888 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:45.888 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:47:45.888 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:45.888 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:47:45.888 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:47:45.889 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:45.889 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:47:45.889 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:45.889 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:45.889 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:47:45.889 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:49.889 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:47:49.890 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:47:49.890 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:47:49.890 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:49.890 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:47:49.890 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:49.890 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:47:49.891 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:47:49.891 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:49.891 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:47:49.891 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:49.891 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:49.891 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:47:49.891 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:53.893 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:47:53.894 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:47:53.894 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:47:53.894 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:53.894 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:47:53.894 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:53.894 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:47:53.894 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:47:53.894 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:53.895 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:47:53.895 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:53.895 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:53.895 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:47:53.895 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:57.897 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:47:57.898 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:47:57.898 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:47:57.898 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:57.899 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:47:57.899 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:47:57.899 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:47:57.899 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:47:57.899 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:57.899 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:47:57.899 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:57.899 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:57.900 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:47:57.900 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:58.341 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/03/2025 00:47:58.682 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:47:58.682 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. database_usage = [ << disk_space_used_1k = 13344548 disk_space_available_1k = 67745936 config_db_size_1k = 1764 >>, ] >> 02/03/2025 00:47:58.683 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:00.388 7f39e3235df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/03/2025 00:48:01.899 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:48:02.132 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:48:02.132 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:48:02.132 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:02.132 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:48:02.132 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:02.132 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:48:02.133 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:48:02.133 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:02.133 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:48:02.133 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:02.133 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:02.133 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:48:02.133 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:05.428 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2672-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13344560, partition_space_available_1k=67745924, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2073100, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=80088, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51076, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6046516, free=310472, buffers=631164, cached=9187484, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.06, five_min_avg=3.75, fifteen_min_avg=2.05, cpu_share=20.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/03/2025 00:48:05.429 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:05.429 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 13344560 partition_space_available_1k = 67745924 percentage_partition_space_used = 16 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 2073100 cpu_share = 0 mem_res = 0 >>zookeeper : << mem_virt = 80088 cpu_share = 0 mem_res = 0 >>contrail-config-database-nodemgr : << mem_virt = 51076 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175636 used = 6046516 free = 310472 buffers = 631164 cached = 9187484 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 6.06 five_min_avg = 3.75 fifteen_min_avg = 2.05 cpu_share = 20.75 node_type = config-database-node >> >> 02/03/2025 00:48:05.429 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:06.132 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:48:06.134 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:48:06.134 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:48:06.134 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:06.134 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:48:06.134 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:06.134 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:48:06.135 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:48:06.135 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:06.135 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:48:06.135 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:06.135 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:06.135 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:48:06.135 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:10.137 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:48:10.138 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:48:10.138 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:48:10.138 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:10.138 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:48:10.139 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:10.139 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:48:10.139 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:48:10.139 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:10.139 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:48:10.139 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:10.139 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:10.139 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:48:10.139 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:14.141 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:48:14.144 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:48:14.147 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:48:14.147 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:14.148 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:48:14.148 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:14.149 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:48:14.151 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:48:14.151 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:14.152 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:48:14.152 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:14.152 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:14.153 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:48:14.153 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:18.145 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:48:18.146 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:48:18.146 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:48:18.146 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:18.146 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:48:18.146 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:18.147 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:48:18.147 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:48:18.147 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:18.147 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:48:18.148 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:18.148 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:18.148 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:48:18.148 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:22.149 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:48:22.151 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:48:22.151 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:48:22.151 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:22.151 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:48:22.151 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:22.151 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:48:22.152 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:48:22.152 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:22.152 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:48:22.152 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:22.152 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:22.152 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:48:22.152 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:26.154 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:48:26.156 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:48:26.156 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:48:26.156 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:26.156 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:48:26.158 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:26.158 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:48:26.158 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:48:26.158 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:26.158 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:48:26.158 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:26.158 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:26.159 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:48:26.159 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:30.158 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:48:30.160 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:48:30.160 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:48:30.160 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:30.160 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:48:30.160 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:30.160 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:48:30.161 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:48:30.161 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:30.161 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:48:30.161 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:30.161 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:30.162 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:48:30.162 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:34.161 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:48:34.162 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:48:34.163 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:48:34.163 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:34.163 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:48:34.163 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:34.164 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:48:34.164 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:48:34.164 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:34.164 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:48:34.164 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:34.165 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:34.165 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:48:34.165 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:38.166 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:48:38.168 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:48:38.168 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:48:38.168 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:38.168 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:48:38.168 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:38.169 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:48:38.169 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:48:38.169 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:38.170 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:48:38.170 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:38.170 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:38.171 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:48:38.171 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:42.169 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:48:42.170 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:48:42.170 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:48:42.170 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:42.170 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:48:42.170 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:42.170 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:48:42.171 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:48:42.171 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:42.171 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:48:42.171 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:42.171 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:42.171 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:48:42.171 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:46.170 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:48:46.171 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:48:46.171 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:48:46.172 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:46.172 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:48:46.172 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:46.172 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:48:46.172 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:48:46.172 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:46.172 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:48:46.173 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:46.173 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:46.173 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:48:46.173 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:50.174 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:48:50.175 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:48:50.175 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:48:50.175 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:50.175 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:48:50.176 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:50.176 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:48:50.176 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:48:50.176 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:50.176 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:48:50.176 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:50.176 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:50.176 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:48:50.176 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:54.177 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:48:54.178 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:48:54.178 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:48:54.178 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:54.178 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:48:54.178 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:54.178 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:48:54.178 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:48:54.178 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:54.179 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:48:54.179 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:54.179 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:54.179 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:48:54.179 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:58.210 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:48:58.218 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:48:58.219 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:48:58.219 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:58.219 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:48:58.219 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:48:58.219 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:48:58.219 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:48:58.219 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:58.220 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:48:58.220 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:58.220 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:58.220 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:48:58.220 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:58.401 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/03/2025 00:48:58.810 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:48:58.810 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. database_usage = [ << disk_space_used_1k = 13734384 disk_space_available_1k = 67356100 config_db_size_1k = 1764 >>, ] >> 02/03/2025 00:48:58.810 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:00.583 7f39e3235df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/03/2025 00:49:02.219 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:49:02.221 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:49:02.221 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:49:02.221 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:02.221 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:49:02.221 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:02.222 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:49:02.222 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:49:02.222 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:02.222 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:49:02.222 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:02.222 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:02.222 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:49:02.223 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:05.606 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2672-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13734288, partition_space_available_1k=67356196, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2069036, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=80272, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51120, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6171588, free=413924, buffers=656508, cached=8933616, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.13, five_min_avg=3.62, fifteen_min_avg=2.12, cpu_share=16.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/03/2025 00:49:05.606 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:05.607 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 13734288 partition_space_available_1k = 67356196 percentage_partition_space_used = 17 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 2069036 cpu_share = 0 mem_res = 0 >>zookeeper : << mem_virt = 80272 cpu_share = 0 mem_res = 0 >>contrail-config-database-nodemgr : << mem_virt = 51120 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175636 used = 6171588 free = 413924 buffers = 656508 cached = 8933616 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 4.13 five_min_avg = 3.62 fifteen_min_avg = 2.12 cpu_share = 16.5 node_type = config-database-node >> >> 02/03/2025 00:49:05.607 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:06.222 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:49:06.223 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:49:06.223 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:49:06.223 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:06.223 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:49:06.223 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:06.223 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:49:06.224 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:49:06.224 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:06.224 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:49:06.224 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:06.224 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:06.224 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:49:06.224 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:10.225 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:49:10.226 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:49:10.226 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:49:10.226 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:10.226 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:49:10.226 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:10.227 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:49:10.227 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:49:10.227 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:10.227 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:49:10.227 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:10.227 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:10.227 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:49:10.227 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:14.230 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:49:14.231 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:49:14.231 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:49:14.232 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:14.232 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:49:14.232 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:14.232 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:49:14.232 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:49:14.232 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:14.232 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:49:14.233 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:14.233 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:14.233 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:49:14.233 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:18.232 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:49:18.233 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:49:18.233 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:49:18.233 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:18.233 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:49:18.233 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:18.233 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:49:18.234 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:49:18.234 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:18.234 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:49:18.234 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:18.234 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:18.234 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:49:18.234 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:22.234 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:49:22.236 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:49:22.236 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:49:22.236 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:22.237 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:49:22.237 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:22.237 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:49:22.238 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:49:22.238 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:22.238 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:49:22.238 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:22.238 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:22.239 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:49:22.239 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:26.238 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:49:26.486 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:49:26.487 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:49:26.487 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:26.487 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:49:26.487 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:26.488 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:49:26.488 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:49:26.488 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:26.488 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:49:26.489 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:26.489 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:26.489 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:49:26.489 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:30.489 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:49:30.490 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:49:30.490 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:49:30.490 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:30.490 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:49:30.490 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:30.490 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:49:30.491 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:49:30.491 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:30.491 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:49:30.491 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:30.491 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:30.491 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:49:30.491 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:34.493 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:49:34.494 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:49:34.494 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:49:34.494 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:34.495 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:49:34.495 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:34.495 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:49:34.495 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:49:34.495 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:34.495 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:49:34.495 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:34.495 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:34.495 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:49:34.495 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:38.497 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:49:38.498 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:49:38.498 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:49:38.498 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:38.499 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:49:38.499 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:38.499 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:49:38.499 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:49:38.499 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:38.499 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:49:38.499 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:38.499 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:38.500 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:49:38.500 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:42.501 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:49:42.502 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:49:42.503 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:49:42.503 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:42.503 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:49:42.503 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:42.503 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:49:42.503 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:49:42.503 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:42.503 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:49:42.504 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:42.504 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:42.504 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:49:42.504 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:46.505 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:49:46.506 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:49:46.506 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:49:46.506 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:46.506 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:49:46.506 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:46.507 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:49:46.507 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:49:46.507 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:46.507 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:49:46.507 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:46.507 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:46.507 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:49:46.507 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:50.509 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:49:50.510 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:49:50.510 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:49:50.510 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:50.511 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:49:50.511 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:50.511 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:49:50.511 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:49:50.511 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:50.511 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:49:50.512 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:50.512 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:50.512 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:49:50.512 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:54.513 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:49:54.514 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:49:54.515 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:49:54.515 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:54.515 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:49:54.515 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:54.515 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:49:54.515 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:49:54.516 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:54.516 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:49:54.516 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:54.516 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:54.516 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:49:54.516 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:58.370 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/03/2025 00:49:58.518 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:49:58.529 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:49:58.529 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:49:58.529 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:58.530 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:49:58.530 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:49:58.530 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:49:58.530 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:49:58.530 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:58.530 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:49:58.530 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:58.531 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:58.531 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:49:58.531 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:58.770 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:49:58.770 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. database_usage = [ << disk_space_used_1k = 13747552 disk_space_available_1k = 67342932 config_db_size_1k = 1764 >>, ] >> 02/03/2025 00:49:58.770 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:50:01.189 7f39e3235df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/03/2025 00:50:02.529 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:50:02.531 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:50:02.531 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:50:02.531 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:50:02.532 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:50:02.532 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:50:02.532 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:50:02.532 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:50:02.532 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:50:02.532 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:50:02.532 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:50:02.532 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:50:02.533 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.32:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:50:02.533 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:50:06.237 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2672-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13747688, partition_space_available_1k=67342796, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2066636, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=80428, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50984, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7554980, free=368444, buffers=658816, cached=7593396, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.73, five_min_avg=3.9, fifteen_min_avg=2.31, cpu_share=18.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/03/2025 00:50:06.238 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:50:06.238 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 13747688 partition_space_available_1k = 67342796 percentage_partition_space_used = 17 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 2066636 cpu_share = 0 mem_res = 0 >>zookeeper : << mem_virt = 80428 cpu_share = 0 mem_res = 0 >>contrail-config-database-nodemgr : << mem_virt = 50984 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175636 used = 7554980 free = 368444 buffers = 658816 cached = 7593396 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 4.73 five_min_avg = 3.9 fifteen_min_avg = 2.31 cpu_share = 18.75 node_type = config-database-node >> >> 02/03/2025 00:50:06.238 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:50:06.532 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:50:06.534 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:50:06.534 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:50:06.534 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:50:06.534 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:50:06.534 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:50:06.534 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:50:06.534 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:50:06.534 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:50:06.535 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.36:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:50:06.535 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:50:06.535 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:50:06.535 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.36:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:50:06.535 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:50:10.537 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:50:10.538 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/03/2025 00:50:10.538 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:50:10.538 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:50:10.538 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:50:10.539 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:50:10.539 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/03/2025 00:50:10.539 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/03/2025 00:50:10.539 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:50:10.539 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.30:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/03/2025 00:50:10.539 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:50:10.539 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:50:10.540 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1738543497745998 collector_name = collector_ip = 10.0.0.30:8086 collector_list = [ 10.0.0.32:8086, 10.0.0.36:8086, 10.0.0.30:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/03/2025 00:50:10.540 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/03/2025 00:50:14.541 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/03/2025 00:50:14.543 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected 02/03/2025 00:50:14.543 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/03/2025 00:50:14.543 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:50:14.544 7f39dda27348 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2672-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.32:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 00:50:14.544 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/03/2025 00:50:14.544 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 02/03/2025 00:50:14.544 7f39dda27348 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 02/03/2025 00:50:14.545 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 02/03/2025 00:50:14.545 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/03/2025 00:50:14.545 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/03/2025 00:50:26.907 7f39dd9f1348 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001016 02/03/2025 00:50:26.913 7f39dd9f1348 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 02/03/2025 00:50:26.917 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 02/03/2025 00:50:26.918 7f39dda27348 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 02/03/2025 00:50:26.918 7f39dda27348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 02/03/2025 00:50:26.918 7f39dda27348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 02/03/2025 00:50:26.918 7f39dda27348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/03/2025 00:50:26.918 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 02/03/2025 00:50:26.918 7f39dda27048 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/03/2025 00:50:26.920 7f39dda27048 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/03/2025 00:50:37.436 7f39dd9f1348 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001439 02/03/2025 00:50:37.437 7f39dd9f1348 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 02/03/2025 00:50:37.437 7f39dda27048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 02/03/2025 00:50:37.437 7f39dda27048 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 3 02/03/2025 00:50:37.437 7f39dda27048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 02/03/2025 00:50:37.438 7f39dda27048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 02/03/2025 00:50:37.438 7f39dda27048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/03/2025 00:50:37.438 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/03/2025 00:50:37.438 7f39dda27348 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/03/2025 00:50:58.487 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/03/2025 00:51:01.697 7f39e3235df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/03/2025 00:51:06.737 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2672-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14171252, partition_space_available_1k=66919232, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2077084, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=156320, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51152, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8630628, free=327036, buffers=671736, cached=6546236, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.25, five_min_avg=4.47, fifteen_min_avg=2.6, cpu_share=19.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/03/2025 00:51:58.436 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/03/2025 00:52:00.761 7f39e3235df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/03/2025 00:52:05.788 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2672-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14235484, partition_space_available_1k=66855000, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2072244, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=162876, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51160, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8546448, free=433896, buffers=672060, cached=6523232, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.07, five_min_avg=4.5, fifteen_min_avg=2.73, cpu_share=14.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/03/2025 00:52:58.414 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/03/2025 00:53:01.143 7f39e3235df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/03/2025 00:53:06.167 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2672-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14271508, partition_space_available_1k=66818976, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2071312, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=144304, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51176, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8531952, free=421896, buffers=672300, cached=6549488, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.5, five_min_avg=4.39, fifteen_min_avg=2.8, cpu_share=11.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/03/2025 00:53:58.449 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/03/2025 00:54:00.617 7f39e3235df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/03/2025 00:54:05.641 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2672-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14288404, partition_space_available_1k=66802080, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2105100, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=135836, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51208, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8591240, free=362044, buffers=672832, cached=6549520, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.12, five_min_avg=4.32, fifteen_min_avg=2.88, cpu_share=12.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/03/2025 00:54:58.431 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/03/2025 00:55:00.902 7f39e3235df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/03/2025 00:55:05.940 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2672-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14295196, partition_space_available_1k=66795288, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2116060, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=130956, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51220, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8619552, free=342116, buffers=672788, cached=6541180, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.53, five_min_avg=4.11, fifteen_min_avg=2.9, cpu_share=11.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/03/2025 00:55:58.474 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/03/2025 00:56:00.789 7f39e3235df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/03/2025 00:56:05.815 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2672-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14291712, partition_space_available_1k=66798772, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2132724, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=135396, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51224, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8612752, free=358476, buffers=672980, cached=6531428, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.77, five_min_avg=3.81, fifteen_min_avg=2.88, cpu_share=11.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/03/2025 00:56:58.445 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/03/2025 00:57:00.943 7f39e3235df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/03/2025 00:57:05.971 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2672-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14301000, partition_space_available_1k=66789484, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2127656, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=138876, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51228, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8617212, free=416508, buffers=673012, cached=6468904, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.78, five_min_avg=3.62, fifteen_min_avg=2.87, cpu_share=11.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/03/2025 00:57:58.430 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/03/2025 00:58:00.799 7f39e3235df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/03/2025 00:58:05.823 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2672-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14308232, partition_space_available_1k=66782252, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2138576, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=142820, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51236, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8653900, free=371496, buffers=673240, cached=6477000, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.97, five_min_avg=3.83, fifteen_min_avg=2.99, cpu_share=11.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/03/2025 00:58:58.428 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/03/2025 00:59:00.820 7f39e3235df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/03/2025 00:59:05.853 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2672-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14320548, partition_space_available_1k=66769936, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2159196, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=146644, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51240, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8689132, free=350960, buffers=673304, cached=6462240, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.04, five_min_avg=3.88, fifteen_min_avg=3.06, cpu_share=11.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/03/2025 00:59:58.453 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/03/2025 01:00:00.675 7f39e3235df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/03/2025 01:00:05.705 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2672-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14327256, partition_space_available_1k=66763228, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2177696, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=150416, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51244, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8713864, free=344500, buffers=673300, cached=6443972, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.57, five_min_avg=3.74, fifteen_min_avg=3.06, cpu_share=11.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/03/2025 01:00:58.446 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/03/2025 01:01:01.002 7f39e3235df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/03/2025 01:01:06.029 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2672-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14335528, partition_space_available_1k=66754956, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2195652, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=154084, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51244, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8742756, free=386832, buffers=673388, cached=6372660, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.84, five_min_avg=3.8, fifteen_min_avg=3.12, cpu_share=11.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/03/2025 01:01:58.443 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/03/2025 01:02:00.936 7f39e3235df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/03/2025 01:02:05.960 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2672-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14343304, partition_space_available_1k=66747180, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2216356, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=158020, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51248, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8776656, free=342660, buffers=673420, cached=6382900, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.79, five_min_avg=3.52, fifteen_min_avg=3.07, cpu_share=10.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/03/2025 01:02:58.462 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/03/2025 01:03:00.723 7f39e3235df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/03/2025 01:03:05.747 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2672-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14353912, partition_space_available_1k=66736572, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2231128, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=161572, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51276, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8792312, free=346420, buffers=673468, cached=6363436, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.11, five_min_avg=3.47, fifteen_min_avg=3.08, cpu_share=11.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/03/2025 01:03:58.482 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/03/2025 01:04:00.870 7f39e3235df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/03/2025 01:04:05.896 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2672-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14366688, partition_space_available_1k=66723796, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2236868, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=165328, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51732, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8804756, free=340888, buffers=673484, cached=6356508, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.57, five_min_avg=3.53, fifteen_min_avg=3.13, cpu_share=11.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/03/2025 01:04:58.478 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/03/2025 01:05:01.254 7f39e3235df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/03/2025 01:05:06.292 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2672-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14375876, partition_space_available_1k=66714608, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2242688, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=168388, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51740, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8799648, free=405636, buffers=673424, cached=6296928, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.3, five_min_avg=3.47, fifteen_min_avg=3.13, cpu_share=11.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/03/2025 01:05:58.483 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/03/2025 01:06:00.791 7f39e3235df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/03/2025 01:06:05.815 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2672-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14384832, partition_space_available_1k=66705652, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2250904, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=172296, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51736, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8836592, free=354220, buffers=673444, cached=6311380, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.5, five_min_avg=3.51, fifteen_min_avg=3.17, cpu_share=11.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/03/2025 01:06:58.488 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/03/2025 01:07:00.735 7f39e3235df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/03/2025 01:07:05.757 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2672-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14391548, partition_space_available_1k=66698936, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2262840, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=184416, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51744, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8845128, free=342044, buffers=673432, cached=6315032, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.6, five_min_avg=3.52, fifteen_min_avg=3.2, cpu_share=11.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/03/2025 01:07:58.490 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/03/2025 01:08:00.923 7f39e3235df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/03/2025 01:08:05.947 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2672-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14398612, partition_space_available_1k=66691872, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2271388, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=187928, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51748, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8864300, free=341784, buffers=673404, cached=6296148, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.81, five_min_avg=3.6, fifteen_min_avg=3.24, cpu_share=11.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/03/2025 01:08:58.527 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/03/2025 01:09:01.260 7f39e3235df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/03/2025 01:09:06.287 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2672-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14406344, partition_space_available_1k=66684140, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2274784, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=185516, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51748, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8879832, free=387700, buffers=673312, cached=6234792, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.54, five_min_avg=3.55, fifteen_min_avg=3.25, cpu_share=11.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/03/2025 01:09:58.525 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/03/2025 01:10:01.015 7f39e3235df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/03/2025 01:10:06.051 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2672-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14412756, partition_space_available_1k=66677728, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2278448, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=189368, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51756, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8855496, free=399840, buffers=673316, cached=6246984, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.09, five_min_avg=3.71, fifteen_min_avg=3.32, cpu_share=10.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/03/2025 01:10:58.626 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 02/03/2025 01:11:01.063 7f39e3235df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 02/03/2025 01:11:06.089 7f39e3235df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2672-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14743356, partition_space_available_1k=66347128, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2284288, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=185436, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51756, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=8859744, free=286100, buffers=676464, cached=6353328, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.95, five_min_avg=4.07, fifteen_min_avg=3.47, cpu_share=12.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 02/03/2025 01:11:11.126 7f39dda27348 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map