09/22/2025 02:02:34.679 7f03bd44c3c0 [contrail-config-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0]
09/22/2025 02:02:34.679 7f03bd44c3c0 [contrail-config-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False]
09/22/2025 02:02:34.679 7f03bd44c3c0 [contrail-config-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False]
09/22/2025 02:02:34.687 7f03ba60e4a0 [contrail-config-database-nodemgr] [ERROR]: Cannot write http_port 8112 to /tmp/contrail-config-database-nodemgr.0.http_port
09/22/2025 02:02:34.687 7f03ba60e4a0 [contrail-config-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8112
09/22/2025 02:02:34.688 7f03ba60e860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvStart] in state[none]
09/22/2025 02:02:34.688 7f03ba60e860 [contrail-config-database-nodemgr] [DEBUG]: Add uve <cn-jenkins-deploy-platform-ansible-os-4162-1., 1> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
09/22/2025 02:02:34.689 7f03ba60e860 [contrail-config-database-nodemgr] [DEBUG]: Add uve <cn-jenkins-deploy-platform-ansible-os-4162-1.:ConfigDatabase:contrail-config-database-nodemgr:0, 1> in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map
09/22/2025 02:02:34.689 7f03ba60e860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle]
09/22/2025 02:02:34.689 7f03ba60e860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:34.689 7f03ba60e860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>,  ]  description = Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
09/22/2025 02:02:34.689 7f03ba60e860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:34.689 7f03ba60e860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:34.689 7f03ba60e860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758506554689242  collector_name =   collector_ip =   collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 1  >>  
09/22/2025 02:02:34.689 7f03ba60e860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:34.816 7f03bd44c3c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 102302485377032080796243889548947077122647512574363160483305850750577158999781, 'start': '1758506552000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'zookeeper', 'group': 'zookeeper', 'pid': 61481572713899344851973454847735309341647972357698541161089820918079667761920, 'start': '1758506552000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-config-database-nodemgr', 'group': 'contrail-config-database-nodemgr', 'pid': 58435539265923028660544156504388396914202213655193017298535739891707561084004, 'start': '1758506553000000', 'statename': 'PROCESS_STATE_RUNNING'}]'
09/22/2025 02:02:34.886 7f03ba4bb180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:34.886 7f03ba4bb180 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  build_info = {"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-22 00:12:25.178504", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11201-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed
"}]}  system_cpu_info = <<  num_socket = 1  num_cpu = 8  num_core_per_socket = 8  num_thread_per_core = 1  >>  running_package_version = package contrail-lib is not installed
  installed_package_version = package contrail-lib is not installed
  >>  
09/22/2025 02:02:34.886 7f03ba4bb180 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:34.887 7f03bd44c3c0 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-4162-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1758506554887145, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4162-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
09/22/2025 02:02:34.887 7f03bd44c3c0 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-4162-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1758506554887424, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4162-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1758506552000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1758506552000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1758506553000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-22 00:12:25.178504", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11201-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed\n"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
09/22/2025 02:02:34.889 7f03ba4bb180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:34.889 7f03ba4bb180 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>,  ]  description = Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
09/22/2025 02:02:34.889 7f03ba4bb180 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:34.889 7f03ba4bb180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:34.889 7f03ba4bb180 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  deleted = False  process_info = [ <<  process_name = cassandra  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1758506552000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>, <<  process_name = zookeeper  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1758506552000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>, <<  process_name = contrail-config-database-nodemgr  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1758506553000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>,  ]  build_info = {"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-22 00:12:25.178504", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11201-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed
"}]}  >>  
09/22/2025 02:02:34.889 7f03ba4bb180 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:35.120 7f03bd44c3c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/22/2025 02:02:35.371 7f03bd44c3c0 [contrail-config-database-nodemgr] [DEBUG]: Add uve <cn-jenkins-deploy-platform-ansible-os-4162-1., 1> in the [ObjectConfigDatabaseInfo:DatabaseUsage] map
09/22/2025 02:02:35.372 7f03ba4a3d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:35.372 7f03ba4a3d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  database_usage = [ <<  disk_space_used_1k = 13684744  disk_space_available_1k = 67405740  config_db_size_1k = 4  >>,  ]  >>  
09/22/2025 02:02:35.372 7f03ba4a3d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:35.491 7f03bd44c3c0 [contrail-config-database-nodemgr] [ERROR]: Failed to connect to database by CQL and socket: Result: 1
09/22/2025 02:02:35.491 7f03bd44c3c0 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-4162-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1758506555491419, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4162-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
09/22/2025 02:02:35.492 7f03ba4a3f40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:35.492 7f03ba4a3f40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
09/22/2025 02:02:35.493 7f03ba4a3f40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:36.331 7f03bd44c3c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/22/2025 02:02:37.369 7f03bd44c3c0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr config_db_uuid'. Error is: Result: 1
09/22/2025 02:02:38.227 7f03bd44c3c0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr to_bgp_keyspace'. Error is: Result: 1
09/22/2025 02:02:38.689 7f03ba4a3ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 02:02:38.690 7f03ba4a3ae0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 02:02:38.691 7f03ba4a3ae0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 02:02:38.691 7f03ba4a3ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:02:38.692 7f03ba4a3ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 02:02:38.692 7f03ba4a3ae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:02:38.692 7f03ba4a3ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 02:02:38.692 7f03ba4a3ae0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 02:02:38.692 7f03ba4a3ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:38.692 7f03ba4a3ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 02:02:38.692 7f03ba4a3ae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:38.692 7f03ba4a3ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:38.693 7f03ba4a3ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758506554689242  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 02:02:38.693 7f03ba4a3ae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:38.857 7f03bd44c3c0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr svc_monitor_keyspace'. Error is: Result: 1
09/22/2025 02:02:39.527 7f03bd44c3c0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr DISCOVERY_SERVER'. Error is: Result: 1
09/22/2025 02:02:40.178 7f03bd44c3c0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr useragent'. Error is: Result: 1
09/22/2025 02:02:40.851 7f03bd44c3c0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr dm_keyspace'. Error is: Result: 1
09/22/2025 02:02:42.691 7f03ba4bbd60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 02:02:42.692 7f03ba4bbd60 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 02:02:42.692 7f03ba4bbd60 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 02:02:42.692 7f03ba4bbd60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:02:42.693 7f03ba4bbd60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 02:02:42.693 7f03ba4bbd60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:02:42.693 7f03ba4bbd60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 02:02:42.693 7f03ba4bbd60 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 02:02:42.693 7f03ba4bbd60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:42.693 7f03ba4bbd60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 02:02:42.693 7f03ba4bbd60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:42.693 7f03ba4bbd60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:42.693 7f03ba4bbd60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758506554689242  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 02:02:42.694 7f03ba4bbd60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:45.880 7f03bd44c3c0 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-4162-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1758506565880802, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4162-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1758506552000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1758506552000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1758506553000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-22 00:12:25.178504", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11201-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed\n"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
09/22/2025 02:02:45.881 7f03bd44c3c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4162-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13876412, partition_space_available_1k=67214072, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1267100, cpu_share=0.0, mem_res=1256188, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54340, cpu_share=0.0, mem_res=52056, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52612, cpu_share=0.0, mem_res=42632, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=5155744, free=12909088, buffers=611716, cached=13988564, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.76, five_min_avg=1.48, fifteen_min_avg=0.99, cpu_share=0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/22/2025 02:02:45.881 7f03ba60ec20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:45.881 7f03ba60ec20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  deleted = False  process_info = [ <<  process_name = cassandra  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1758506552000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>, <<  process_name = zookeeper  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1758506552000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>, <<  process_name = contrail-config-database-nodemgr  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1758506553000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>,  ]  build_info = {"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-22 00:12:25.178504", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11201-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed
"}]}  >>  
09/22/2025 02:02:45.881 7f03ba60ec20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:45.881 7f03ba60ec20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:45.881 7f03ba60ec20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  disk_usage_info = { /dev/vda1 : <<  partition_type = ext4  partition_space_used_1k = 13876412  partition_space_available_1k = 67214072  percentage_partition_space_used = 17  >> }  all_core_file_list = [  ]  process_mem_cpu_usage = { cassandra : <<  mem_virt = 1267100  cpu_share = 0.0  mem_res = 1256188  >>zookeeper : <<  mem_virt = 54340  cpu_share = 0.0  mem_res = 52056  >>contrail-config-database-nodemgr : <<  mem_virt = 52612  cpu_share = 0.0  mem_res = 42632  >> }  system_mem_usage = <<  total = 32665112  used = 5155744  free = 12909088  buffers = 611716  cached = 13988564  node_type = config-database-node  >>  system_cpu_usage = <<  one_min_avg = 1.76  five_min_avg = 1.48  fifteen_min_avg = 0.99  cpu_share = 0  node_type = config-database-node  >>  >>  
09/22/2025 02:02:45.881 7f03ba60ec20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:46.694 7f03ba60ec20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 02:02:46.695 7f03ba60ec20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 02:02:46.695 7f03ba60ec20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 02:02:46.695 7f03ba60ec20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:02:46.695 7f03ba60ec20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 02:02:46.695 7f03ba60ec20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:02:46.695 7f03ba60ec20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 02:02:46.696 7f03ba60ec20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 02:02:46.696 7f03ba60ec20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:46.696 7f03ba60ec20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 02:02:46.696 7f03ba60ec20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:46.696 7f03ba60ec20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:46.696 7f03ba60ec20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758506554689242  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 02:02:46.696 7f03ba60ec20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:50.695 7f03ba60e900 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 02:02:50.696 7f03ba60e900 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 02:02:50.697 7f03ba60e900 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 02:02:50.697 7f03ba60e900 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:02:50.697 7f03ba60e900 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 02:02:50.697 7f03ba60e900 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:02:50.697 7f03ba60e900 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 02:02:50.697 7f03ba60e900 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 02:02:50.697 7f03ba60e900 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:50.697 7f03ba60e900 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 02:02:50.697 7f03ba60e900 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:50.697 7f03ba60e900 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:50.698 7f03ba60e900 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758506554689242  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 02:02:50.698 7f03ba60e900 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:54.698 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 02:02:54.699 7f03ba60eea0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 02:02:54.699 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 02:02:54.699 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:02:54.700 7f03ba60eea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 02:02:54.700 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:02:54.700 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 02:02:54.700 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 02:02:54.700 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:54.700 7f03ba60eea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 02:02:54.700 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:54.700 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:54.700 7f03ba60eea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758506554689242  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 02:02:54.701 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:58.702 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 02:02:58.703 7f03ba60eea0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 02:02:58.704 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 02:02:58.704 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:02:58.704 7f03ba60eea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 02:02:58.704 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:02:58.704 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 02:02:58.704 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 02:02:58.704 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:58.704 7f03ba60eea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 02:02:58.704 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:58.705 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:02:58.705 7f03ba60eea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758506554689242  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 02:02:58.705 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:02.707 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 02:03:02.708 7f03ba60eea0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 02:03:02.708 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 02:03:02.709 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:03:02.709 7f03ba60eea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 02:03:02.709 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:03:02.709 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 02:03:02.709 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 02:03:02.709 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:02.710 7f03ba60eea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 02:03:02.710 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:02.710 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:02.710 7f03ba60eea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758506554689242  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 02:03:02.710 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:06.709 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 02:03:06.711 7f03ba60eea0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 02:03:06.711 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 02:03:06.711 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:03:06.711 7f03ba60eea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 02:03:06.711 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:03:06.711 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 02:03:06.711 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 02:03:06.712 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:06.712 7f03ba60eea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 02:03:06.712 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:06.712 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:06.712 7f03ba60eea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758506554689242  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 02:03:06.712 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:10.714 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 02:03:10.716 7f03ba60eea0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 02:03:10.716 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 02:03:10.716 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:03:10.716 7f03ba60eea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 02:03:10.716 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:03:10.716 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 02:03:10.717 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 02:03:10.717 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:10.717 7f03ba60eea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 02:03:10.717 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:10.717 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:10.717 7f03ba60eea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758506554689242  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 02:03:10.717 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:14.718 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 02:03:14.720 7f03ba60eea0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 02:03:14.720 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 02:03:14.720 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:03:14.720 7f03ba60eea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 02:03:14.720 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:03:14.720 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 02:03:14.720 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 02:03:14.721 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:14.721 7f03ba60eea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 02:03:14.721 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:14.721 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:14.721 7f03ba60eea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758506554689242  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 02:03:14.721 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:18.721 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 02:03:18.722 7f03ba60eea0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 02:03:18.722 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 02:03:18.722 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:03:18.722 7f03ba60eea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 02:03:18.722 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:03:18.722 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 02:03:18.722 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 02:03:18.723 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:18.723 7f03ba60eea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 02:03:18.723 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:18.723 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:18.723 7f03ba60eea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758506554689242  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 02:03:18.723 7f03ba60eea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:22.725 7f03ba4a39a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 02:03:22.732 7f03ba4a39a0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 02:03:22.732 7f03ba4a39a0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 02:03:22.733 7f03ba4a39a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:03:22.733 7f03ba4a39a0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 02:03:22.734 7f03ba4a39a0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:03:22.734 7f03ba4a39a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 02:03:22.736 7f03ba4a39a0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 02:03:22.747 7f03ba4a39a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:22.747 7f03ba4a39a0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 02:03:22.747 7f03ba4a39a0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:22.747 7f03ba4a39a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:22.748 7f03ba4a39a0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758506554689242  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 02:03:22.748 7f03ba4a39a0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:26.736 7f03ba4a39a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 02:03:26.737 7f03ba4a39a0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 02:03:26.737 7f03ba4a39a0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 02:03:26.737 7f03ba4a39a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:03:26.737 7f03ba4a39a0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 02:03:26.737 7f03ba4a39a0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:03:26.738 7f03ba4a39a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 02:03:26.738 7f03ba4a39a0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 02:03:26.738 7f03ba4a39a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:26.738 7f03ba4a39a0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 02:03:26.738 7f03ba4a39a0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:26.738 7f03ba4a39a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:26.738 7f03ba4a39a0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758506554689242  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 02:03:26.738 7f03ba4a39a0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:30.742 7f03ba4a39a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 02:03:30.743 7f03ba4a39a0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 02:03:30.743 7f03ba4a39a0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 02:03:30.743 7f03ba4a39a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:03:30.743 7f03ba4a39a0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 02:03:30.743 7f03ba4a39a0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:03:30.743 7f03ba4a39a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 02:03:30.744 7f03ba4a39a0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 02:03:30.744 7f03ba4a39a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:30.744 7f03ba4a39a0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 02:03:30.744 7f03ba4a39a0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:30.744 7f03ba4a39a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:30.744 7f03ba4a39a0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758506554689242  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 02:03:30.744 7f03ba4a39a0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:34.747 7f03ba4a39a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 02:03:34.748 7f03ba4a39a0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 02:03:34.748 7f03ba4a39a0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 02:03:34.748 7f03ba4a39a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:03:34.748 7f03ba4a39a0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 02:03:34.748 7f03ba4a39a0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:03:34.748 7f03ba4a39a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 02:03:34.748 7f03ba4a39a0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 02:03:34.749 7f03ba4a39a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:34.749 7f03ba4a39a0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 02:03:34.749 7f03ba4a39a0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:34.749 7f03ba4a39a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:34.749 7f03ba4a39a0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758506554689242  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 02:03:34.749 7f03ba4a39a0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:35.082 7f03bd44c3c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/22/2025 02:03:35.331 7f03ba4a3360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:35.332 7f03ba4a3360 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  database_usage = [ <<  disk_space_used_1k = 14694268  disk_space_available_1k = 66396216  config_db_size_1k = 992  >>,  ]  >>  
09/22/2025 02:03:35.332 7f03ba4a3360 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:35.456 7f03bd44c3c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open
09/22/2025 02:03:35.456 7f03bd44c3c0 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-4162-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1758506615456662, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4162-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.50:8086'], status='Down', description='Connect to Idle on EvTcpConnectFail')]), flag_infos=None, description='Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
09/22/2025 02:03:35.458 7f03ba4a3360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:35.458 7f03ba4a3360 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 02:03:35.458 7f03ba4a3360 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:36.159 7f03bd44c3c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/22/2025 02:03:38.750 7f03ba4a3720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 02:03:38.751 7f03ba4a3720 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 02:03:38.752 7f03ba4a3720 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 02:03:38.752 7f03ba4a3720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:03:38.752 7f03ba4a3720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 02:03:38.752 7f03ba4a3720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:03:38.752 7f03ba4a3720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 02:03:38.753 7f03ba4a3720 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 02:03:38.753 7f03ba4a3720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:38.753 7f03ba4a3720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 02:03:38.753 7f03ba4a3720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:38.753 7f03ba4a3720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:38.753 7f03ba4a3720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758506554689242  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 02:03:38.753 7f03ba4a3720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:41.192 7f03bd44c3c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4162-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14694552, partition_space_available_1k=66395932, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1341272, cpu_share=0.28, mem_res=1327300, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=71240, cpu_share=0.02, mem_res=68132, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52708, cpu_share=0.01, mem_res=42632, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=6372560, free=10325812, buffers=642620, cached=15324120, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.44, five_min_avg=1.71, fifteen_min_avg=1.1, cpu_share=3.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/22/2025 02:03:41.192 7f03ba4a3360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:41.192 7f03ba4a3360 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  disk_usage_info = { /dev/vda1 : <<  partition_type = ext4  partition_space_used_1k = 14694552  partition_space_available_1k = 66395932  percentage_partition_space_used = 18  >> }  process_mem_cpu_usage = { cassandra : <<  mem_virt = 1341272  cpu_share = 0.28  mem_res = 1327300  >>zookeeper : <<  mem_virt = 71240  cpu_share = 0.02  mem_res = 68132  >>contrail-config-database-nodemgr : <<  mem_virt = 52708  cpu_share = 0.01  mem_res = 42632  >> }  system_mem_usage = <<  total = 32665112  used = 6372560  free = 10325812  buffers = 642620  cached = 15324120  node_type = config-database-node  >>  system_cpu_usage = <<  one_min_avg = 2.44  five_min_avg = 1.71  fifteen_min_avg = 1.1  cpu_share = 3.75  node_type = config-database-node  >>  >>  
09/22/2025 02:03:41.192 7f03ba4a3360 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:42.753 7f03ba4a3360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 02:03:42.754 7f03ba4a3360 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 02:03:42.755 7f03ba4a3360 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 02:03:42.755 7f03ba4a3360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:03:42.755 7f03ba4a3360 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 02:03:42.755 7f03ba4a3360 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:03:42.755 7f03ba4a3360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 02:03:42.755 7f03ba4a3360 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 02:03:42.755 7f03ba4a3360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:42.755 7f03ba4a3360 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 02:03:42.755 7f03ba4a3360 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:42.755 7f03ba4a3360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:42.756 7f03ba4a3360 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758506554689242  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 02:03:42.756 7f03ba4a3360 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:46.759 7f03ba4a3360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 02:03:46.760 7f03ba4a3360 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 02:03:46.760 7f03ba4a3360 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 02:03:46.760 7f03ba4a3360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:03:46.761 7f03ba4a3360 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 02:03:46.761 7f03ba4a3360 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:03:46.761 7f03ba4a3360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 02:03:46.761 7f03ba4a3360 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 02:03:46.761 7f03ba4a3360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:46.761 7f03ba4a3360 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 02:03:46.761 7f03ba4a3360 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:46.761 7f03ba4a3360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:46.761 7f03ba4a3360 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758506554689242  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 02:03:46.762 7f03ba4a3360 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:50.762 7f03ba4a3360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 02:03:50.764 7f03ba4a3360 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 02:03:50.764 7f03ba4a3360 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 02:03:50.764 7f03ba4a3360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:03:50.764 7f03ba4a3360 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 02:03:50.765 7f03ba4a3360 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:03:50.765 7f03ba4a3360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 02:03:50.765 7f03ba4a3360 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 02:03:50.765 7f03ba4a3360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:50.765 7f03ba4a3360 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 02:03:50.766 7f03ba4a3360 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:50.766 7f03ba4a3360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:50.766 7f03ba4a3360 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758506554689242  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 02:03:50.766 7f03ba4a3360 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:54.765 7f03ba4a3180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 02:03:54.766 7f03ba4a3180 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 02:03:54.767 7f03ba4a3180 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 02:03:54.767 7f03ba4a3180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:03:54.767 7f03ba4a3180 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 02:03:54.767 7f03ba4a3180 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:03:54.767 7f03ba4a3180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 02:03:54.767 7f03ba4a3180 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 02:03:54.767 7f03ba4a3180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:54.767 7f03ba4a3180 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 02:03:54.768 7f03ba4a3180 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:54.768 7f03ba4a3180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:54.768 7f03ba4a3180 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758506554689242  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 02:03:54.768 7f03ba4a3180 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:58.771 7f03ba4a3180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 02:03:58.772 7f03ba4a3180 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 02:03:58.772 7f03ba4a3180 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 02:03:58.772 7f03ba4a3180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:03:58.772 7f03ba4a3180 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 02:03:58.773 7f03ba4a3180 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:03:58.773 7f03ba4a3180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 02:03:58.773 7f03ba4a3180 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 02:03:58.773 7f03ba4a3180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:58.773 7f03ba4a3180 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 02:03:58.773 7f03ba4a3180 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:58.773 7f03ba4a3180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:03:58.773 7f03ba4a3180 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758506554689242  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 02:03:58.773 7f03ba4a3180 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:02.777 7f03ba4a3180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 02:04:02.778 7f03ba4a3180 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 02:04:02.778 7f03ba4a3180 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 02:04:02.778 7f03ba4a3180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:04:02.779 7f03ba4a3180 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 02:04:02.779 7f03ba4a3180 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:04:02.779 7f03ba4a3180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 02:04:02.779 7f03ba4a3180 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 02:04:02.779 7f03ba4a3180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:02.779 7f03ba4a3180 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 02:04:02.779 7f03ba4a3180 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:02.779 7f03ba4a3180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:02.780 7f03ba4a3180 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758506554689242  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 02:04:02.780 7f03ba4a3180 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:06.783 7f03ba4a3180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 02:04:06.784 7f03ba4a3180 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 02:04:06.784 7f03ba4a3180 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 02:04:06.784 7f03ba4a3180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:04:06.784 7f03ba4a3180 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 02:04:06.784 7f03ba4a3180 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:04:06.784 7f03ba4a3180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 02:04:06.785 7f03ba4a3180 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 02:04:06.785 7f03ba4a3180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:06.785 7f03ba4a3180 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 02:04:06.785 7f03ba4a3180 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:06.785 7f03ba4a3180 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:06.785 7f03ba4a3180 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758506554689242  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 02:04:06.785 7f03ba4a3180 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:10.786 7f03ba60ecc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 02:04:10.787 7f03ba60ecc0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 02:04:10.787 7f03ba60ecc0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 02:04:10.788 7f03ba60ecc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:04:10.788 7f03ba60ecc0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 02:04:10.788 7f03ba60ecc0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:04:10.788 7f03ba60ecc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 02:04:10.788 7f03ba60ecc0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 02:04:10.788 7f03ba60ecc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:10.788 7f03ba60ecc0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 02:04:10.788 7f03ba60ecc0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:10.788 7f03ba60ecc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:10.789 7f03ba60ecc0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758506554689242  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 02:04:10.789 7f03ba60ecc0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:14.790 7f03ba60ecc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 02:04:14.791 7f03ba60ecc0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 02:04:14.792 7f03ba60ecc0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 02:04:14.792 7f03ba60ecc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:04:14.792 7f03ba60ecc0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 02:04:14.792 7f03ba60ecc0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:04:14.792 7f03ba60ecc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 02:04:14.792 7f03ba60ecc0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 02:04:14.793 7f03ba60ecc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:14.793 7f03ba60ecc0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 02:04:14.793 7f03ba60ecc0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:14.793 7f03ba60ecc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:14.793 7f03ba60ecc0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758506554689242  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 02:04:14.793 7f03ba60ecc0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:18.794 7f03ba60ecc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 02:04:18.796 7f03ba60ecc0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 02:04:18.796 7f03ba60ecc0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 02:04:18.796 7f03ba60ecc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:04:18.796 7f03ba60ecc0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 02:04:18.796 7f03ba60ecc0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:04:18.796 7f03ba60ecc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 02:04:18.796 7f03ba60ecc0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 02:04:18.796 7f03ba60ecc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:18.797 7f03ba60ecc0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 02:04:18.797 7f03ba60ecc0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:18.797 7f03ba60ecc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:18.797 7f03ba60ecc0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758506554689242  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 02:04:18.797 7f03ba60ecc0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:22.796 7f03ba4a34a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 02:04:22.797 7f03ba4a34a0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 02:04:22.798 7f03ba4a34a0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 02:04:22.798 7f03ba4a34a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:04:22.798 7f03ba4a34a0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 02:04:22.798 7f03ba4a34a0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:04:22.798 7f03ba4a34a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 02:04:22.798 7f03ba4a34a0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 02:04:22.798 7f03ba4a34a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:22.798 7f03ba4a34a0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 02:04:22.798 7f03ba4a34a0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:22.799 7f03ba4a34a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:22.799 7f03ba4a34a0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758506554689242  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 02:04:22.799 7f03ba4a34a0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:26.802 7f03ba4a34a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 02:04:26.803 7f03ba4a34a0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 02:04:26.803 7f03ba4a34a0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 02:04:26.803 7f03ba4a34a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:04:26.803 7f03ba4a34a0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 02:04:26.803 7f03ba4a34a0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:04:26.803 7f03ba4a34a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 02:04:26.804 7f03ba4a34a0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 02:04:26.804 7f03ba4a34a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:26.804 7f03ba4a34a0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 02:04:26.804 7f03ba4a34a0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:26.804 7f03ba4a34a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:26.804 7f03ba4a34a0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758506554689242  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 02:04:26.804 7f03ba4a34a0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:30.806 7f03ba4a34a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 02:04:30.807 7f03ba4a34a0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 02:04:30.807 7f03ba4a34a0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 02:04:30.807 7f03ba4a34a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:04:30.808 7f03ba4a34a0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 02:04:30.808 7f03ba4a34a0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:04:30.808 7f03ba4a34a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 02:04:30.808 7f03ba4a34a0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 02:04:30.808 7f03ba4a34a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:30.808 7f03ba4a34a0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 02:04:30.808 7f03ba4a34a0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:30.808 7f03ba4a34a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:30.808 7f03ba4a34a0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758506554689242  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 02:04:30.809 7f03ba4a34a0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:34.810 7f03ba4a34a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 02:04:34.811 7f03ba4a34a0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 02:04:34.811 7f03ba4a34a0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 02:04:34.812 7f03ba4a34a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:04:34.812 7f03ba4a34a0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 02:04:34.812 7f03ba4a34a0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:04:34.812 7f03ba4a34a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 02:04:34.812 7f03ba4a34a0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 02:04:34.812 7f03ba4a34a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:34.812 7f03ba4a34a0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 02:04:34.813 7f03ba4a34a0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:34.813 7f03ba4a34a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:34.813 7f03ba4a34a0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758506554689242  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 02:04:34.813 7f03ba4a34a0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:35.136 7f03bd44c3c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/22/2025 02:04:35.449 7f03ba4a3540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:35.450 7f03ba4a3540 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  database_usage = [ <<  disk_space_used_1k = 16462828  disk_space_available_1k = 64627656  config_db_size_1k = 992  >>,  ]  >>  
09/22/2025 02:04:35.450 7f03ba4a3540 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:35.603 7f03bd44c3c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open
09/22/2025 02:04:36.310 7f03bd44c3c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/22/2025 02:04:38.813 7f03ba60ec20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 02:04:38.814 7f03ba60ec20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 02:04:38.815 7f03ba60ec20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 02:04:38.815 7f03ba60ec20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:04:38.815 7f03ba60ec20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 02:04:38.815 7f03ba60ec20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:04:38.815 7f03ba60ec20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 02:04:38.816 7f03ba60ec20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 02:04:38.816 7f03ba60ec20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:38.816 7f03ba60ec20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 02:04:38.816 7f03ba60ec20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:38.816 7f03ba60ec20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:38.816 7f03ba60ec20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758506554689242  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 02:04:38.816 7f03ba60ec20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:42.050 7f03bd44c3c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4162-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=16544544, partition_space_available_1k=64545940, percentage_partition_space_used=20, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1339048, cpu_share=0.04, mem_res=1325792, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=72940, cpu_share=0.0, mem_res=69828, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52812, cpu_share=0.01, mem_res=42632, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=7787404, free=7459148, buffers=652000, cached=16766560, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.35, five_min_avg=1.79, fifteen_min_avg=1.16, cpu_share=2.88, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/22/2025 02:04:42.051 7f03ba4a3860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:42.051 7f03ba4a3860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  disk_usage_info = { /dev/vda1 : <<  partition_type = ext4  partition_space_used_1k = 16544544  partition_space_available_1k = 64545940  percentage_partition_space_used = 20  >> }  process_mem_cpu_usage = { cassandra : <<  mem_virt = 1339048  cpu_share = 0.04  mem_res = 1325792  >>zookeeper : <<  mem_virt = 72940  cpu_share = 0.0  mem_res = 69828  >>contrail-config-database-nodemgr : <<  mem_virt = 52812  cpu_share = 0.01  mem_res = 42632  >> }  system_mem_usage = <<  total = 32665112  used = 7787404  free = 7459148  buffers = 652000  cached = 16766560  node_type = config-database-node  >>  system_cpu_usage = <<  one_min_avg = 2.35  five_min_avg = 1.79  fifteen_min_avg = 1.16  cpu_share = 2.88  node_type = config-database-node  >>  >>  
09/22/2025 02:04:42.051 7f03ba4a3860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:42.816 7f03ba4a3860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 02:04:42.817 7f03ba4a3860 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 02:04:42.817 7f03ba4a3860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 02:04:42.817 7f03ba4a3860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:04:42.817 7f03ba4a3860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 02:04:42.817 7f03ba4a3860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:04:42.817 7f03ba4a3860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 02:04:42.818 7f03ba4a3860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 02:04:42.818 7f03ba4a3860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:42.818 7f03ba4a3860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 02:04:42.818 7f03ba4a3860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:42.818 7f03ba4a3860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:42.818 7f03ba4a3860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758506554689242  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 02:04:42.818 7f03ba4a3860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:46.819 7f03ba4a3860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 02:04:46.820 7f03ba4a3860 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 02:04:46.820 7f03ba4a3860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 02:04:46.821 7f03ba4a3860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:04:46.821 7f03ba4a3860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 02:04:46.821 7f03ba4a3860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:04:46.821 7f03ba4a3860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 02:04:46.821 7f03ba4a3860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 02:04:46.821 7f03ba4a3860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:46.821 7f03ba4a3860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 02:04:46.821 7f03ba4a3860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:46.821 7f03ba4a3860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:46.821 7f03ba4a3860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758506554689242  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 02:04:46.822 7f03ba4a3860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:50.822 7f03ba4a3860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 02:04:50.823 7f03ba4a3860 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 02:04:50.824 7f03ba4a3860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 02:04:50.824 7f03ba4a3860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:04:50.824 7f03ba4a3860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 02:04:50.824 7f03ba4a3860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:04:50.824 7f03ba4a3860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 02:04:50.824 7f03ba4a3860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 02:04:50.824 7f03ba4a3860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:50.824 7f03ba4a3860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 02:04:50.824 7f03ba4a3860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:50.825 7f03ba4a3860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:50.825 7f03ba4a3860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758506554689242  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 02:04:50.825 7f03ba4a3860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:54.826 7f03ba60ef40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 02:04:54.827 7f03ba60ef40 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 02:04:54.827 7f03ba60ef40 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 02:04:54.827 7f03ba60ef40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:04:54.828 7f03ba60ef40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 02:04:54.828 7f03ba60ef40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:04:54.828 7f03ba60ef40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 02:04:54.828 7f03ba60ef40 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 02:04:54.828 7f03ba60ef40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:54.828 7f03ba60ef40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 02:04:54.828 7f03ba60ef40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:54.828 7f03ba60ef40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:54.828 7f03ba60ef40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758506554689242  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 02:04:54.828 7f03ba60ef40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:04:58.832 7f03ba60ef40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 02:04:58.833 7f03ba60ef40 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected
09/22/2025 02:04:58.833 7f03ba60ef40 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 02:04:58.833 7f03ba60ef40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:04:58.833 7f03ba60ef40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 02:04:58.833 7f03ba60ef40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:04:58.834 7f03ba60ef40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect]
09/22/2025 02:04:58.834 7f03ba60ef40 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6
09/22/2025 02:04:58.834 7f03ba60ef40 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit]
09/22/2025 02:04:58.834 7f03ba60ef40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
09/22/2025 02:04:58.834 7f03ba60ef40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
09/22/2025 02:05:07.526 7f03ba4a3e00 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed
09/22/2025 02:05:07.526 7f03bb2055e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit]
09/22/2025 02:05:07.527 7f03bb2055e0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle]
09/22/2025 02:05:07.527 7f03bb2055e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:05:07.527 7f03bb2055e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = ClientInit to Idle on EvTcpClose  >>,  ]  description = Collector:Collector[ClientInit to Idle on EvTcpClose] connection down  >>,  ]  >>  
09/22/2025 02:05:07.527 7f03bb2055e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:05:07.527 7f03bb2055e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:05:07.528 7f03bb2055e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 1  pid = 1  http_port = 8112  start_time = 1758506554689242  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 02:05:07.528 7f03bb2055e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 02:05:11.532 7f03bb2055e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 02:05:11.533 7f03bb2055e0 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected
09/22/2025 02:05:11.533 7f03bb2055e0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 02:05:11.533 7f03bb2055e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:05:11.533 7f03bb2055e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4162-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 02:05:11.533 7f03bb2055e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 02:05:11.533 7f03bb2055e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect]
09/22/2025 02:05:11.534 7f03bb2055e0 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6
09/22/2025 02:05:11.534 7f03bb2055e0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit]
09/22/2025 02:05:11.534 7f03bb2055e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
09/22/2025 02:05:11.534 7f03bb2055e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
09/22/2025 02:05:11.539 7f03ba4bb180 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001017
09/22/2025 02:05:11.540 7f03ba4bb180 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient]
09/22/2025 02:05:11.540 7f03bb2055e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit]
09/22/2025 02:05:11.540 7f03bb2055e0 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0
09/22/2025 02:05:11.541 7f03bb2055e0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-4162-1.: 3> in the [ObjectConfigDatabaseInfo:DatabaseUsage] map
09/22/2025 02:05:11.541 7f03bb2055e0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-4162-1.: 85> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
09/22/2025 02:05:11.541 7f03bb2055e0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-4162-1.:ConfigDatabase:contrail-config-database-nodemgr:0: 39> in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map
09/22/2025 02:05:11.541 7f03bb2055e0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established]
09/22/2025 02:05:11.542 7f03ba60ef40 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG]
09/22/2025 02:05:11.542 7f03ba60ef40 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID]
09/22/2025 02:05:17.545 7f03ba4bb180 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001438
09/22/2025 02:05:17.545 7f03ba4bb180 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient]
09/22/2025 02:05:17.546 7f03ba4a3720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established]
09/22/2025 02:05:17.546 7f03ba4a3720 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 3
09/22/2025 02:05:17.548 7f03ba4a3720 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-4162-1.: 3> in the [ObjectConfigDatabaseInfo:DatabaseUsage] map
09/22/2025 02:05:17.548 7f03ba4a3720 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-4162-1.: 86> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
09/22/2025 02:05:17.549 7f03ba4a3720 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-4162-1.:ConfigDatabase:contrail-config-database-nodemgr:0: 40> in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map
09/22/2025 02:05:17.549 7f03ba4a3ea0 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG]
09/22/2025 02:05:17.550 7f03ba4a3ea0 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID]
09/22/2025 02:05:35.121 7f03bd44c3c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/22/2025 02:05:35.619 7f03bd44c3c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open
09/22/2025 02:05:36.386 7f03bd44c3c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/22/2025 02:05:42.306 7f03bd44c3c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4162-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=18271940, partition_space_available_1k=62818544, percentage_partition_space_used=23, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1339824, cpu_share=0.04, mem_res=1326404, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=104080, cpu_share=0.04, mem_res=100416, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52828, cpu_share=0.01, mem_res=42628, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=9463860, free=3663172, buffers=687540, cached=18850540, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.29, five_min_avg=2.39, fifteen_min_avg=1.4, cpu_share=4.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/22/2025 02:06:35.235 7f03bd44c3c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/22/2025 02:06:35.789 7f03bd44c3c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open
09/22/2025 02:06:36.623 7f03bd44c3c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/22/2025 02:06:42.534 7f03bd44c3c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4162-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19540712, partition_space_available_1k=61549772, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1337284, cpu_share=0.03, mem_res=1326428, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=108564, cpu_share=0.0, mem_res=105244, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44900, cpu_share=0.02, mem_res=42628, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=8904400, free=18461300, buffers=694616, cached=4604796, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.1, five_min_avg=2.67, fifteen_min_avg=1.56, cpu_share=5.62, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/22/2025 02:07:06.950 7f03ba60e900 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-4162-1.: 88> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
09/22/2025 02:07:09.036 7f03ba4a3720 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-4162-1.: 88> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
09/22/2025 02:07:35.147 7f03bd44c3c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/22/2025 02:07:35.654 7f03bd44c3c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open
09/22/2025 02:07:36.401 7f03bd44c3c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/22/2025 02:07:42.328 7f03bd44c3c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4162-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=20114900, partition_space_available_1k=60975584, percentage_partition_space_used=25, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1337348, cpu_share=0.04, mem_res=1326456, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=108900, cpu_share=0.0, mem_res=105580, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=48140, cpu_share=0.02, mem_res=42628, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=9617248, free=16324184, buffers=712188, cached=6011492, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.29, five_min_avg=2.7, fifteen_min_avg=1.65, cpu_share=3.88, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/22/2025 02:08:35.169 7f03bd44c3c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/22/2025 02:08:35.703 7f03bd44c3c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open
09/22/2025 02:08:36.459 7f03bd44c3c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/22/2025 02:08:42.180 7f03bd44c3c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4162-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21050364, partition_space_available_1k=60040120, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1339844, cpu_share=0.11, mem_res=1328192, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=110744, cpu_share=0.0, mem_res=107408, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=48148, cpu_share=0.02, mem_res=42628, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=9987816, free=15705660, buffers=739540, cached=6232096, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.02, five_min_avg=2.72, fifteen_min_avg=1.72, cpu_share=3.88, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/22/2025 02:09:35.154 7f03bd44c3c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/22/2025 02:09:35.698 7f03bd44c3c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open
09/22/2025 02:09:36.491 7f03bd44c3c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/22/2025 02:09:42.201 7f03bd44c3c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4162-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21081704, partition_space_available_1k=60008780, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1345116, cpu_share=0.19, mem_res=1333200, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=128108, cpu_share=0.02, mem_res=124672, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=48152, cpu_share=0.02, mem_res=42628, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=10769752, free=14875736, buffers=739672, cached=6279952, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.59, five_min_avg=2.91, fifteen_min_avg=1.85, cpu_share=4.62, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/22/2025 02:10:35.181 7f03bd44c3c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/22/2025 02:10:35.801 7f03bd44c3c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open
09/22/2025 02:10:36.665 7f03bd44c3c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/22/2025 02:10:42.246 7f03bd44c3c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4162-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21225752, partition_space_available_1k=59864732, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1346636, cpu_share=0.1, mem_res=1334300, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=131052, cpu_share=0.0, mem_res=127580, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=48152, cpu_share=0.02, mem_res=42628, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=10594224, free=14873788, buffers=740116, cached=6456984, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.23, five_min_avg=3.55, fifteen_min_avg=2.12, cpu_share=5.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/22/2025 02:11:35.168 7f03bd44c3c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/22/2025 02:11:35.774 7f03bd44c3c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open
09/22/2025 02:11:36.578 7f03bd44c3c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/22/2025 02:11:42.232 7f03bd44c3c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4162-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21263560, partition_space_available_1k=59826924, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1346832, cpu_share=0.12, mem_res=1334324, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=135028, cpu_share=0.0, mem_res=131508, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=48160, cpu_share=0.02, mem_res=42628, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=10929348, free=14508656, buffers=740180, cached=6486928, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.47, five_min_avg=3.49, fifteen_min_avg=2.19, cpu_share=4.38, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/22/2025 02:12:35.193 7f03bd44c3c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/22/2025 02:12:35.729 7f03bd44c3c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open
09/22/2025 02:12:36.801 7f03bd44c3c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/22/2025 02:12:42.779 7f03bd44c3c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4162-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21329500, partition_space_available_1k=59760984, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1363860, cpu_share=0.1, mem_res=1351124, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=139896, cpu_share=0.0, mem_res=136336, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=48168, cpu_share=0.02, mem_res=42628, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=11099940, free=14280000, buffers=740432, cached=6544740, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.05, five_min_avg=3.47, fifteen_min_avg=2.26, cpu_share=4.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/22/2025 02:13:35.253 7f03bd44c3c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/22/2025 02:13:35.967 7f03bd44c3c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open
09/22/2025 02:13:37.203 7f03bd44c3c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/22/2025 02:13:43.121 7f03bd44c3c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4162-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21350220, partition_space_available_1k=59740264, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1364484, cpu_share=0.07, mem_res=1351084, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=141952, cpu_share=0.01, mem_res=138372, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=48176, cpu_share=0.02, mem_res=42628, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=10463244, free=14894736, buffers=740508, cached=6566624, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.55, five_min_avg=3.15, fifteen_min_avg=2.23, cpu_share=3.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/22/2025 02:14:35.223 7f03bd44c3c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/22/2025 02:14:35.720 7f03bd44c3c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open
09/22/2025 02:14:36.501 7f03bd44c3c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/22/2025 02:14:42.220 7f03bd44c3c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4162-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21369056, partition_space_available_1k=59721428, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1364652, cpu_share=0.06, mem_res=1351152, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=143540, cpu_share=0.0, mem_res=139944, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=48172, cpu_share=0.01, mem_res=42628, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=10713708, free=14612696, buffers=740580, cached=6598128, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.7, five_min_avg=3.06, fifteen_min_avg=2.25, cpu_share=3.62, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/22/2025 02:15:35.189 7f03bd44c3c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/22/2025 02:15:35.671 7f03bd44c3c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open
09/22/2025 02:15:36.409 7f03bd44c3c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/22/2025 02:15:42.229 7f03bd44c3c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4162-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21394988, partition_space_available_1k=59695496, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1376932, cpu_share=0.09, mem_res=1363180, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=145936, cpu_share=0.0, mem_res=142324, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=48176, cpu_share=0.02, mem_res=42628, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=10767064, free=14527508, buffers=740656, cached=6629884, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.3, five_min_avg=3.14, fifteen_min_avg=2.33, cpu_share=4.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/22/2025 02:16:35.208 7f03bd44c3c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/22/2025 02:16:35.685 7f03bd44c3c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open
09/22/2025 02:16:36.395 7f03bd44c3c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/22/2025 02:16:42.428 7f03bd44c3c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4162-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21511400, partition_space_available_1k=59579084, percentage_partition_space_used=27, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1376964, cpu_share=0.06, mem_res=1363188, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=147412, cpu_share=0.01, mem_res=143788, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=48184, cpu_share=0.02, mem_res=42628, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=10754120, free=14431984, buffers=740712, cached=6738296, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.45, five_min_avg=3.2, fifteen_min_avg=2.4, cpu_share=4.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/22/2025 02:17:35.208 7f03bd44c3c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/22/2025 02:17:35.668 7f03bd44c3c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open
09/22/2025 02:17:36.378 7f03bd44c3c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/22/2025 02:17:42.169 7f03bd44c3c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4162-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21517368, partition_space_available_1k=59573116, percentage_partition_space_used=27, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1376992, cpu_share=0.04, mem_res=1363204, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=147916, cpu_share=0.0, mem_res=144288, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=48184, cpu_share=0.02, mem_res=42628, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=10502920, free=14677304, buffers=740728, cached=6744160, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.95, five_min_avg=2.83, fifteen_min_avg=2.32, cpu_share=2.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/22/2025 02:18:35.230 7f03bd44c3c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/22/2025 02:18:35.706 7f03bd44c3c0 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open
09/22/2025 02:18:36.452 7f03bd44c3c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/22/2025 02:18:42.301 7f03bd44c3c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4162-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21530624, partition_space_available_1k=59559860, percentage_partition_space_used=27, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1377712, cpu_share=0.08, mem_res=1363212, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=150476, cpu_share=0.0, mem_res=146572, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=48188, cpu_share=0.02, mem_res=42628, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=10331716, free=14778452, buffers=743136, cached=6811808, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.58, five_min_avg=3.15, fifteen_min_avg=2.46, cpu_share=3.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/22/2025 02:19:05.850 7f03ba4a3f40 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-4162-1.: 100> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map