09/22/2025 05:04:27.608 7f42cc817df0 [contrail-config-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0]
09/22/2025 05:04:27.608 7f42cc817df0 [contrail-config-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False]
09/22/2025 05:04:27.608 7f42cc817df0 [contrail-config-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False]
09/22/2025 05:04:27.633 7f42c6fb9548 [contrail-config-database-nodemgr] [ERROR]: Cannot write http_port 8112 to /tmp/contrail-config-database-nodemgr.0.http_port
09/22/2025 05:04:27.634 7f42c6fb9548 [contrail-config-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8112
09/22/2025 05:04:27.634 7f42c6fb9d48 [contrail-config-database-nodemgr] [INFO]: Processing event[EvStart] in state[none]
09/22/2025 05:04:27.635 7f42c6fb9d48 [contrail-config-database-nodemgr] [DEBUG]: Add uve <cn-jenkins-deploy-platform-ansible-os-4163-2., 1> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
09/22/2025 05:04:27.635 7f42c6fb9d48 [contrail-config-database-nodemgr] [DEBUG]: Add uve <cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0, 1> in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map
09/22/2025 05:04:27.635 7f42c6fb9d48 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle]
09/22/2025 05:04:27.635 7f42c6fb9d48 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:27.635 7f42c6fb9d48 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>,  ]  description = Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
09/22/2025 05:04:27.635 7f42c6fb9d48 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:27.636 7f42c6fb9d48 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:27.636 7f42c6fb9d48 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip =   collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 1  >>  
09/22/2025 05:04:27.636 7f42c6fb9d48 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:27.678 7f42cc817df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 95034453791907801992148824685902008011444469441675698532249073803127961644519, 'start': '1758517465000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'zookeeper', 'group': 'zookeeper', 'pid': 14896535570721688359471846882725439527744837604198329236370389205576510147274, 'start': '1758517465000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-config-database-nodemgr', 'group': 'contrail-config-database-nodemgr', 'pid': 33336458802046305178427569149550435183280111497783743554489341708465100119346, 'start': '1758517465000000', 'statename': 'PROCESS_STATE_RUNNING'}]'
09/22/2025 05:04:27.799 7f42c6e1c048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:27.800 7f42c6e1c048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  build_info = {"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-09-22 01:08:48.599626", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11203-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}  system_cpu_info = <<  num_socket = 1  num_cpu = 4  num_core_per_socket = 4  num_thread_per_core = 1  >>  running_package_version = v0+nightly.r24.1.centos-nightly_r24.1_centos.el7  installed_package_version = v0+nightly.r24.1.centos-nightly_r24.1_centos.el7  >>  
09/22/2025 05:04:27.800 7f42c6e1c048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:27.800 7f42cc817df0 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-4163-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1758517467800767, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4163-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
09/22/2025 05:04:27.801 7f42cc817df0 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-4163-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1758517467801092, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4163-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1758517465000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1758517465000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1758517465000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-09-22 01:08:48.599626", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11203-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
09/22/2025 05:04:27.802 7f42c6e1c048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:27.803 7f42c6e1c048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>,  ]  description = Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
09/22/2025 05:04:27.803 7f42c6e1c048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:27.803 7f42c6e1c048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:27.803 7f42c6e1c048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  deleted = False  process_info = [ <<  process_name = cassandra  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1758517465000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>, <<  process_name = zookeeper  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1758517465000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>, <<  process_name = contrail-config-database-nodemgr  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1758517465000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>,  ]  build_info = {"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-09-22 01:08:48.599626", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11203-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}  >>  
09/22/2025 05:04:27.803 7f42c6e1c048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:27.988 7f42cc817df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/22/2025 05:04:28.153 7f42cc817df0 [contrail-config-database-nodemgr] [DEBUG]: Add uve <cn-jenkins-deploy-platform-ansible-os-4163-2., 1> in the [ObjectConfigDatabaseInfo:DatabaseUsage] map
09/22/2025 05:04:28.154 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:28.155 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  database_usage = [ <<  disk_space_used_1k = 4711556  disk_space_available_1k = 76378928  config_db_size_1k = 4  >>,  ]  >>  
09/22/2025 05:04:28.155 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:28.557 7f42cc817df0 [contrail-config-database-nodemgr] [ERROR]: Failed to connect to database by CQL: Result: 1
09/22/2025 05:04:28.557 7f42cc817df0 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-4163-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1758517468557476, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4163-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
09/22/2025 05:04:28.558 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:28.559 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
09/22/2025 05:04:28.559 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:30.132 7f42cc817df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/22/2025 05:04:31.635 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:04:31.637 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:04:31.637 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:04:31.637 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:04:31.637 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.48:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:04:31.638 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:04:31.638 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:04:31.638 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:04:31.638 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:31.638 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.48:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:04:31.638 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:31.638 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:31.638 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.48:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:04:31.638 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:32.682 7f42cc817df0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr config_db_uuid'. Error is: Result: 1
09/22/2025 05:04:33.763 7f42cc817df0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr to_bgp_keyspace'. Error is: Result: 1
09/22/2025 05:04:34.575 7f42cc817df0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr svc_monitor_keyspace'. Error is: Result: 1
09/22/2025 05:04:35.357 7f42cc817df0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr DISCOVERY_SERVER'. Error is: Result: 1
09/22/2025 05:04:35.637 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:04:35.638 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:04:35.639 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:04:35.639 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:04:35.639 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.38:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:04:35.640 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:04:35.640 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:04:35.640 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:04:35.641 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:35.641 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.38:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:04:35.641 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:35.641 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:35.642 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.38:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:04:35.642 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:36.300 7f42cc817df0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr useragent'. Error is: Result: 1
09/22/2025 05:04:37.382 7f42cc817df0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr dm_keyspace'. Error is: Result: 1
09/22/2025 05:04:39.639 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:04:39.641 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:04:39.641 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:04:39.641 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:04:39.641 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.254:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:04:39.641 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:04:39.642 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:04:39.642 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:04:39.642 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:39.642 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.254:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:04:39.642 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:39.643 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:39.643 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.254:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:04:39.643 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:42.404 7f42cc817df0 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-4163-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1758517482403997, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4163-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1758517465000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1758517465000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1758517465000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-09-22 01:08:48.599626", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11203-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
09/22/2025 05:04:42.404 7f42cc817df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4163-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=5064104, partition_space_available_1k=76026380, percentage_partition_space_used=6, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1240036, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57684, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51048, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=1725056, free=9343240, buffers=102668, cached=5004676, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.74, five_min_avg=0.48, fifteen_min_avg=0.2, cpu_share=0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/22/2025 05:04:42.404 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:42.404 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  deleted = False  process_info = [ <<  process_name = cassandra  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1758517465000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>, <<  process_name = zookeeper  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1758517465000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>, <<  process_name = contrail-config-database-nodemgr  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1758517465000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>,  ]  build_info = {"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-09-22 01:08:48.599626", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11203-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}  >>  
09/22/2025 05:04:42.405 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:42.405 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:42.405 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  disk_usage_info = { /dev/vda1 : <<  partition_type = ext4  partition_space_used_1k = 5064104  partition_space_available_1k = 76026380  percentage_partition_space_used = 6  >> }  all_core_file_list = [  ]  process_mem_cpu_usage = { cassandra : <<  mem_virt = 1240036  cpu_share = 0  mem_res = 0  >>zookeeper : <<  mem_virt = 57684  cpu_share = 0  mem_res = 0  >>contrail-config-database-nodemgr : <<  mem_virt = 51048  cpu_share = 0  mem_res = 0  >> }  system_mem_usage = <<  total = 16175640  used = 1725056  free = 9343240  buffers = 102668  cached = 5004676  node_type = config-database-node  >>  system_cpu_usage = <<  one_min_avg = 1.74  five_min_avg = 0.48  fifteen_min_avg = 0.2  cpu_share = 0  node_type = config-database-node  >>  >>  
09/22/2025 05:04:42.405 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:43.642 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:04:43.643 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:04:43.643 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:04:43.644 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:04:43.644 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.48:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:04:43.644 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:04:43.644 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:04:43.645 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:04:43.645 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:43.645 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.48:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:04:43.645 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:43.645 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:43.645 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.48:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:04:43.645 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:47.644 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:04:47.646 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:04:47.646 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:04:47.646 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:04:47.646 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.38:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:04:47.646 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:04:47.646 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:04:47.647 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:04:47.648 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:47.648 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.38:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:04:47.648 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:47.648 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:47.648 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.38:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:04:47.649 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:51.649 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:04:51.650 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:04:51.651 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:04:51.651 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:04:51.651 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.254:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:04:51.651 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:04:51.651 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:04:51.651 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:04:51.652 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:51.652 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.254:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:04:51.652 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:51.652 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:51.652 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.254:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:04:51.652 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:55.652 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:04:55.654 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:04:55.654 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:04:55.655 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:04:55.655 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.48:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:04:55.655 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:04:55.655 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:04:55.656 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:04:55.658 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:55.658 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.48:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:04:55.658 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:55.658 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:55.658 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.48:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:04:55.658 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:59.655 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:04:59.656 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:04:59.657 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:04:59.657 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:04:59.657 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.38:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:04:59.657 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:04:59.657 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:04:59.658 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:04:59.659 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:59.659 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.38:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:04:59.659 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:59.659 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:04:59.659 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.38:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:04:59.659 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:03.656 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:05:03.658 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:05:03.659 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:05:03.659 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:05:03.659 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.254:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:05:03.659 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:05:03.659 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:05:03.659 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:05:03.659 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:03.660 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.254:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:05:03.660 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:03.660 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:03.660 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.254:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:05:03.660 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:07.660 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:05:07.662 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:05:07.662 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:05:07.662 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:05:07.663 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.48:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:05:07.663 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:05:07.663 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:05:07.663 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:05:07.663 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:07.664 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.48:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:05:07.664 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:07.664 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:07.664 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.48:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:05:07.664 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:11.664 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:05:11.665 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:05:11.666 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:05:11.666 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:05:11.666 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.38:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:05:11.666 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:05:11.666 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:05:11.666 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:05:11.666 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:11.667 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.38:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:05:11.667 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:11.667 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:11.667 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.38:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:05:11.667 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:15.668 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:05:15.669 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:05:15.670 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:05:15.670 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:05:15.670 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.254:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:05:15.670 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:05:15.670 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:05:15.670 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:05:15.670 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:15.671 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.254:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:05:15.671 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:15.671 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:15.671 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.254:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:05:15.671 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:19.673 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:05:19.675 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:05:19.675 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:05:19.675 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:05:19.675 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.48:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:05:19.675 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:05:19.676 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:05:19.676 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:05:19.676 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:19.676 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.48:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:05:19.676 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:19.676 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:19.676 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.48:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:05:19.676 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:23.676 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:05:23.678 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:05:23.678 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:05:23.678 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:05:23.678 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.38:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:05:23.678 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:05:23.678 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:05:23.679 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:05:23.679 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:23.679 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.38:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:05:23.679 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:23.679 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:23.679 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.38:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:05:23.679 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:27.680 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:05:27.682 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:05:27.682 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:05:27.682 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:05:27.682 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.254:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:05:27.683 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:05:27.683 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:05:27.683 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:05:27.683 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:27.683 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.254:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:05:27.683 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:27.683 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:27.684 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.254:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:05:27.684 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:28.161 7f42cc817df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/22/2025 05:05:28.478 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:28.478 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  database_usage = [ <<  disk_space_used_1k = 6266052  disk_space_available_1k = 74824432  config_db_size_1k = 1504  >>,  ]  >>  
09/22/2025 05:05:28.478 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:29.041 7f42cc817df0 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-4163-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1758517529041280, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4163-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.254:8086'], status='Down', description='Connect to Idle on EvTcpConnectFail')]), flag_infos=None, description='Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
09/22/2025 05:05:29.042 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:29.043 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.254:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:05:29.043 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:29.862 7f42cc817df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/22/2025 05:05:31.683 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:05:31.684 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:05:31.685 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:05:31.685 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:05:31.685 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.48:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:05:31.685 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:05:31.685 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:05:31.685 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:05:31.685 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:31.686 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.48:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:05:31.686 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:31.686 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:31.686 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.48:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:05:31.686 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:35.655 7f42cc817df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4163-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6283556, partition_space_available_1k=74806928, percentage_partition_space_used=8, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1324204, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=63692, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51496, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=2232904, free=7980724, buffers=204356, cached=5757656, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.12, five_min_avg=1.11, fifteen_min_avg=0.43, cpu_share=10.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/22/2025 05:05:35.655 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:35.656 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  disk_usage_info = { /dev/vda1 : <<  partition_type = ext4  partition_space_used_1k = 6283556  partition_space_available_1k = 74806928  percentage_partition_space_used = 8  >> }  process_mem_cpu_usage = { cassandra : <<  mem_virt = 1324204  cpu_share = 0  mem_res = 0  >>zookeeper : <<  mem_virt = 63692  cpu_share = 0  mem_res = 0  >>contrail-config-database-nodemgr : <<  mem_virt = 51496  cpu_share = 0  mem_res = 0  >> }  system_mem_usage = <<  total = 16175640  used = 2232904  free = 7980724  buffers = 204356  cached = 5757656  node_type = config-database-node  >>  system_cpu_usage = <<  one_min_avg = 3.12  five_min_avg = 1.11  fifteen_min_avg = 0.43  cpu_share = 10.25  node_type = config-database-node  >>  >>  
09/22/2025 05:05:35.656 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:35.685 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:05:35.686 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:05:35.686 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:05:35.687 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:05:35.687 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.38:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:05:35.687 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:05:35.687 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:05:35.687 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:05:35.688 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:35.688 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.38:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:05:35.688 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:35.688 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:35.688 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.38:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:05:35.688 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:39.688 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:05:39.690 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:05:39.691 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:05:39.691 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:05:39.691 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.254:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:05:39.691 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:05:39.691 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:05:39.692 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:05:39.692 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:39.692 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.254:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:05:39.693 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:39.693 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:39.693 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.254:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:05:39.693 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:43.691 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:05:43.693 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:05:43.693 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:05:43.693 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:05:43.694 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.48:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:05:43.694 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:05:43.694 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:05:43.694 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:05:43.695 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:43.695 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.48:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:05:43.695 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:43.695 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:43.695 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.48:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:05:43.695 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:47.694 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:05:47.694 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:05:47.695 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:05:47.695 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:05:47.695 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.38:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:05:47.695 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:05:47.695 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:05:47.695 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:05:47.695 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:47.695 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.38:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:05:47.696 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:47.696 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:47.696 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.38:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:05:47.696 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:51.696 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:05:51.698 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:05:51.698 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:05:51.698 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:05:51.698 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.254:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:05:51.698 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:05:51.698 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:05:51.698 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:05:51.698 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:51.699 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.254:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:05:51.699 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:51.699 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:51.699 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.254:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:05:51.699 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:55.700 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:05:55.702 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:05:55.702 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:05:55.702 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:05:55.702 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.48:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:05:55.702 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:05:55.702 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:05:55.703 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:05:55.704 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:55.705 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.48:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:05:55.705 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:55.705 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:55.706 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.48:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:05:55.706 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:59.702 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:05:59.704 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:05:59.704 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:05:59.704 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:05:59.705 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.38:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:05:59.705 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:05:59.705 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:05:59.705 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:05:59.705 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:59.706 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.38:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:05:59.706 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:59.706 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:05:59.706 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.38:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:05:59.706 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:03.704 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:06:03.706 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:06:03.706 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:06:03.706 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:06:03.706 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.254:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:06:03.706 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:06:03.706 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:06:03.707 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:06:03.707 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:03.707 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.254:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:06:03.707 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:03.707 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:03.707 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.254:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:06:03.707 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:07.710 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:06:07.712 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:06:07.712 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:06:07.712 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:06:07.712 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.48:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:06:07.712 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:06:07.712 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:06:07.713 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:06:07.713 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:07.713 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.48:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:06:07.713 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:07.713 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:07.713 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.48:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:06:07.713 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:11.713 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:06:11.714 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:06:11.714 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:06:11.714 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:06:11.715 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.38:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:06:11.715 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:06:11.715 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:06:11.715 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:06:11.715 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:11.715 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.38:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:06:11.715 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:11.715 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:11.716 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.38:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:06:11.716 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:15.716 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:06:15.717 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:06:15.717 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:06:15.718 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:06:15.718 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.254:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:06:15.718 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:06:15.718 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:06:15.718 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:06:15.718 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:15.718 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.254:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:06:15.718 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:15.719 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:15.719 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.254:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:06:15.719 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:19.720 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:06:19.722 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:06:19.722 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:06:19.722 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:06:19.722 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.48:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:06:19.722 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:06:19.722 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:06:19.723 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:06:19.723 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:19.723 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.48:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:06:19.723 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:19.723 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:19.723 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.48:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:06:19.723 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:23.724 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:06:23.726 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:06:23.726 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:06:23.726 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:06:23.726 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.38:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:06:23.726 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:06:23.726 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:06:23.726 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:06:23.726 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:23.727 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.38:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:06:23.727 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:23.727 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:23.727 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.38:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:06:23.727 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:27.728 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:06:27.730 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:06:27.730 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:06:27.730 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:06:27.730 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.254:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:06:27.730 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:06:27.730 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:06:27.731 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:06:27.731 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:27.731 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.254:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:06:27.731 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:27.731 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:27.731 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.254:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:06:27.731 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:28.011 7f42cc817df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/22/2025 05:06:28.309 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:28.309 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  database_usage = [ <<  disk_space_used_1k = 6411340  disk_space_available_1k = 74679144  config_db_size_1k = 1728  >>,  ]  >>  
09/22/2025 05:06:28.309 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:30.319 7f42cc817df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/22/2025 05:06:31.730 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:06:31.732 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:06:31.732 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:06:31.732 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:06:31.733 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.48:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:06:31.733 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:06:31.733 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:06:31.733 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:06:31.733 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:31.734 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.48:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:06:31.734 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:31.734 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:31.734 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.48:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:06:31.734 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:35.673 7f42cc817df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4163-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6584380, partition_space_available_1k=74506104, percentage_partition_space_used=8, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1659152, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=75736, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51724, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=3029588, free=6575176, buffers=235848, cached=6335028, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.61, five_min_avg=1.34, fifteen_min_avg=0.56, cpu_share=8.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/22/2025 05:06:35.673 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:35.674 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  disk_usage_info = { /dev/vda1 : <<  partition_type = ext4  partition_space_used_1k = 6584380  partition_space_available_1k = 74506104  percentage_partition_space_used = 8  >> }  process_mem_cpu_usage = { cassandra : <<  mem_virt = 1659152  cpu_share = 0  mem_res = 0  >>zookeeper : <<  mem_virt = 75736  cpu_share = 0  mem_res = 0  >>contrail-config-database-nodemgr : <<  mem_virt = 51724  cpu_share = 0  mem_res = 0  >> }  system_mem_usage = <<  total = 16175640  used = 3029588  free = 6575176  buffers = 235848  cached = 6335028  node_type = config-database-node  >>  system_cpu_usage = <<  one_min_avg = 2.61  five_min_avg = 1.34  fifteen_min_avg = 0.56  cpu_share = 8.25  node_type = config-database-node  >>  >>  
09/22/2025 05:06:35.674 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:35.733 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:06:35.734 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:06:35.734 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:06:35.734 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:06:35.735 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.38:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:06:35.735 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:06:35.735 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:06:35.735 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:06:35.735 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:35.735 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.38:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:06:35.735 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:35.735 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:35.736 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.38:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:06:35.736 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:39.737 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:06:39.738 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:06:39.738 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:06:39.739 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:06:39.739 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.254:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:06:39.739 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:06:39.739 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:06:39.739 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:06:39.739 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:39.739 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.254:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:06:39.739 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:39.740 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:39.740 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.254:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:06:39.740 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:43.739 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:06:43.740 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:06:43.740 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:06:43.740 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:06:43.741 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.48:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:06:43.741 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:06:43.741 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:06:43.741 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:06:43.741 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:43.741 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.48:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:06:43.742 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:43.742 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:43.742 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.48:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:06:43.742 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:47.740 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:06:47.741 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:06:47.742 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:06:47.742 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:06:47.742 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.38:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:06:47.742 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:06:47.742 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:06:47.742 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:06:47.742 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:47.742 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.38:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:06:47.742 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:47.743 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:47.743 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.38:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:06:47.743 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:51.744 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:06:51.746 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:06:51.746 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:06:51.746 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:06:51.746 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.254:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:06:51.746 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:06:51.746 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:06:51.746 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:06:51.746 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:51.747 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.254:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:06:51.747 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:51.747 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:51.747 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.254:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:06:51.747 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:55.748 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:06:55.750 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:06:55.750 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:06:55.750 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:06:55.750 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.48:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:06:55.750 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:06:55.750 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:06:55.751 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:06:55.751 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:55.751 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.48:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:06:55.751 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:55.751 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:55.751 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.48:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:06:55.751 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:59.751 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:06:59.752 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:06:59.752 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:06:59.752 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:06:59.752 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.38:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:06:59.752 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:06:59.752 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:06:59.753 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:06:59.753 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:59.753 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.38:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:06:59.753 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:59.753 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:06:59.753 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.38:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:06:59.753 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:07:03.753 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:07:03.754 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:07:03.754 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:07:03.754 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:07:03.755 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.254:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:07:03.755 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:07:03.755 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:07:03.755 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:07:03.755 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:07:03.756 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.254:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:07:03.756 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:07:03.756 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:07:03.756 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.254:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:07:03.756 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:07:07.756 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:07:07.758 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:07:07.758 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:07:07.758 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:07:07.758 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.48:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:07:07.758 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:07:07.758 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:07:07.759 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:07:07.759 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:07:07.759 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.48:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:07:07.759 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:07:07.759 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:07:07.759 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.48:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:07:07.759 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:07:11.762 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:07:11.763 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:07:11.763 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:07:11.763 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:07:11.763 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.38:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:07:11.763 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:07:11.763 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:07:11.764 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:07:11.764 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:07:11.764 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.38:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:07:11.764 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:07:11.764 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:07:11.764 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.38:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:07:11.764 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:07:15.764 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:07:15.766 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:07:15.766 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:07:15.766 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:07:15.766 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.254:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:07:15.766 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:07:15.766 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:07:15.767 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:07:15.767 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:07:15.767 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.254:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:07:15.767 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:07:15.767 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:07:15.767 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.254:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:07:15.767 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:07:19.769 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:07:19.770 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:07:19.770 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:07:19.771 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:07:19.771 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.48:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:07:19.771 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:07:19.771 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:07:19.771 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:07:19.771 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:07:19.771 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.48:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:07:19.772 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:07:19.772 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:07:19.772 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.48:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:07:19.772 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:07:23.774 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:07:23.775 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:07:23.775 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:07:23.775 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:07:23.775 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.38:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:07:23.776 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:07:23.776 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:07:23.776 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:07:23.776 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:07:23.776 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.38:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:07:23.776 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:07:23.776 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:07:23.776 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.38:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:07:23.776 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:07:27.776 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:07:27.778 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:07:27.778 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:07:27.778 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:07:27.779 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.254:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:07:27.779 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:07:27.779 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:07:27.779 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:07:27.779 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:07:27.780 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.254:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:07:27.780 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:07:27.780 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:07:27.780 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.254:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:07:27.780 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:07:28.010 7f42cc817df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/22/2025 05:07:28.239 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:07:28.239 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  database_usage = [ <<  disk_space_used_1k = 6817408  disk_space_available_1k = 74273076  config_db_size_1k = 1692  >>,  ]  >>  
09/22/2025 05:07:28.239 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:07:29.551 7f42cc817df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/22/2025 05:07:31.779 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:07:31.780 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/22/2025 05:07:31.781 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:07:31.781 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:07:31.781 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.48:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:07:31.781 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:07:31.781 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/22/2025 05:07:31.782 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/22/2025 05:07:31.783 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:07:31.784 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.48:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/22/2025 05:07:31.784 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:07:31.785 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:07:31.786 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758517467635457  collector_name =   collector_ip = 10.0.0.48:8086  collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/22/2025 05:07:31.786 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:07:34.680 7f42cc817df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4163-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6817436, partition_space_available_1k=74273048, percentage_partition_space_used=8, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1719708, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=86156, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51980, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=4432828, free=5127276, buffers=243284, cached=6372252, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.43, five_min_avg=1.25, fifteen_min_avg=0.57, cpu_share=7.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/22/2025 05:07:34.680 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:07:34.681 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  disk_usage_info = { /dev/vda1 : <<  partition_type = ext4  partition_space_used_1k = 6817436  partition_space_available_1k = 74273048  percentage_partition_space_used = 8  >> }  process_mem_cpu_usage = { cassandra : <<  mem_virt = 1719708  cpu_share = 0  mem_res = 0  >>zookeeper : <<  mem_virt = 86156  cpu_share = 0  mem_res = 0  >>contrail-config-database-nodemgr : <<  mem_virt = 51980  cpu_share = 0  mem_res = 0  >> }  system_mem_usage = <<  total = 16175640  used = 4432828  free = 5127276  buffers = 243284  cached = 6372252  node_type = config-database-node  >>  system_cpu_usage = <<  one_min_avg = 1.43  five_min_avg = 1.25  fifteen_min_avg = 0.57  cpu_share = 7.25  node_type = config-database-node  >>  >>  
09/22/2025 05:07:34.681 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/22/2025 05:07:35.780 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/22/2025 05:07:35.782 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected
09/22/2025 05:07:35.782 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/22/2025 05:07:35.782 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:07:35.782 7f42c6f0b248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4163-2.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.38:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/22/2025 05:07:35.782 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/22/2025 05:07:35.783 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect]
09/22/2025 05:07:35.783 7f42c6f0b248 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6
09/22/2025 05:07:35.783 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit]
09/22/2025 05:07:35.783 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
09/22/2025 05:07:35.783 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
09/22/2025 05:07:42.565 7f42c6e1c048 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001016
09/22/2025 05:07:42.565 7f42c6e1c048 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient]
09/22/2025 05:07:42.565 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit]
09/22/2025 05:07:42.573 7f42c6f0b248 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0
09/22/2025 05:07:42.573 7f42c6f0b248 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-4163-2.: 4> in the [ObjectConfigDatabaseInfo:DatabaseUsage] map
09/22/2025 05:07:42.573 7f42c6f0b248 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-4163-2.: 105> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
09/22/2025 05:07:42.592 7f42c6f0b248 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0: 48> in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map
09/22/2025 05:07:42.592 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established]
09/22/2025 05:07:42.593 7f42c6fb9e48 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG]
09/22/2025 05:07:42.595 7f42c6fb9e48 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID]
09/22/2025 05:07:52.675 7f42c6e1c048 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001439
09/22/2025 05:07:52.676 7f42c6e1c048 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient]
09/22/2025 05:07:52.676 7f42c6fb9e48 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established]
09/22/2025 05:07:52.677 7f42c6fb9e48 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 3
09/22/2025 05:07:52.679 7f42c6fb9e48 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-4163-2.: 4> in the [ObjectConfigDatabaseInfo:DatabaseUsage] map
09/22/2025 05:07:52.679 7f42c6fb9e48 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-4163-2.: 106> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
09/22/2025 05:07:52.680 7f42c6fb9e48 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-4163-2.:ConfigDatabase:contrail-config-database-nodemgr:0: 49> in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map
09/22/2025 05:07:52.680 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG]
09/22/2025 05:07:52.681 7f42c6f0b248 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID]
09/22/2025 05:08:28.059 7f42cc817df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/22/2025 05:08:30.093 7f42cc817df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/22/2025 05:08:35.691 7f42cc817df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4163-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7263984, partition_space_available_1k=73826500, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1714132, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=159936, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52324, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=5497716, free=3553764, buffers=263620, cached=6860540, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.83, five_min_avg=1.71, fifteen_min_avg=0.78, cpu_share=12.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/22/2025 05:09:28.039 7f42cc817df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/22/2025 05:09:29.943 7f42cc817df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/22/2025 05:09:35.693 7f42cc817df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4163-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7337108, partition_space_available_1k=73753376, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1713264, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=138476, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52336, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=5474432, free=3503964, buffers=264676, cached=6932568, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.51, five_min_avg=2.13, fifteen_min_avg=0.98, cpu_share=12.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/22/2025 05:09:59.459 7f42c6fb9e48 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-4163-2.: 108> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
09/22/2025 05:10:28.071 7f42cc817df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/22/2025 05:10:30.501 7f42cc817df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/22/2025 05:10:30.781 7f42c6f0b248 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-4163-2.: 108> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
09/22/2025 05:10:35.717 7f42cc817df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4163-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7801632, partition_space_available_1k=73288852, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1718460, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=123912, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52428, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=5500496, free=2996760, buffers=270948, cached=7407436, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.43, five_min_avg=2.37, fifteen_min_avg=1.14, cpu_share=13.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/22/2025 05:11:28.067 7f42cc817df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/22/2025 05:11:29.924 7f42cc817df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/22/2025 05:11:35.728 7f42cc817df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4163-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7801344, partition_space_available_1k=73289140, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1720032, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=129984, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52484, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=5520952, free=2972124, buffers=272092, cached=7410472, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.52, five_min_avg=2.56, fifteen_min_avg=1.28, cpu_share=11.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/22/2025 05:11:52.998 7f42c6fb9e48 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-4163-2.: 110> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map