09/25/2025 01:40:21.003 7f329ef71c80 [contrail-config-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0]
09/25/2025 01:40:21.003 7f329ef71c80 [contrail-config-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False]
09/25/2025 01:40:21.003 7f329ef71c80 [contrail-config-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False]
09/25/2025 01:40:21.011 7f329c1724a0 [contrail-config-database-nodemgr] [ERROR]: Cannot write http_port 8112 to /tmp/contrail-config-database-nodemgr.0.http_port
09/25/2025 01:40:21.012 7f329c1724a0 [contrail-config-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8112
09/25/2025 01:40:21.012 7f329c172860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvStart] in state[none]
09/25/2025 01:40:21.013 7f329c172860 [contrail-config-database-nodemgr] [DEBUG]: Add uve <cn-jenkins-deploy-platform-ansible-os-4185-1., 1> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
09/25/2025 01:40:21.013 7f329c172860 [contrail-config-database-nodemgr] [DEBUG]: Add uve <cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0, 1> in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map
09/25/2025 01:40:21.013 7f329c172860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle]
09/25/2025 01:40:21.013 7f329c172860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:21.013 7f329c172860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>,  ]  description = Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
09/25/2025 01:40:21.013 7f329c172860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:21.014 7f329c172860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:21.014 7f329c172860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758764421013481  collector_name =   collector_ip =   collector_list = [ 10.0.0.249:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 1  >>  
09/25/2025 01:40:21.014 7f329c172860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:21.124 7f329ef71c80 [contrail-config-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 6401515666385044987004183242965650095644827286469960591684306282786983109886, 'start': '1758764419000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'zookeeper', 'group': 'zookeeper', 'pid': 45190143558734391972194315978539225437866203450605416820491641723567643831578, 'start': '1758764419000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-config-database-nodemgr', 'group': 'contrail-config-database-nodemgr', 'pid': 96240552440908033697941444180916643747770971983049279886510663438836002213362, 'start': '1758764419000000', 'statename': 'PROCESS_STATE_RUNNING'}]'
09/25/2025 01:40:21.188 7f329c33d0e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:21.189 7f329c33d0e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  build_info = {"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-25 00:15:10.514512", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11279-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed
"}]}  system_cpu_info = <<  num_socket = 1  num_cpu = 8  num_core_per_socket = 8  num_thread_per_core = 1  >>  running_package_version = package contrail-lib is not installed
  installed_package_version = package contrail-lib is not installed
  >>  
09/25/2025 01:40:21.189 7f329c33d0e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:21.189 7f329ef71c80 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-4185-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1758764421189356, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4185-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
09/25/2025 01:40:21.189 7f329ef71c80 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-4185-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1758764421189648, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4185-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1758764419000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1758764419000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1758764419000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-25 00:15:10.514512", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11279-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed\n"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
09/25/2025 01:40:21.194 7f329c33d0e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:21.195 7f329c33d0e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>,  ]  description = Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
09/25/2025 01:40:21.195 7f329c33d0e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:21.195 7f329c33d0e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:21.195 7f329c33d0e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  deleted = False  process_info = [ <<  process_name = cassandra  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1758764419000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>, <<  process_name = zookeeper  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1758764419000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>, <<  process_name = contrail-config-database-nodemgr  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1758764419000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>,  ]  build_info = {"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-25 00:15:10.514512", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11279-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed
"}]}  >>  
09/25/2025 01:40:21.195 7f329c33d0e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:21.435 7f329ef71c80 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/25/2025 01:40:21.659 7f329ef71c80 [contrail-config-database-nodemgr] [DEBUG]: Add uve <cn-jenkins-deploy-platform-ansible-os-4185-1., 1> in the [ObjectConfigDatabaseInfo:DatabaseUsage] map
09/25/2025 01:40:21.660 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:21.661 7f329c320a40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  database_usage = [ <<  disk_space_used_1k = 13650768  disk_space_available_1k = 67439716  config_db_size_1k = 4  >>,  ]  >>  
09/25/2025 01:40:21.661 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:21.772 7f329ef71c80 [contrail-config-database-nodemgr] [ERROR]: Failed to connect to database by CQL and socket: Result: 1
09/25/2025 01:40:21.772 7f329ef71c80 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-4185-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1758764421772707, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4185-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
09/25/2025 01:40:21.774 7f329c320e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:21.774 7f329c320e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
09/25/2025 01:40:21.775 7f329c320e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:22.513 7f329ef71c80 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/25/2025 01:40:23.526 7f329ef71c80 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr config_db_uuid'. Error is: Result: 1
09/25/2025 01:40:24.434 7f329ef71c80 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr to_bgp_keyspace'. Error is: Result: 1
09/25/2025 01:40:25.014 7f329c320680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/25/2025 01:40:25.015 7f329c320680 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/25/2025 01:40:25.016 7f329c320680 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/25/2025 01:40:25.016 7f329c320680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:40:25.016 7f329c320680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/25/2025 01:40:25.016 7f329c320680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:40:25.016 7f329c320680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/25/2025 01:40:25.016 7f329c320680 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/25/2025 01:40:25.016 7f329c320680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:25.016 7f329c320680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/25/2025 01:40:25.017 7f329c320680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:25.017 7f329c320680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:25.017 7f329c320680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758764421013481  collector_name =   collector_ip = 10.0.0.249:8086  collector_list = [ 10.0.0.249:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/25/2025 01:40:25.017 7f329c320680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:25.116 7f329ef71c80 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr svc_monitor_keyspace'. Error is: Result: 1
09/25/2025 01:40:25.819 7f329ef71c80 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr DISCOVERY_SERVER'. Error is: Result: 1
09/25/2025 01:40:26.570 7f329ef71c80 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr useragent'. Error is: Result: 1
09/25/2025 01:40:27.235 7f329ef71c80 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr dm_keyspace'. Error is: Result: 1
09/25/2025 01:40:29.016 7f329c33d900 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/25/2025 01:40:29.018 7f329c33d900 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/25/2025 01:40:29.018 7f329c33d900 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/25/2025 01:40:29.018 7f329c33d900 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:40:29.018 7f329c33d900 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/25/2025 01:40:29.018 7f329c33d900 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:40:29.018 7f329c33d900 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/25/2025 01:40:29.019 7f329c33d900 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/25/2025 01:40:29.019 7f329c33d900 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:29.019 7f329c33d900 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/25/2025 01:40:29.019 7f329c33d900 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:29.019 7f329c33d900 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:29.019 7f329c33d900 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758764421013481  collector_name =   collector_ip = 10.0.0.249:8086  collector_list = [ 10.0.0.249:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/25/2025 01:40:29.019 7f329c33d900 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:32.264 7f329ef71c80 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-4185-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1758764432264457, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4185-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1758764419000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1758764419000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1758764419000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-25 00:15:10.514512", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11279-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed\n"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
09/25/2025 01:40:32.264 7f329ef71c80 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4185-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13823904, partition_space_available_1k=67266580, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1281332, cpu_share=0.0, mem_res=1270376, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53964, cpu_share=0.0, mem_res=51676, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52648, cpu_share=0.0, mem_res=42668, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=5299520, free=13370236, buffers=601316, cached=13936840, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.91, five_min_avg=1.53, fifteen_min_avg=1.05, cpu_share=0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/25/2025 01:40:32.265 7f329c172c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:32.265 7f329c172c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  deleted = False  process_info = [ <<  process_name = cassandra  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1758764419000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>, <<  process_name = zookeeper  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1758764419000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>, <<  process_name = contrail-config-database-nodemgr  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1758764419000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>,  ]  build_info = {"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-25 00:15:10.514512", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11279-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed
"}]}  >>  
09/25/2025 01:40:32.265 7f329c172c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:32.265 7f329c172c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:32.265 7f329c172c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  disk_usage_info = { /dev/vda1 : <<  partition_type = ext4  partition_space_used_1k = 13823904  partition_space_available_1k = 67266580  percentage_partition_space_used = 17  >> }  all_core_file_list = [  ]  process_mem_cpu_usage = { cassandra : <<  mem_virt = 1281332  cpu_share = 0.0  mem_res = 1270376  >>zookeeper : <<  mem_virt = 53964  cpu_share = 0.0  mem_res = 51676  >>contrail-config-database-nodemgr : <<  mem_virt = 52648  cpu_share = 0.0  mem_res = 42668  >> }  system_mem_usage = <<  total = 32665116  used = 5299520  free = 13370236  buffers = 601316  cached = 13936840  node_type = config-database-node  >>  system_cpu_usage = <<  one_min_avg = 1.91  five_min_avg = 1.53  fifteen_min_avg = 1.05  cpu_share = 0  node_type = config-database-node  >>  >>  
09/25/2025 01:40:32.265 7f329c172c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:33.020 7f329c172c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/25/2025 01:40:33.021 7f329c172c20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/25/2025 01:40:33.021 7f329c172c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/25/2025 01:40:33.021 7f329c172c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:40:33.021 7f329c172c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/25/2025 01:40:33.022 7f329c172c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:40:33.022 7f329c172c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/25/2025 01:40:33.022 7f329c172c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/25/2025 01:40:33.022 7f329c172c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:33.022 7f329c172c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/25/2025 01:40:33.023 7f329c172c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:33.023 7f329c172c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:33.023 7f329c172c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758764421013481  collector_name =   collector_ip = 10.0.0.249:8086  collector_list = [ 10.0.0.249:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/25/2025 01:40:33.023 7f329c172c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:37.022 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/25/2025 01:40:37.023 7f329c320860 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/25/2025 01:40:37.023 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/25/2025 01:40:37.023 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:40:37.024 7f329c320860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/25/2025 01:40:37.024 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:40:37.024 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/25/2025 01:40:37.024 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/25/2025 01:40:37.024 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:37.024 7f329c320860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/25/2025 01:40:37.025 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:37.025 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:37.025 7f329c320860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758764421013481  collector_name =   collector_ip = 10.0.0.249:8086  collector_list = [ 10.0.0.249:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/25/2025 01:40:37.025 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:41.028 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/25/2025 01:40:41.029 7f329c320860 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/25/2025 01:40:41.029 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/25/2025 01:40:41.029 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:40:41.030 7f329c320860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/25/2025 01:40:41.030 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:40:41.030 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/25/2025 01:40:41.030 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/25/2025 01:40:41.030 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:41.030 7f329c320860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/25/2025 01:40:41.030 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:41.030 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:41.031 7f329c320860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758764421013481  collector_name =   collector_ip = 10.0.0.249:8086  collector_list = [ 10.0.0.249:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/25/2025 01:40:41.031 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:45.034 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/25/2025 01:40:45.035 7f329c320860 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/25/2025 01:40:45.035 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/25/2025 01:40:45.035 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:40:45.035 7f329c320860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/25/2025 01:40:45.035 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:40:45.035 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/25/2025 01:40:45.036 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/25/2025 01:40:45.036 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:45.036 7f329c320860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/25/2025 01:40:45.036 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:45.036 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:45.036 7f329c320860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758764421013481  collector_name =   collector_ip = 10.0.0.249:8086  collector_list = [ 10.0.0.249:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/25/2025 01:40:45.036 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:49.038 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/25/2025 01:40:49.039 7f329c320860 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/25/2025 01:40:49.039 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/25/2025 01:40:49.040 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:40:49.040 7f329c320860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/25/2025 01:40:49.040 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:40:49.040 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/25/2025 01:40:49.040 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/25/2025 01:40:49.040 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:49.041 7f329c320860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/25/2025 01:40:49.041 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:49.041 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:49.041 7f329c320860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758764421013481  collector_name =   collector_ip = 10.0.0.249:8086  collector_list = [ 10.0.0.249:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/25/2025 01:40:49.041 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:53.041 7f329c320d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/25/2025 01:40:53.043 7f329c320d60 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/25/2025 01:40:53.043 7f329c320d60 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/25/2025 01:40:53.043 7f329c320d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:40:53.043 7f329c320d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/25/2025 01:40:53.043 7f329c320d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:40:53.044 7f329c320d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/25/2025 01:40:53.044 7f329c320d60 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/25/2025 01:40:53.044 7f329c320d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:53.044 7f329c320d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/25/2025 01:40:53.045 7f329c320d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:53.045 7f329c320d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:53.045 7f329c320d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758764421013481  collector_name =   collector_ip = 10.0.0.249:8086  collector_list = [ 10.0.0.249:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/25/2025 01:40:53.045 7f329c320d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:57.046 7f329c320d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/25/2025 01:40:57.047 7f329c320d60 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/25/2025 01:40:57.047 7f329c320d60 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/25/2025 01:40:57.047 7f329c320d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:40:57.047 7f329c320d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/25/2025 01:40:57.047 7f329c320d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:40:57.047 7f329c320d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/25/2025 01:40:57.048 7f329c320d60 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/25/2025 01:40:57.048 7f329c320d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:57.048 7f329c320d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/25/2025 01:40:57.048 7f329c320d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:57.048 7f329c320d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:40:57.048 7f329c320d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758764421013481  collector_name =   collector_ip = 10.0.0.249:8086  collector_list = [ 10.0.0.249:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/25/2025 01:40:57.048 7f329c320d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:01.051 7f329c320d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/25/2025 01:41:01.052 7f329c320d60 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/25/2025 01:41:01.052 7f329c320d60 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/25/2025 01:41:01.052 7f329c320d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:41:01.052 7f329c320d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/25/2025 01:41:01.052 7f329c320d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:41:01.052 7f329c320d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/25/2025 01:41:01.053 7f329c320d60 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/25/2025 01:41:01.053 7f329c320d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:01.053 7f329c320d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/25/2025 01:41:01.053 7f329c320d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:01.053 7f329c320d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:01.053 7f329c320d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758764421013481  collector_name =   collector_ip = 10.0.0.249:8086  collector_list = [ 10.0.0.249:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/25/2025 01:41:01.053 7f329c320d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:05.054 7f329c320d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/25/2025 01:41:05.055 7f329c320d60 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/25/2025 01:41:05.056 7f329c320d60 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/25/2025 01:41:05.057 7f329c320d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:41:05.057 7f329c320d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/25/2025 01:41:05.057 7f329c320d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:41:05.057 7f329c320d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/25/2025 01:41:05.057 7f329c320d60 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/25/2025 01:41:05.057 7f329c320d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:05.057 7f329c320d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/25/2025 01:41:05.057 7f329c320d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:05.057 7f329c320d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:05.058 7f329c320d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758764421013481  collector_name =   collector_ip = 10.0.0.249:8086  collector_list = [ 10.0.0.249:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/25/2025 01:41:05.059 7f329c320d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:09.058 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/25/2025 01:41:09.059 7f329c320a40 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/25/2025 01:41:09.059 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/25/2025 01:41:09.059 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:41:09.060 7f329c320a40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/25/2025 01:41:09.060 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:41:09.060 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/25/2025 01:41:09.060 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/25/2025 01:41:09.060 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:09.060 7f329c320a40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/25/2025 01:41:09.061 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:09.061 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:09.061 7f329c320a40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758764421013481  collector_name =   collector_ip = 10.0.0.249:8086  collector_list = [ 10.0.0.249:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/25/2025 01:41:09.061 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:13.062 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/25/2025 01:41:13.063 7f329c320a40 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/25/2025 01:41:13.063 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/25/2025 01:41:13.063 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:41:13.063 7f329c320a40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/25/2025 01:41:13.063 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:41:13.063 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/25/2025 01:41:13.064 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/25/2025 01:41:13.064 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:13.064 7f329c320a40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/25/2025 01:41:13.064 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:13.064 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:13.064 7f329c320a40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758764421013481  collector_name =   collector_ip = 10.0.0.249:8086  collector_list = [ 10.0.0.249:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/25/2025 01:41:13.064 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:17.066 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/25/2025 01:41:17.067 7f329c320a40 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/25/2025 01:41:17.067 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/25/2025 01:41:17.067 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:41:17.067 7f329c320a40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/25/2025 01:41:17.067 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:41:17.067 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/25/2025 01:41:17.068 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/25/2025 01:41:17.068 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:17.068 7f329c320a40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/25/2025 01:41:17.068 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:17.068 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:17.068 7f329c320a40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758764421013481  collector_name =   collector_ip = 10.0.0.249:8086  collector_list = [ 10.0.0.249:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/25/2025 01:41:17.068 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:21.068 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/25/2025 01:41:21.070 7f329c320a40 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/25/2025 01:41:21.070 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/25/2025 01:41:21.070 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:41:21.070 7f329c320a40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/25/2025 01:41:21.070 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:41:21.070 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/25/2025 01:41:21.071 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/25/2025 01:41:21.071 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:21.071 7f329c320a40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/25/2025 01:41:21.071 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:21.071 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:21.071 7f329c320a40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758764421013481  collector_name =   collector_ip = 10.0.0.249:8086  collector_list = [ 10.0.0.249:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/25/2025 01:41:21.071 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:21.371 7f329ef71c80 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/25/2025 01:41:21.630 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:21.630 7f329c320360 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  database_usage = [ <<  disk_space_used_1k = 14768320  disk_space_available_1k = 66322164  config_db_size_1k = 956  >>,  ]  >>  
09/25/2025 01:41:21.630 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:21.754 7f329ef71c80 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open
09/25/2025 01:41:21.754 7f329ef71c80 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-4185-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1758764481754495, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4185-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.249:8086'], status='Down', description='Connect to Idle on EvTcpConnectFail')]), flag_infos=None, description='Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0)
09/25/2025 01:41:21.755 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:21.755 7f329c320a40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/25/2025 01:41:21.756 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:22.451 7f329ef71c80 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/25/2025 01:41:25.072 7f329c33d540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/25/2025 01:41:25.073 7f329c33d540 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/25/2025 01:41:25.073 7f329c33d540 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/25/2025 01:41:25.073 7f329c33d540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:41:25.074 7f329c33d540 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/25/2025 01:41:25.074 7f329c33d540 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:41:25.074 7f329c33d540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/25/2025 01:41:25.074 7f329c33d540 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/25/2025 01:41:25.074 7f329c33d540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:25.074 7f329c33d540 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/25/2025 01:41:25.074 7f329c33d540 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:25.074 7f329c33d540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:25.075 7f329c33d540 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758764421013481  collector_name =   collector_ip = 10.0.0.249:8086  collector_list = [ 10.0.0.249:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/25/2025 01:41:25.075 7f329c33d540 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:27.481 7f329ef71c80 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4185-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14768580, partition_space_available_1k=66321904, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1348624, cpu_share=0.29, mem_res=1335152, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=66644, cpu_share=0.01, mem_res=63544, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52748, cpu_share=0.01, mem_res=42668, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=6312780, free=11611576, buffers=641976, cached=14641524, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.02, five_min_avg=1.61, fifteen_min_avg=1.1, cpu_share=3.38, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/25/2025 01:41:27.481 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:27.481 7f329c320a40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  disk_usage_info = { /dev/vda1 : <<  partition_type = ext4  partition_space_used_1k = 14768580  partition_space_available_1k = 66321904  percentage_partition_space_used = 18  >> }  process_mem_cpu_usage = { cassandra : <<  mem_virt = 1348624  cpu_share = 0.29  mem_res = 1335152  >>zookeeper : <<  mem_virt = 66644  cpu_share = 0.01  mem_res = 63544  >>contrail-config-database-nodemgr : <<  mem_virt = 52748  cpu_share = 0.01  mem_res = 42668  >> }  system_mem_usage = <<  total = 32665116  used = 6312780  free = 11611576  buffers = 641976  cached = 14641524  node_type = config-database-node  >>  system_cpu_usage = <<  one_min_avg = 2.02  five_min_avg = 1.61  fifteen_min_avg = 1.1  cpu_share = 3.38  node_type = config-database-node  >>  >>  
09/25/2025 01:41:27.481 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:29.075 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/25/2025 01:41:29.076 7f329c320a40 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/25/2025 01:41:29.076 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/25/2025 01:41:29.076 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:41:29.077 7f329c320a40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/25/2025 01:41:29.077 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:41:29.077 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/25/2025 01:41:29.077 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/25/2025 01:41:29.077 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:29.077 7f329c320a40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/25/2025 01:41:29.077 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:29.077 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:29.077 7f329c320a40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758764421013481  collector_name =   collector_ip = 10.0.0.249:8086  collector_list = [ 10.0.0.249:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/25/2025 01:41:29.078 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:33.077 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/25/2025 01:41:33.078 7f329c320a40 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/25/2025 01:41:33.078 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/25/2025 01:41:33.079 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:41:33.079 7f329c320a40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/25/2025 01:41:33.079 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:41:33.079 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/25/2025 01:41:33.079 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/25/2025 01:41:33.079 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:33.079 7f329c320a40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/25/2025 01:41:33.079 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:33.079 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:33.079 7f329c320a40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758764421013481  collector_name =   collector_ip = 10.0.0.249:8086  collector_list = [ 10.0.0.249:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/25/2025 01:41:33.080 7f329c320a40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:37.079 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/25/2025 01:41:37.080 7f329c320360 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/25/2025 01:41:37.080 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/25/2025 01:41:37.080 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:41:37.080 7f329c320360 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/25/2025 01:41:37.080 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:41:37.080 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/25/2025 01:41:37.080 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/25/2025 01:41:37.081 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:37.081 7f329c320360 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/25/2025 01:41:37.081 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:37.081 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:37.081 7f329c320360 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758764421013481  collector_name =   collector_ip = 10.0.0.249:8086  collector_list = [ 10.0.0.249:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/25/2025 01:41:37.081 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:41.084 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/25/2025 01:41:41.085 7f329c320360 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/25/2025 01:41:41.085 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/25/2025 01:41:41.085 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:41:41.085 7f329c320360 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/25/2025 01:41:41.085 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:41:41.085 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/25/2025 01:41:41.086 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/25/2025 01:41:41.086 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:41.086 7f329c320360 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/25/2025 01:41:41.086 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:41.086 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:41.086 7f329c320360 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758764421013481  collector_name =   collector_ip = 10.0.0.249:8086  collector_list = [ 10.0.0.249:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/25/2025 01:41:41.086 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:45.086 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/25/2025 01:41:45.087 7f329c320360 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/25/2025 01:41:45.087 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/25/2025 01:41:45.087 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:41:45.087 7f329c320360 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/25/2025 01:41:45.087 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:41:45.087 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/25/2025 01:41:45.088 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/25/2025 01:41:45.088 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:45.088 7f329c320360 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/25/2025 01:41:45.088 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:45.088 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:45.088 7f329c320360 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758764421013481  collector_name =   collector_ip = 10.0.0.249:8086  collector_list = [ 10.0.0.249:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/25/2025 01:41:45.088 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:49.090 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/25/2025 01:41:49.091 7f329c320360 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/25/2025 01:41:49.091 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/25/2025 01:41:49.091 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:41:49.092 7f329c320360 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/25/2025 01:41:49.092 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:41:49.092 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/25/2025 01:41:49.092 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/25/2025 01:41:49.092 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:49.092 7f329c320360 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/25/2025 01:41:49.092 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:49.092 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:49.092 7f329c320360 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758764421013481  collector_name =   collector_ip = 10.0.0.249:8086  collector_list = [ 10.0.0.249:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/25/2025 01:41:49.093 7f329c320360 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:53.093 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/25/2025 01:41:53.094 7f329c320860 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/25/2025 01:41:53.094 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/25/2025 01:41:53.094 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:41:53.094 7f329c320860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/25/2025 01:41:53.094 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:41:53.094 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/25/2025 01:41:53.095 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/25/2025 01:41:53.095 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:53.095 7f329c320860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/25/2025 01:41:53.095 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:53.095 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:53.095 7f329c320860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758764421013481  collector_name =   collector_ip = 10.0.0.249:8086  collector_list = [ 10.0.0.249:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/25/2025 01:41:53.095 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:57.098 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/25/2025 01:41:57.099 7f329c320860 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/25/2025 01:41:57.099 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/25/2025 01:41:57.099 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:41:57.099 7f329c320860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/25/2025 01:41:57.099 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:41:57.100 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/25/2025 01:41:57.100 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/25/2025 01:41:57.100 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:57.100 7f329c320860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/25/2025 01:41:57.100 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:57.100 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:41:57.100 7f329c320860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758764421013481  collector_name =   collector_ip = 10.0.0.249:8086  collector_list = [ 10.0.0.249:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/25/2025 01:41:57.100 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:01.101 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/25/2025 01:42:01.102 7f329c320860 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/25/2025 01:42:01.102 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/25/2025 01:42:01.102 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:42:01.103 7f329c320860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/25/2025 01:42:01.103 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:42:01.103 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/25/2025 01:42:01.103 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/25/2025 01:42:01.103 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:01.103 7f329c320860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/25/2025 01:42:01.103 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:01.103 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:01.104 7f329c320860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758764421013481  collector_name =   collector_ip = 10.0.0.249:8086  collector_list = [ 10.0.0.249:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/25/2025 01:42:01.104 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:05.106 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/25/2025 01:42:05.107 7f329c320860 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/25/2025 01:42:05.107 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/25/2025 01:42:05.107 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:42:05.107 7f329c320860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/25/2025 01:42:05.107 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:42:05.107 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/25/2025 01:42:05.108 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/25/2025 01:42:05.108 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:05.108 7f329c320860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/25/2025 01:42:05.108 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:05.108 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:05.108 7f329c320860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758764421013481  collector_name =   collector_ip = 10.0.0.249:8086  collector_list = [ 10.0.0.249:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/25/2025 01:42:05.108 7f329c320860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:09.109 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/25/2025 01:42:09.110 7f329c172b80 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/25/2025 01:42:09.110 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/25/2025 01:42:09.111 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:42:09.111 7f329c172b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/25/2025 01:42:09.111 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:42:09.111 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/25/2025 01:42:09.111 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/25/2025 01:42:09.111 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:09.111 7f329c172b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/25/2025 01:42:09.111 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:09.111 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:09.111 7f329c172b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758764421013481  collector_name =   collector_ip = 10.0.0.249:8086  collector_list = [ 10.0.0.249:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/25/2025 01:42:09.112 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:13.115 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/25/2025 01:42:13.116 7f329c172b80 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/25/2025 01:42:13.116 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/25/2025 01:42:13.116 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:42:13.116 7f329c172b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/25/2025 01:42:13.116 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:42:13.116 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/25/2025 01:42:13.117 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/25/2025 01:42:13.117 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:13.117 7f329c172b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/25/2025 01:42:13.117 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:13.117 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:13.117 7f329c172b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758764421013481  collector_name =   collector_ip = 10.0.0.249:8086  collector_list = [ 10.0.0.249:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/25/2025 01:42:13.117 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:17.120 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/25/2025 01:42:17.121 7f329c172b80 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/25/2025 01:42:17.121 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/25/2025 01:42:17.121 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:42:17.121 7f329c172b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/25/2025 01:42:17.121 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:42:17.121 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/25/2025 01:42:17.121 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/25/2025 01:42:17.121 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:17.122 7f329c172b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/25/2025 01:42:17.122 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:17.122 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:17.122 7f329c172b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758764421013481  collector_name =   collector_ip = 10.0.0.249:8086  collector_list = [ 10.0.0.249:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/25/2025 01:42:17.122 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:21.122 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/25/2025 01:42:21.123 7f329c172b80 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/25/2025 01:42:21.123 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/25/2025 01:42:21.123 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:42:21.123 7f329c172b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/25/2025 01:42:21.123 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:42:21.123 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/25/2025 01:42:21.124 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/25/2025 01:42:21.124 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:21.124 7f329c172b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/25/2025 01:42:21.124 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:21.124 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:21.124 7f329c172b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758764421013481  collector_name =   collector_ip = 10.0.0.249:8086  collector_list = [ 10.0.0.249:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/25/2025 01:42:21.124 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:21.365 7f329ef71c80 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/25/2025 01:42:21.597 7f329c172d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:21.597 7f329c172d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  database_usage = [ <<  disk_space_used_1k = 16476676  disk_space_available_1k = 64613808  config_db_size_1k = 956  >>,  ]  >>  
09/25/2025 01:42:21.597 7f329c172d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:21.715 7f329ef71c80 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open
09/25/2025 01:42:22.364 7f329ef71c80 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/25/2025 01:42:25.126 7f329c33d400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/25/2025 01:42:25.129 7f329c33d400 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/25/2025 01:42:25.129 7f329c33d400 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/25/2025 01:42:25.129 7f329c33d400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:42:25.130 7f329c33d400 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/25/2025 01:42:25.130 7f329c33d400 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:42:25.130 7f329c33d400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/25/2025 01:42:25.130 7f329c33d400 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/25/2025 01:42:25.130 7f329c33d400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:25.131 7f329c33d400 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/25/2025 01:42:25.131 7f329c33d400 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:25.131 7f329c33d400 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:25.131 7f329c33d400 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758764421013481  collector_name =   collector_ip = 10.0.0.249:8086  collector_list = [ 10.0.0.249:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/25/2025 01:42:25.131 7f329c33d400 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:27.394 7f329ef71c80 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4185-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=16476684, partition_space_available_1k=64613800, percentage_partition_space_used=20, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1348188, cpu_share=0.04, mem_res=1334920, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=70492, cpu_share=0.0, mem_res=67384, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52848, cpu_share=0.01, mem_res=42668, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=7791720, free=8267100, buffers=648392, cached=16427524, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.34, five_min_avg=1.74, fifteen_min_avg=1.18, cpu_share=2.62, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/25/2025 01:42:27.395 7f329c320540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:27.395 7f329c320540 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  disk_usage_info = { /dev/vda1 : <<  partition_type = ext4  partition_space_used_1k = 16476684  partition_space_available_1k = 64613800  percentage_partition_space_used = 20  >> }  process_mem_cpu_usage = { cassandra : <<  mem_virt = 1348188  cpu_share = 0.04  mem_res = 1334920  >>zookeeper : <<  mem_virt = 70492  cpu_share = 0.0  mem_res = 67384  >>contrail-config-database-nodemgr : <<  mem_virt = 52848  cpu_share = 0.01  mem_res = 42668  >> }  system_mem_usage = <<  total = 32665116  used = 7791720  free = 8267100  buffers = 648392  cached = 16427524  node_type = config-database-node  >>  system_cpu_usage = <<  one_min_avg = 2.34  five_min_avg = 1.74  fifteen_min_avg = 1.18  cpu_share = 2.62  node_type = config-database-node  >>  >>  
09/25/2025 01:42:27.395 7f329c320540 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:29.132 7f329c320540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/25/2025 01:42:29.133 7f329c320540 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/25/2025 01:42:29.133 7f329c320540 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/25/2025 01:42:29.133 7f329c320540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:42:29.133 7f329c320540 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/25/2025 01:42:29.134 7f329c320540 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:42:29.134 7f329c320540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/25/2025 01:42:29.134 7f329c320540 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/25/2025 01:42:29.134 7f329c320540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:29.134 7f329c320540 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/25/2025 01:42:29.134 7f329c320540 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:29.134 7f329c320540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:29.135 7f329c320540 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758764421013481  collector_name =   collector_ip = 10.0.0.249:8086  collector_list = [ 10.0.0.249:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/25/2025 01:42:29.135 7f329c320540 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:33.134 7f329c320540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/25/2025 01:42:33.135 7f329c320540 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/25/2025 01:42:33.136 7f329c320540 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/25/2025 01:42:33.136 7f329c320540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:42:33.136 7f329c320540 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/25/2025 01:42:33.136 7f329c320540 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:42:33.136 7f329c320540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/25/2025 01:42:33.136 7f329c320540 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/25/2025 01:42:33.137 7f329c320540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:33.137 7f329c320540 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/25/2025 01:42:33.137 7f329c320540 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:33.137 7f329c320540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:33.137 7f329c320540 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758764421013481  collector_name =   collector_ip = 10.0.0.249:8086  collector_list = [ 10.0.0.249:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/25/2025 01:42:33.137 7f329c320540 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:37.136 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/25/2025 01:42:37.137 7f329c172b80 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/25/2025 01:42:37.137 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/25/2025 01:42:37.137 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:42:37.137 7f329c172b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/25/2025 01:42:37.138 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:42:37.138 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/25/2025 01:42:37.138 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/25/2025 01:42:37.138 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:37.138 7f329c172b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/25/2025 01:42:37.138 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:37.138 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:37.138 7f329c172b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758764421013481  collector_name =   collector_ip = 10.0.0.249:8086  collector_list = [ 10.0.0.249:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/25/2025 01:42:37.139 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:41.140 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/25/2025 01:42:41.141 7f329c172b80 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/25/2025 01:42:41.141 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/25/2025 01:42:41.142 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:42:41.142 7f329c172b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/25/2025 01:42:41.142 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:42:41.142 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/25/2025 01:42:41.142 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/25/2025 01:42:41.142 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:41.142 7f329c172b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/25/2025 01:42:41.142 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:41.142 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:41.143 7f329c172b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758764421013481  collector_name =   collector_ip = 10.0.0.249:8086  collector_list = [ 10.0.0.249:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/25/2025 01:42:41.143 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:45.146 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/25/2025 01:42:45.147 7f329c172b80 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/25/2025 01:42:45.147 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/25/2025 01:42:45.147 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:42:45.147 7f329c172b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/25/2025 01:42:45.147 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:42:45.147 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/25/2025 01:42:45.148 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/25/2025 01:42:45.148 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:45.148 7f329c172b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/25/2025 01:42:45.148 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:45.148 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:45.148 7f329c172b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758764421013481  collector_name =   collector_ip = 10.0.0.249:8086  collector_list = [ 10.0.0.249:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/25/2025 01:42:45.148 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:49.151 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/25/2025 01:42:49.152 7f329c172b80 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
09/25/2025 01:42:49.152 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/25/2025 01:42:49.152 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:42:49.152 7f329c172b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/25/2025 01:42:49.152 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:42:49.152 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
09/25/2025 01:42:49.152 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
09/25/2025 01:42:49.153 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:49.153 7f329c172b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
09/25/2025 01:42:49.153 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:49.153 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:49.153 7f329c172b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1758764421013481  collector_name =   collector_ip = 10.0.0.249:8086  collector_list = [ 10.0.0.249:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/25/2025 01:42:49.153 7f329c172b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:42:53.152 7f329c3204a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/25/2025 01:42:53.153 7f329c3204a0 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected
09/25/2025 01:42:53.154 7f329c3204a0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/25/2025 01:42:53.154 7f329c3204a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:42:53.154 7f329c3204a0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/25/2025 01:42:53.154 7f329c3204a0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:42:53.154 7f329c3204a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect]
09/25/2025 01:42:53.154 7f329c3204a0 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6
09/25/2025 01:42:53.154 7f329c3204a0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit]
09/25/2025 01:42:53.155 7f329c3204a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
09/25/2025 01:42:53.155 7f329c3204a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
09/25/2025 01:43:01.378 7f329c172cc0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed
09/25/2025 01:43:01.379 7f329ce655e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit]
09/25/2025 01:43:01.379 7f329ce655e0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle]
09/25/2025 01:43:01.379 7f329ce655e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:43:01.379 7f329ce655e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Down  description = ClientInit to Idle on EvTcpClose  >>,  ]  description = Collector:Collector[ClientInit to Idle on EvTcpClose] connection down  >>,  ]  >>  
09/25/2025 01:43:01.379 7f329ce655e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:43:01.379 7f329ce655e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:43:01.379 7f329ce655e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 1  pid = 1  http_port = 8112  start_time = 1758764421013481  collector_name =   collector_ip = 10.0.0.249:8086  collector_list = [ 10.0.0.249:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
09/25/2025 01:43:01.379 7f329ce655e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
09/25/2025 01:43:05.381 7f329ce655e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
09/25/2025 01:43:05.383 7f329ce655e0 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected
09/25/2025 01:43:05.383 7f329ce655e0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
09/25/2025 01:43:05.383 7f329ce655e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:43:05.383 7f329ce655e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-4185-1.  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.249:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
09/25/2025 01:43:05.383 7f329ce655e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
09/25/2025 01:43:05.383 7f329ce655e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect]
09/25/2025 01:43:05.383 7f329ce655e0 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6
09/25/2025 01:43:05.383 7f329ce655e0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit]
09/25/2025 01:43:05.384 7f329ce655e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
09/25/2025 01:43:05.384 7f329ce655e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
09/25/2025 01:43:05.386 7f329c3204a0 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001017
09/25/2025 01:43:05.387 7f329c3204a0 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient]
09/25/2025 01:43:05.387 7f329ce655e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit]
09/25/2025 01:43:05.387 7f329ce655e0 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0
09/25/2025 01:43:05.387 7f329ce655e0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-4185-1.: 3> in the [ObjectConfigDatabaseInfo:DatabaseUsage] map
09/25/2025 01:43:05.387 7f329ce655e0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-4185-1.: 89> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
09/25/2025 01:43:05.387 7f329ce655e0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0: 41> in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map
09/25/2025 01:43:05.388 7f329ce655e0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established]
09/25/2025 01:43:05.388 7f329c172cc0 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG]
09/25/2025 01:43:05.389 7f329c172cc0 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID]
09/25/2025 01:43:11.380 7f329c3204a0 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001438
09/25/2025 01:43:11.380 7f329c3204a0 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient]
09/25/2025 01:43:11.381 7f329ce65040 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established]
09/25/2025 01:43:11.381 7f329ce65040 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 3
09/25/2025 01:43:11.382 7f329ce65040 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-4185-1.: 3> in the [ObjectConfigDatabaseInfo:DatabaseUsage] map
09/25/2025 01:43:11.382 7f329ce65040 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-4185-1.: 90> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
09/25/2025 01:43:11.383 7f329ce65040 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-4185-1.:ConfigDatabase:contrail-config-database-nodemgr:0: 42> in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map
09/25/2025 01:43:11.383 7f329c320f40 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG]
09/25/2025 01:43:11.384 7f329c320f40 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID]
09/25/2025 01:43:21.407 7f329ef71c80 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/25/2025 01:43:21.849 7f329ef71c80 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open
09/25/2025 01:43:22.576 7f329ef71c80 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/25/2025 01:43:28.039 7f329ef71c80 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4185-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=17732444, partition_space_available_1k=63358040, percentage_partition_space_used=22, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1348196, cpu_share=0.04, mem_res=1335036, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=110224, cpu_share=0.03, mem_res=106552, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52876, cpu_share=0.01, mem_res=42664, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=9509924, free=4746788, buffers=683384, cached=18195840, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.55, five_min_avg=2.16, fifteen_min_avg=1.35, cpu_share=4.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/25/2025 01:44:21.400 7f329ef71c80 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/25/2025 01:44:21.847 7f329ef71c80 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open
09/25/2025 01:44:22.580 7f329ef71c80 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/25/2025 01:44:28.047 7f329ef71c80 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4185-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19561252, partition_space_available_1k=61529232, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1345284, cpu_share=0.04, mem_res=1335204, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=115348, cpu_share=0.0, mem_res=111892, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44932, cpu_share=0.01, mem_res=42664, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=9483160, free=18418168, buffers=694420, cached=4539964, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.17, five_min_avg=2.3, fifteen_min_avg=1.46, cpu_share=4.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/25/2025 01:44:59.848 7f329c320360 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-4185-1.: 92> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
09/25/2025 01:45:01.742 7f329c320360 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-4185-1.: 92> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
09/25/2025 01:45:21.430 7f329ef71c80 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/25/2025 01:45:21.876 7f329ef71c80 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open
09/25/2025 01:45:22.586 7f329ef71c80 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/25/2025 01:45:28.059 7f329ef71c80 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4185-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19746116, partition_space_available_1k=61344368, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1346352, cpu_share=0.04, mem_res=1335296, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=115544, cpu_share=0.0, mem_res=112220, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=48188, cpu_share=0.02, mem_res=42664, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=10109320, free=17426828, buffers=703456, cached=4896096, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.03, five_min_avg=2.43, fifteen_min_avg=1.55, cpu_share=2.88, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/25/2025 01:46:21.470 7f329ef71c80 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/25/2025 01:46:21.988 7f329ef71c80 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open
09/25/2025 01:46:22.696 7f329ef71c80 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/25/2025 01:46:28.078 7f329ef71c80 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4185-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21033248, partition_space_available_1k=60057236, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1348056, cpu_share=0.08, mem_res=1336260, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=116464, cpu_share=0.0, mem_res=113132, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=48200, cpu_share=0.02, mem_res=42664, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=10508460, free=15681060, buffers=738976, cached=6207268, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.71, five_min_avg=2.66, fifteen_min_avg=1.69, cpu_share=4.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/25/2025 01:47:21.455 7f329ef71c80 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/25/2025 01:47:21.912 7f329ef71c80 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open
09/25/2025 01:47:22.599 7f329ef71c80 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/25/2025 01:47:28.098 7f329ef71c80 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4185-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21073868, partition_space_available_1k=60016616, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1350112, cpu_share=0.2, mem_res=1338040, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=136032, cpu_share=0.02, mem_res=132600, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=48204, cpu_share=0.01, mem_res=42664, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=11272452, free=14858240, buffers=739080, cached=6265076, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.59, five_min_avg=3.31, fifteen_min_avg=1.97, cpu_share=4.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/25/2025 01:48:21.460 7f329ef71c80 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/25/2025 01:48:21.954 7f329ef71c80 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open
09/25/2025 01:48:22.856 7f329ef71c80 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/25/2025 01:48:28.108 7f329ef71c80 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4185-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21220848, partition_space_available_1k=59869636, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1351932, cpu_share=0.1, mem_res=1339400, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=139196, cpu_share=0.0, mem_res=135728, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=48204, cpu_share=0.02, mem_res=42664, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=11170924, free=14784504, buffers=739516, cached=6439756, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=8.8, five_min_avg=4.69, fifteen_min_avg=2.53, cpu_share=5.12, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/25/2025 01:49:21.479 7f329ef71c80 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/25/2025 01:49:21.999 7f329ef71c80 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open
09/25/2025 01:49:22.720 7f329ef71c80 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/25/2025 01:49:28.108 7f329ef71c80 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4185-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21259408, partition_space_available_1k=59831076, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1351964, cpu_share=0.12, mem_res=1339288, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=142780, cpu_share=0.0, mem_res=139280, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=48212, cpu_share=0.02, mem_res=42664, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=10917968, free=15008136, buffers=739632, cached=6468916, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.49, five_min_avg=4.46, fifteen_min_avg=2.59, cpu_share=4.38, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/25/2025 01:50:21.486 7f329ef71c80 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/25/2025 01:50:21.958 7f329ef71c80 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open
09/25/2025 01:50:22.696 7f329ef71c80 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/25/2025 01:50:28.117 7f329ef71c80 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4185-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21334320, partition_space_available_1k=59756164, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1348232, cpu_share=0.1, mem_res=1335360, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=156424, cpu_share=0.01, mem_res=152844, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=48212, cpu_share=0.02, mem_res=42664, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=11877188, free=13994284, buffers=739904, cached=6523504, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.58, five_min_avg=4.67, fifteen_min_avg=2.78, cpu_share=5.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/25/2025 01:51:21.476 7f329ef71c80 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/25/2025 01:51:21.941 7f329ef71c80 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open
09/25/2025 01:51:22.648 7f329ef71c80 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/25/2025 01:51:28.128 7f329ef71c80 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4185-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21349432, partition_space_available_1k=59741052, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1349128, cpu_share=0.08, mem_res=1335544, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=158804, cpu_share=0.01, mem_res=155212, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=48224, cpu_share=0.02, mem_res=42664, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=11290612, free=14553096, buffers=739960, cached=6551092, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.26, five_min_avg=4.14, fifteen_min_avg=2.71, cpu_share=3.62, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/25/2025 01:52:21.570 7f329ef71c80 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/25/2025 01:52:22.074 7f329ef71c80 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open
09/25/2025 01:52:22.814 7f329ef71c80 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/25/2025 01:52:28.136 7f329ef71c80 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4185-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21370824, partition_space_available_1k=59719660, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1349296, cpu_share=0.05, mem_res=1335636, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=160064, cpu_share=0.0, mem_res=156456, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=48232, cpu_share=0.02, mem_res=42664, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=11640544, free=14179012, buffers=740020, cached=6574308, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.83, five_min_avg=3.84, fifteen_min_avg=2.7, cpu_share=3.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/25/2025 01:53:21.564 7f329ef71c80 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/25/2025 01:53:22.133 7f329ef71c80 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open
09/25/2025 01:53:22.907 7f329ef71c80 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/25/2025 01:53:28.197 7f329ef71c80 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4185-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21399988, partition_space_available_1k=59690496, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1358244, cpu_share=0.09, mem_res=1344400, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=162484, cpu_share=0.0, mem_res=158856, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=48236, cpu_share=0.01, mem_res=42664, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=11878600, free=13908780, buffers=740112, cached=6606616, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.72, five_min_avg=3.86, fifteen_min_avg=2.77, cpu_share=4.38, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/25/2025 01:54:21.497 7f329ef71c80 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/25/2025 01:54:21.951 7f329ef71c80 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open
09/25/2025 01:54:22.625 7f329ef71c80 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/25/2025 01:54:28.147 7f329ef71c80 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4185-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21508712, partition_space_available_1k=59581772, percentage_partition_space_used=27, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1358444, cpu_share=0.06, mem_res=1344500, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=163880, cpu_share=0.0, mem_res=160232, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=48232, cpu_share=0.01, mem_res=42664, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=11225172, free=14450404, buffers=740168, cached=6718180, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.94, five_min_avg=3.86, fifteen_min_avg=2.84, cpu_share=4.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/25/2025 01:55:21.525 7f329ef71c80 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/25/2025 01:55:22.015 7f329ef71c80 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open
09/25/2025 01:55:22.726 7f329ef71c80 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/25/2025 01:55:28.155 7f329ef71c80 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4185-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21514736, partition_space_available_1k=59575748, percentage_partition_space_used=27, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1358504, cpu_share=0.04, mem_res=1344540, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=164744, cpu_share=0.0, mem_res=161100, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=48240, cpu_share=0.02, mem_res=42664, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=11261048, free=14408340, buffers=740188, cached=6724272, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.62, five_min_avg=3.47, fifteen_min_avg=2.77, cpu_share=2.38, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')
09/25/2025 01:56:21.527 7f329ef71c80 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data
09/25/2025 01:56:22.011 7f329ef71c80 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open
09/25/2025 01:56:22.869 7f329ef71c80 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1
09/25/2025 01:56:24.624 7f329c33d7c0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-4185-1.: 103> in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map
09/25/2025 01:56:28.166 7f329ef71c80 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4185-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21571828, partition_space_available_1k=59518656, percentage_partition_space_used=27, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1359452, cpu_share=0.08, mem_res=1344808, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=166648, cpu_share=0.0, mem_res=162984, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=48240, cpu_share=0.02, mem_res=42664, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=11113696, free=14371140, buffers=743376, cached=6904436, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.43, five_min_avg=3.25, fifteen_min_avg=2.73, cpu_share=3.12, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo')