05/26/2025 02:14:33.226 7f6f370f4f48 [contrail-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0]
05/26/2025 02:14:33.227 7f6f370f4f48 [contrail-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False]
05/26/2025 02:14:33.227 7f6f370f4f48 [contrail-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False]
05/26/2025 02:14:33.235 7f6f335386d8 [contrail-database-nodemgr] [ERROR]: Cannot write http_port 8103 to /tmp/contrail-database-nodemgr.0.http_port
05/26/2025 02:14:33.235 7f6f335386d8 [contrail-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8103
05/26/2025 02:14:33.236 7f6f3353f048 [contrail-database-nodemgr] [INFO]: Processing event[EvStart] in state[none]
05/26/2025 02:14:33.237 7f6f3353f048 [contrail-database-nodemgr] [DEBUG]: Add uve <cn-jenkins-deploy-platform-ansible-os-3336-1., 1> in the [ObjectDatabaseInfo:NodeStatusUVE] map
05/26/2025 02:14:33.237 7f6f3353f048 [contrail-database-nodemgr] [DEBUG]: Add uve <cn-jenkins-deploy-platform-ansible-os-3336-1.:Database:contrail-database-nodemgr:0, 1> in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map
05/26/2025 02:14:33.237 7f6f3353f048 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle]
05/26/2025 02:14:33.237 7f6f3353f048 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:33.237 7f6f3353f048 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3336-1.  process_status = [ <<  module_id = contrail-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>,  ]  description = Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
05/26/2025 02:14:33.237 7f6f3353f048 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:33.237 7f6f3353f048 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:33.238 7f6f3353f048 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3336-1.:Database:contrail-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8103  start_time = 1748225673237377  collector_name =   collector_ip =   collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 1  >>  
05/26/2025 02:14:33.238 7f6f3353f048 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:33.388 7f6f370f4f48 [contrail-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 19327230938738727453502360673239384069786505128061970404302270551603945051513, 'start': '1748225671000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-query-engine', 'group': 'contrail-query-engine', 'pid': 59679414887324235366528498632252229007915362677914344034009341193622219398254, 'start': '1748225671000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-database-nodemgr', 'group': 'contrail-database-nodemgr', 'pid': 43647998091385185153946734498010301293874188299315284270980481880750302994143, 'start': '1748225671000000', 'statename': 'PROCESS_STATE_RUNNING'}]'
05/26/2025 02:14:33.521 7f6f3dbcda20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:33.521 7f6f3dbcda20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3336-1.  build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-05-26 00:49:41.880059", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-9215-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}  system_cpu_info = <<  num_socket = 1  num_cpu = 8  num_core_per_socket = 8  num_thread_per_core = 1  >>  running_package_version = nightly-nightly.el7  installed_package_version = nightly-nightly.el7  >>  
05/26/2025 02:14:33.521 7f6f3dbcda20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:33.522 7f6f370f4f48 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-3336-1.', _node_type='Database', _instance_id='0', _timestamp=1748225673522109, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3336-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0)
05/26/2025 02:14:33.522 7f6f370f4f48 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-3336-1.', _node_type='Database', _instance_id='0', _timestamp=1748225673522505, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3336-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1748225671000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1748225671000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1748225671000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2025-05-26 00:49:41.880059", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-9215-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0)
05/26/2025 02:14:33.524 7f6f32be7198 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:33.524 7f6f32be7198 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3336-1.  process_status = [ <<  module_id = contrail-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>,  ]  description = Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
05/26/2025 02:14:33.524 7f6f32be7198 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:33.524 7f6f32be7198 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:33.524 7f6f32be7198 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3336-1.  deleted = False  process_info = [ <<  process_name = cassandra  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1748225671000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>, <<  process_name = contrail-query-engine  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1748225671000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>, <<  process_name = contrail-database-nodemgr  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1748225671000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>,  ]  build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-05-26 00:49:41.880059", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-9215-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}  >>  
05/26/2025 02:14:33.524 7f6f32be7198 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:33.910 7f6f370f4f48 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data
05/26/2025 02:14:34.336 7f6f370f4f48 [contrail-database-nodemgr] [DEBUG]: Add uve <cn-jenkins-deploy-platform-ansible-os-3336-1., 1> in the [ObjectDatabaseInfo:DatabaseUsage] map
05/26/2025 02:14:34.337 7f6f3490c2e8 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:34.337 7f6f3490c2e8 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3336-1.  database_usage = [ <<  disk_space_used_1k = 15279056  disk_space_available_1k = 65811428  analytics_db_size_1k = 108  >>,  ]  >>  
05/26/2025 02:14:34.338 7f6f3490c2e8 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:34.778 7f6f370f4f48 [contrail-database-nodemgr] [ERROR]: Failed to connect to database by CQL: Result: 1
05/26/2025 02:14:34.778 7f6f370f4f48 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-3336-1.', _node_type='Database', _instance_id='0', _timestamp=1748225674778392, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3336-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0)
05/26/2025 02:14:34.779 7f6f336a0eb8 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:34.780 7f6f336a0eb8 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3336-1.  process_status = [ <<  module_id = contrail-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
05/26/2025 02:14:34.780 7f6f336a0eb8 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:37.238 7f6f34a77ac8 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/26/2025 02:14:37.239 7f6f34a77ac8 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/26/2025 02:14:37.239 7f6f34a77ac8 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/26/2025 02:14:37.240 7f6f34a77ac8 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/26/2025 02:14:37.240 7f6f34a77ac8 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3336-1.  process_status = [ <<  module_id = contrail-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/26/2025 02:14:37.240 7f6f34a77ac8 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/26/2025 02:14:37.240 7f6f34a77ac8 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/26/2025 02:14:37.240 7f6f34a77ac8 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/26/2025 02:14:37.240 7f6f34a77ac8 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:37.240 7f6f34a77ac8 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3336-1.  process_status = [ <<  module_id = contrail-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/26/2025 02:14:37.241 7f6f34a77ac8 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:37.241 7f6f34a77ac8 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:37.241 7f6f34a77ac8 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3336-1.:Database:contrail-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8103  start_time = 1748225673237377  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/26/2025 02:14:37.241 7f6f34a77ac8 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:37.313 7f6f370f4f48 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-3336-1.', _node_type='Database', _instance_id='0', _timestamp=1748225677313892, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-3336-1.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=1), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0)], _table='ObjectDatabaseInfo'), _seqnum=0)
05/26/2025 02:14:37.314 7f6f370f4f48 [contrail-database-nodemgr] [DEBUG]: Add uve <cn-jenkins-deploy-platform-ansible-os-3336-1., 1> in the [ObjectDatabaseInfo:CassandraStatusUVE] map
05/26/2025 02:14:37.315 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:37.315 7f6f3490c0f0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] CassandraStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3336-1.  cassandra_compaction_task = <<  pending_compaction_tasks = 1  >>  thread_pool_stats = [ <<  pool_name = ReadStage  active = 0  pending = 0  all_time_blocked = 0  >>, <<  pool_name = CompactionExecutor  active = 0  pending = 0  all_time_blocked = 0  >>, <<  pool_name = MutationStage  active = 0  pending = 0  all_time_blocked = 0  >>, <<  pool_name = MemtableFlushWriter  active = 0  pending = 0  all_time_blocked = 0  >>,  ]  >>  
05/26/2025 02:14:37.315 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:38.488 7f6f370f4f48 [contrail-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7200 repair -pr ContrailAnalyticsCql'. Error is: Result: 1
05/26/2025 02:14:41.242 7f6f3353c438 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/26/2025 02:14:41.243 7f6f3353c438 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/26/2025 02:14:41.244 7f6f3353c438 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/26/2025 02:14:41.244 7f6f3353c438 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/26/2025 02:14:41.244 7f6f3353c438 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3336-1.  process_status = [ <<  module_id = contrail-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/26/2025 02:14:41.244 7f6f3353c438 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/26/2025 02:14:41.244 7f6f3353c438 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/26/2025 02:14:41.245 7f6f3353c438 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/26/2025 02:14:41.245 7f6f3353c438 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:41.245 7f6f3353c438 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3336-1.  process_status = [ <<  module_id = contrail-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/26/2025 02:14:41.245 7f6f3353c438 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:41.246 7f6f3353c438 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:41.246 7f6f3353c438 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3336-1.:Database:contrail-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8103  start_time = 1748225673237377  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/26/2025 02:14:41.246 7f6f3353c438 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:43.513 7f6f370f4f48 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-3336-1.', _node_type='Database', _instance_id='0', _timestamp=1748225683513179, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3336-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1748225671000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1748225671000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1748225671000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2025-05-26 00:49:41.880059", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-9215-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0)
05/26/2025 02:14:43.513 7f6f370f4f48 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3336-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15506992, partition_space_available_1k=65583492, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1220520, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=8628, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51380, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=7725024, free=8506388, buffers=695424, cached=15738280, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.64, five_min_avg=2.23, fifteen_min_avg=1.38, cpu_share=0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo')
05/26/2025 02:14:43.514 7f6f32be7a20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:43.514 7f6f32be7a20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3336-1.  deleted = False  process_info = [ <<  process_name = cassandra  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1748225671000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>, <<  process_name = contrail-query-engine  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1748225671000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>, <<  process_name = contrail-database-nodemgr  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1748225671000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>,  ]  build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-05-26 00:49:41.880059", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-9215-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}  >>  
05/26/2025 02:14:43.514 7f6f32be7a20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:43.514 7f6f32be7a20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:43.514 7f6f32be7a20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3336-1.  disk_usage_info = { /dev/vda1 : <<  partition_type = ext4  partition_space_used_1k = 15506992  partition_space_available_1k = 65583492  percentage_partition_space_used = 19  >> }  all_core_file_list = [  ]  process_mem_cpu_usage = { cassandra : <<  mem_virt = 1220520  cpu_share = 0  mem_res = 0  >>contrail-query-engine : <<  mem_virt = 8628  cpu_share = 0  mem_res = 0  >>contrail-database-nodemgr : <<  mem_virt = 51380  cpu_share = 0  mem_res = 0  >> }  system_mem_usage = <<  total = 32665116  used = 7725024  free = 8506388  buffers = 695424  cached = 15738280  node_type = database-node  >>  system_cpu_usage = <<  one_min_avg = 3.64  five_min_avg = 2.23  fifteen_min_avg = 1.38  cpu_share = 0  node_type = database-node  >>  >>  
05/26/2025 02:14:43.514 7f6f32be7a20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:45.245 7f6f3355eeb8 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/26/2025 02:14:45.246 7f6f3355eeb8 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/26/2025 02:14:45.247 7f6f3355eeb8 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/26/2025 02:14:45.247 7f6f3355eeb8 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/26/2025 02:14:45.247 7f6f3355eeb8 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3336-1.  process_status = [ <<  module_id = contrail-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/26/2025 02:14:45.247 7f6f3355eeb8 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/26/2025 02:14:45.247 7f6f3355eeb8 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/26/2025 02:14:45.247 7f6f3355eeb8 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/26/2025 02:14:45.248 7f6f3355eeb8 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:45.248 7f6f3355eeb8 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3336-1.  process_status = [ <<  module_id = contrail-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/26/2025 02:14:45.248 7f6f3355eeb8 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:45.248 7f6f3355eeb8 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:45.248 7f6f3355eeb8 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3336-1.:Database:contrail-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8103  start_time = 1748225673237377  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/26/2025 02:14:45.248 7f6f3355eeb8 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:49.247 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/26/2025 02:14:49.249 7f6f3490c0f0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/26/2025 02:14:49.249 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/26/2025 02:14:49.249 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/26/2025 02:14:49.249 7f6f3490c0f0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3336-1.  process_status = [ <<  module_id = contrail-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/26/2025 02:14:49.249 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/26/2025 02:14:49.250 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/26/2025 02:14:49.250 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/26/2025 02:14:49.250 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:49.250 7f6f3490c0f0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3336-1.  process_status = [ <<  module_id = contrail-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/26/2025 02:14:49.251 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:49.251 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:49.251 7f6f3490c0f0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3336-1.:Database:contrail-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8103  start_time = 1748225673237377  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/26/2025 02:14:49.251 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:53.251 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/26/2025 02:14:53.252 7f6f3490c0f0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/26/2025 02:14:53.253 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/26/2025 02:14:53.253 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/26/2025 02:14:53.253 7f6f3490c0f0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3336-1.  process_status = [ <<  module_id = contrail-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/26/2025 02:14:53.253 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/26/2025 02:14:53.253 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/26/2025 02:14:53.253 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/26/2025 02:14:53.253 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:53.254 7f6f3490c0f0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3336-1.  process_status = [ <<  module_id = contrail-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/26/2025 02:14:53.254 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:53.254 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:53.254 7f6f3490c0f0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3336-1.:Database:contrail-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8103  start_time = 1748225673237377  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/26/2025 02:14:53.254 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:57.257 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/26/2025 02:14:57.258 7f6f3490c0f0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/26/2025 02:14:57.259 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/26/2025 02:14:57.259 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/26/2025 02:14:57.259 7f6f3490c0f0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3336-1.  process_status = [ <<  module_id = contrail-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/26/2025 02:14:57.259 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/26/2025 02:14:57.259 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/26/2025 02:14:57.259 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/26/2025 02:14:57.260 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:57.260 7f6f3490c0f0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3336-1.  process_status = [ <<  module_id = contrail-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/26/2025 02:14:57.260 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:57.260 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:14:57.260 7f6f3490c0f0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3336-1.:Database:contrail-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8103  start_time = 1748225673237377  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/26/2025 02:14:57.260 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:15:01.260 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/26/2025 02:15:01.261 7f6f3490c0f0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/26/2025 02:15:01.262 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/26/2025 02:15:01.262 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/26/2025 02:15:01.262 7f6f3490c0f0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3336-1.  process_status = [ <<  module_id = contrail-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/26/2025 02:15:01.262 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/26/2025 02:15:01.262 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/26/2025 02:15:01.263 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/26/2025 02:15:01.263 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:15:01.263 7f6f3490c0f0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3336-1.  process_status = [ <<  module_id = contrail-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/26/2025 02:15:01.263 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:15:01.263 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:15:01.263 7f6f3490c0f0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3336-1.:Database:contrail-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8103  start_time = 1748225673237377  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/26/2025 02:15:01.263 7f6f3490c0f0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:15:05.263 7f6f34978048 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/26/2025 02:15:05.264 7f6f34978048 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/26/2025 02:15:05.265 7f6f34978048 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/26/2025 02:15:05.265 7f6f34978048 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/26/2025 02:15:05.265 7f6f34978048 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3336-1.  process_status = [ <<  module_id = contrail-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/26/2025 02:15:05.265 7f6f34978048 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/26/2025 02:15:05.265 7f6f34978048 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect]
05/26/2025 02:15:05.266 7f6f34978048 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle]
05/26/2025 02:15:05.266 7f6f34978048 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:15:05.266 7f6f34978048 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3336-1.  process_status = [ <<  module_id = contrail-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/26/2025 02:15:05.266 7f6f34978048 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:15:05.267 7f6f34978048 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:15:05.267 7f6f34978048 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3336-1.:Database:contrail-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8103  start_time = 1748225673237377  collector_name =   collector_ip = 10.0.0.130:8086  collector_list = [ 10.0.0.130:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/26/2025 02:15:05.267 7f6f34978048 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/26/2025 02:15:09.269 7f6f34978048 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/26/2025 02:15:09.270 7f6f34978048 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected
05/26/2025 02:15:09.270 7f6f34978048 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/26/2025 02:15:09.270 7f6f34978048 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/26/2025 02:15:09.270 7f6f34978048 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3336-1.  process_status = [ <<  module_id = contrail-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.130:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/26/2025 02:15:09.270 7f6f34978048 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/26/2025 02:15:09.271 7f6f34978048 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect]
05/26/2025 02:15:09.272 7f6f34978048 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6
05/26/2025 02:15:09.272 7f6f34978048 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit]
05/26/2025 02:15:09.272 7f6f34978048 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
05/26/2025 02:15:09.273 7f6f34978048 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
05/26/2025 02:15:14.927 7f6f3498c048 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001016
05/26/2025 02:15:14.930 7f6f3498c048 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient]
05/26/2025 02:15:14.930 7f6f36af2d68 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit]
05/26/2025 02:15:14.931 7f6f36af2d68 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0
05/26/2025 02:15:14.931 7f6f36af2d68 [contrail-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-3336-1.: 1> in the [ObjectDatabaseInfo:DatabaseUsage] map
05/26/2025 02:15:14.931 7f6f36af2d68 [contrail-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-3336-1.: 1> in the [ObjectDatabaseInfo:CassandraStatusUVE] map
05/26/2025 02:15:14.931 7f6f36af2d68 [contrail-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-3336-1.: 25> in the [ObjectDatabaseInfo:NodeStatusUVE] map
05/26/2025 02:15:14.931 7f6f36af2d68 [contrail-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-3336-1.:Database:contrail-database-nodemgr:0: 10> in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map
05/26/2025 02:15:14.932 7f6f36af2d68 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established]
05/26/2025 02:15:14.932 7f6f34a34eb8 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG]
05/26/2025 02:15:14.934 7f6f34a34eb8 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID]
05/26/2025 02:15:25.092 7f6f3498c048 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001582
05/26/2025 02:15:25.092 7f6f3498c048 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient]
05/26/2025 02:15:25.093 7f6f32c0ec18 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established]
05/26/2025 02:15:25.093 7f6f32c0ec18 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 4
05/26/2025 02:15:25.099 7f6f32c0ec18 [contrail-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-3336-1.: 1> in the [ObjectDatabaseInfo:DatabaseUsage] map
05/26/2025 02:15:25.100 7f6f32c0ec18 [contrail-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-3336-1.: 1> in the [ObjectDatabaseInfo:CassandraStatusUVE] map
05/26/2025 02:15:25.100 7f6f32c0ec18 [contrail-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-3336-1.: 26> in the [ObjectDatabaseInfo:NodeStatusUVE] map
05/26/2025 02:15:25.101 7f6f32c0ec18 [contrail-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-3336-1.:Database:contrail-database-nodemgr:0: 11> in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map
05/26/2025 02:15:25.101 7f6f37065a20 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG]
05/26/2025 02:15:25.102 7f6f37065a20 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID]
05/26/2025 02:15:33.837 7f6f370f4f48 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data
05/26/2025 02:15:35.032 7f6f370f4f48 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-3336-1.', _node_type='Database', _instance_id='0', _timestamp=1748225735032078, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3336-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.130:8086'], status='Up', description='ClientInit to Established on EvSandeshCtrlMessageRecv')]), flag_infos=None, description='')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0)
05/26/2025 02:15:37.355 7f6f370f4f48 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-3336-1.', _node_type='Database', _instance_id='0', _timestamp=1748225737355276, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-3336-1.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=0, pending=0, all_time_blocked=2), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0)], _table='ObjectDatabaseInfo'), _seqnum=0)
05/26/2025 02:15:42.377 7f6f370f4f48 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3336-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=16665368, partition_space_available_1k=64425116, percentage_partition_space_used=21, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1323404, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10484, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51824, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=8995928, free=5791808, buffers=722256, cached=17155124, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.5, five_min_avg=2.41, fifteen_min_avg=1.5, cpu_share=3.88, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo')
05/26/2025 02:16:34.049 7f6f370f4f48 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data
05/26/2025 02:16:42.502 7f6f370f4f48 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3336-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=17543364, partition_space_available_1k=63547120, percentage_partition_space_used=22, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1311104, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10184, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42612, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=8956648, free=20379868, buffers=738924, cached=2589676, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.53, five_min_avg=2.65, fifteen_min_avg=1.64, cpu_share=4.12, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo')
05/26/2025 02:17:10.930 7f6f33535198 [contrail-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-3336-1.: 29> in the [ObjectDatabaseInfo:NodeStatusUVE] map
05/26/2025 02:17:13.216 7f6f33535198 [contrail-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-3336-1.: 29> in the [ObjectDatabaseInfo:NodeStatusUVE] map
05/26/2025 02:17:33.901 7f6f370f4f48 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data
05/26/2025 02:17:42.169 7f6f370f4f48 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3336-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=18891292, partition_space_available_1k=62199192, percentage_partition_space_used=23, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1307884, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10516, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42608, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=9549612, free=18002476, buffers=772708, cached=4340320, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.6, five_min_avg=3.19, fifteen_min_avg=1.9, cpu_share=3.88, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo')
05/26/2025 02:18:07.906 7f6f32be6eb8 [contrail-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-3336-1.: 30> in the [ObjectDatabaseInfo:NodeStatusUVE] map
05/26/2025 02:18:33.884 7f6f370f4f48 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data
05/26/2025 02:18:37.292 7f6f370f4f48 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-3336-1.', _node_type='Database', _instance_id='0', _timestamp=1748225917292053, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-3336-1.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=0, pending=0, all_time_blocked=77), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0)], _table='ObjectDatabaseInfo'), _seqnum=0)
05/26/2025 02:18:42.318 7f6f370f4f48 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3336-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19107148, partition_space_available_1k=61983336, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1360356, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=15416, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42612, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=10079544, free=17475524, buffers=778040, cached=4332008, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.29, five_min_avg=3.58, fifteen_min_avg=2.11, cpu_share=4.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo')
05/26/2025 02:19:33.894 7f6f370f4f48 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data
05/26/2025 02:19:42.210 7f6f370f4f48 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3336-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19139424, partition_space_available_1k=61951060, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1358152, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=15492, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42624, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=10143856, free=17368864, buffers=778472, cached=4373924, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.35, five_min_avg=3.89, fifteen_min_avg=2.31, cpu_share=5.12, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo')
05/26/2025 02:20:33.902 7f6f370f4f48 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data
05/26/2025 02:20:42.894 7f6f370f4f48 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3336-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19210268, partition_space_available_1k=61880216, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1365060, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=15548, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42628, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=10996360, free=16453304, buffers=778564, cached=4436888, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.61, five_min_avg=3.93, fifteen_min_avg=2.42, cpu_share=5.38, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo')
05/26/2025 02:21:34.026 7f6f370f4f48 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data
05/26/2025 02:21:42.605 7f6f370f4f48 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3336-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19229796, partition_space_available_1k=61860688, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1394580, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=15296, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42632, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=10337828, free=17087512, buffers=778596, cached=4461180, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.69, five_min_avg=3.78, fifteen_min_avg=2.46, cpu_share=3.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo')
05/26/2025 02:22:33.940 7f6f370f4f48 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data
05/26/2025 02:22:42.235 7f6f370f4f48 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3336-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19255968, partition_space_available_1k=61834516, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1383048, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=15364, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42636, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=10886376, free=16511396, buffers=778808, cached=4488536, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.28, five_min_avg=3.89, fifteen_min_avg=2.58, cpu_share=4.38, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo')
05/26/2025 02:23:33.961 7f6f370f4f48 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data
05/26/2025 02:23:43.308 7f6f370f4f48 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3336-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19286392, partition_space_available_1k=61804092, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1385764, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=15432, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42640, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=11050512, free=16313188, buffers=778876, cached=4522540, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.21, five_min_avg=3.9, fifteen_min_avg=2.67, cpu_share=4.62, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo')
05/26/2025 02:24:18.449 7f6f36deb0f0 [contrail-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-3336-1.: 36> in the [ObjectDatabaseInfo:NodeStatusUVE] map
05/26/2025 02:24:33.923 7f6f370f4f48 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data
05/26/2025 02:24:42.468 7f6f370f4f48 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-3336-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19435556, partition_space_available_1k=61654928, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1395952, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=15504, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42644, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665116, used=10187392, free=17014508, buffers=778996, cached=4684220, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.86, five_min_avg=3.86, fifteen_min_avg=2.73, cpu_share=4.38, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo')
05/26/2025 02:24:56.643 7f6f349edac8 [contrail-database-nodemgr] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-3336-1.: 37> in the [ObjectDatabaseInfo:NodeStatusUVE] map