05/12/2025 01:56:46.546 7fc60f4886d8 [contrail-database-nodemgr] [ERROR]: Cannot write http_port 8103 to /tmp/contrail-database-nodemgr.0.http_port
05/12/2025 01:56:46.547 7fc60f4886d8 [contrail-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8103
05/12/2025 01:56:46.549 7fc60f48f048 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-187-1.cluster.local  process_status = [ <<  module_id = contrail-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>,  ]  description = Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
05/12/2025 01:56:46.549 7fc60f48f048 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-187-1.cluster.local:Database:contrail-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8103  start_time = 1747015006548748  collector_name =   collector_ip =   collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 1  >>  
05/12/2025 01:56:46.872 7fc60f49e978 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-187-1.cluster.local  build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-05-12 00:51:39.361756", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-9142-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}  system_cpu_info = <<  num_socket = 1  num_cpu = 4  num_core_per_socket = 4  num_thread_per_core = 1  >>  running_package_version = nightly-nightly.el7  installed_package_version = nightly-nightly.el7  >>  
05/12/2025 01:56:46.875 7fc60f49eac8 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-187-1.cluster.local  process_status = [ <<  module_id = contrail-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>,  ]  description = Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
05/12/2025 01:56:46.876 7fc60f49eac8 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-187-1.cluster.local  deleted = False  process_info = [ <<  process_name = cassandra  process_state = PROCESS_STATE_EXITED  start_count = 0  stop_count = 0  exit_count = 0  last_start_time =   last_stop_time =   last_exit_time =   core_file_list = [  ]  >>, <<  process_name = contrail-query-engine  process_state = PROCESS_STATE_EXITED  start_count = 0  stop_count = 0  exit_count = 0  last_start_time =   last_stop_time =   last_exit_time =   core_file_list = [  ]  >>, <<  process_name = contrail-database-nodemgr  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1747015004000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>,  ]  build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-05-12 00:51:39.361756", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-9142-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}  >>  
05/12/2025 01:56:47.111 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get database usage, will retry: cassandra
05/12/2025 01:56:47.296 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get database usage, will retry: cassandra
05/12/2025 01:56:47.552 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get database usage, will retry: cassandra
05/12/2025 01:56:47.784 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get database usage, giving up: cassandra
05/12/2025 01:56:47.980 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to connect to database by CQL: cassandra
05/12/2025 01:56:47.983 7fc6108ba240 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-187-1.cluster.local  process_status = [ <<  module_id = contrail-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>,  ]  description = Cassandra state detected DOWN. Disk space for DB not retrievable. Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
05/12/2025 01:56:48.155 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: cassandra
05/12/2025 01:56:48.165 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 01:56:49.178 7fc60f49e0f0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-187-1.cluster.local  disk_usage_info = { /dev/vda5 : <<  partition_type = xfs  partition_space_used_1k = 9786220  partition_space_available_1k = 72957056  percentage_partition_space_used = 12  >> }  process_mem_cpu_usage = { contrail-database-nodemgr : <<  mem_virt = 51420  cpu_share = 0  mem_res = 0  >> }  system_mem_usage = <<  total = 16115472  used = 3620312  free = 4010220  buffers = 5196  cached = 8479744  node_type = database-node  >>  system_cpu_usage = <<  one_min_avg = 6.65  five_min_avg = 2.62  fifteen_min_avg = 1.02  cpu_share = 0  node_type = database-node  >>  >>  
05/12/2025 01:56:50.551 7fc610c4a0f0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/12/2025 01:56:50.552 7fc610c4a0f0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-187-1.cluster.local  process_status = [ <<  module_id = contrail-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Disk space for DB not retrievable. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/12/2025 01:56:50.552 7fc610c4a0f0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-187-1.cluster.local  process_status = [ <<  module_id = contrail-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Disk space for DB not retrievable. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/12/2025 01:56:50.552 7fc610c4a0f0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-187-1.cluster.local:Database:contrail-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8103  start_time = 1747015006548748  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/12/2025 01:56:54.557 7fc610c4a0f0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/12/2025 01:56:54.558 7fc610c4a0f0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-187-1.cluster.local  process_status = [ <<  module_id = contrail-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Disk space for DB not retrievable. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/12/2025 01:56:54.558 7fc610c4a0f0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-187-1.cluster.local  process_status = [ <<  module_id = contrail-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Disk space for DB not retrievable. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/12/2025 01:56:54.558 7fc610c4a0f0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-187-1.cluster.local:Database:contrail-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8103  start_time = 1747015006548748  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/12/2025 01:56:58.563 7fc610c4a0f0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/12/2025 01:56:58.683 7fc610c4a0f0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-187-1.cluster.local  process_status = [ <<  module_id = contrail-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Disk space for DB not retrievable. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/12/2025 01:56:58.683 7fc610c4a0f0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-187-1.cluster.local  process_status = [ <<  module_id = contrail-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Disk space for DB not retrievable. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/12/2025 01:56:58.684 7fc610c4a0f0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-187-1.cluster.local:Database:contrail-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8103  start_time = 1747015006548748  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/12/2025 01:57:02.565 7fc618d7bf60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/12/2025 01:57:02.565 7fc618d7bf60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-187-1.cluster.local  process_status = [ <<  module_id = contrail-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Disk space for DB not retrievable. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/12/2025 01:57:02.566 7fc618d7bf60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-187-1.cluster.local  process_status = [ <<  module_id = contrail-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Disk space for DB not retrievable. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/12/2025 01:57:02.566 7fc618d7bf60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-187-1.cluster.local:Database:contrail-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8103  start_time = 1747015006548748  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/12/2025 01:57:06.570 7fc618d7bf60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/12/2025 01:57:06.570 7fc618d7bf60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-187-1.cluster.local  process_status = [ <<  module_id = contrail-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Disk space for DB not retrievable. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/12/2025 01:57:06.571 7fc618d7bf60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-187-1.cluster.local  process_status = [ <<  module_id = contrail-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Disk space for DB not retrievable. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/12/2025 01:57:06.571 7fc618d7bf60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-187-1.cluster.local:Database:contrail-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8103  start_time = 1747015006548748  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/12/2025 01:57:10.576 7fc618d7bf60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/12/2025 01:57:10.577 7fc618d7bf60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-187-1.cluster.local  process_status = [ <<  module_id = contrail-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Disk space for DB not retrievable. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/12/2025 01:57:10.577 7fc618d7bf60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-187-1.cluster.local  process_status = [ <<  module_id = contrail-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Disk space for DB not retrievable. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/12/2025 01:57:10.577 7fc618d7bf60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-187-1.cluster.local:Database:contrail-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8103  start_time = 1747015006548748  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/12/2025 01:57:14.582 7fc618d7bf60 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail
05/12/2025 01:57:14.582 7fc618d7bf60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-187-1.cluster.local  process_status = [ <<  module_id = contrail-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Disk space for DB not retrievable. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/12/2025 01:57:14.583 7fc618d7bf60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-187-1.cluster.local  process_status = [ <<  module_id = contrail-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = Connect to Idle on EvTcpConnectFail  >>,  ]  description = Cassandra state detected DOWN. Disk space for DB not retrievable. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down  >>,  ]  >>  
05/12/2025 01:57:14.583 7fc618d7bf60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-187-1.cluster.local:Database:contrail-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8103  start_time = 1747015006548748  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/12/2025 01:57:18.657 7fc610abf0f0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-187-1.cluster.local  process_status = [ <<  module_id = contrail-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Disk space for DB not retrievable. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/12/2025 01:57:19.550 7fc6108c5630 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed
05/12/2025 01:57:19.564 7fc60f488eb8 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-187-1.cluster.local  process_status = [ <<  module_id = contrail-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Down  description = ClientInit to Idle on EvTcpClose  >>,  ]  description = Cassandra state detected DOWN. Disk space for DB not retrievable. Collector:Collector[ClientInit to Idle on EvTcpClose] connection down  >>,  ]  >>  
05/12/2025 01:57:19.564 7fc60f488eb8 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-187-1.cluster.local:Database:contrail-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 1  pid = 1  http_port = 8103  start_time = 1747015006548748  collector_name =   collector_ip = 10.0.0.50:8086  collector_list = [ 10.0.0.50:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 2  >>  
05/12/2025 01:57:23.561 7fc60f488eb8 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-187-1.cluster.local  process_status = [ <<  module_id = contrail-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.50:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Disk space for DB not retrievable. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/12/2025 01:57:52.204 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to connect to database by CQL: Result: 1
05/12/2025 01:58:03.150 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 01:58:54.583 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 01:59:52.060 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 02:00:51.543 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 02:01:53.113 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 02:02:52.540 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 02:03:52.087 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 02:04:51.980 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 02:05:53.526 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 02:06:52.370 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 02:07:52.307 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 02:08:52.206 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 02:09:52.187 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 02:10:52.225 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 02:11:52.392 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 02:12:52.434 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 02:13:52.610 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 02:14:52.221 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 02:15:52.662 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 02:16:52.516 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 02:17:52.088 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 02:18:52.327 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 02:19:52.005 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 02:20:52.098 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 02:21:51.980 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 02:22:52.072 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 02:23:52.142 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 02:24:52.110 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 02:25:52.234 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 02:26:52.535 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 02:27:52.287 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 02:28:52.331 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 02:29:52.415 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 02:30:52.176 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 02:31:52.184 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 02:32:52.243 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 02:33:52.103 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 02:34:52.338 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 02:35:52.542 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 02:36:52.299 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 02:37:52.593 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 02:38:52.316 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
05/12/2025 02:39:52.437 7fc6130d0048 [contrail-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'