10/30/2025 01:45:07.949 7f396fd6f4a0 [contrail-config-database-nodemgr] [ERROR]: Cannot write http_port 8112 to /tmp/contrail-config-database-nodemgr.0.http_port
10/30/2025 01:45:07.950 7f396fd6f4a0 [contrail-config-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8112
10/30/2025 01:45:07.951 7f396fd6f860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-266-1.cluster.local  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>,  ]  description = Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
10/30/2025 01:45:07.951 7f396fd6f860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-266-1.cluster.local:ConfigDatabase:contrail-config-database-nodemgr:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 8112  start_time = 1761788707951303  collector_name =   collector_ip =   collector_list = [ 10.0.0.241:8086,  ]  >>  sm_queue_count = 1  max_sm_queue_count = 1  >>  
10/30/2025 01:45:08.208 7f396fd6fd60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-266-1.cluster.local  build_info = {"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-10-30 00:56:42.728728", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-12281-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed
"}]}  system_cpu_info = <<  num_socket = 1  num_cpu = 4  num_core_per_socket = 4  num_thread_per_core = 1  >>  running_package_version = package contrail-lib is not installed
  installed_package_version = package contrail-lib is not installed
  >>  
10/30/2025 01:45:08.212 7f396fd6fd60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-266-1.cluster.local  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>,  ]  description = Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
10/30/2025 01:45:08.212 7f396fd6fd60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-266-1.cluster.local  deleted = False  process_info = [ <<  process_name = cassandra  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1761788673000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>, <<  process_name = zookeeper  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1761788663000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>, <<  process_name = contrail-config-database-nodemgr  process_state = PROCESS_STATE_RUNNING  start_count = 1  stop_count = 0  exit_count = 0  last_start_time = 1761788706000000  last_stop_time =   last_exit_time =   core_file_list = [  ]  >>,  ]  build_info = {"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-10-30 00:56:42.728728", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-12281-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed
"}]}  >>  
10/30/2025 01:45:08.655 7f396fc3ab80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-266-1.cluster.local  database_usage = [ <<  disk_space_used_1k = 12065996  disk_space_available_1k = 70625056  config_db_size_1k = 84  >>,  ]  >>  
10/30/2025 01:45:08.982 7f3976bf1140 [contrail-config-database-nodemgr] [ERROR]: Failed to connect to database by CQL and socket: Result: 1
10/30/2025 01:45:08.984 7f396fc3a220 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-266-1.cluster.local  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
10/30/2025 01:45:11.679 7f396ff28ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] CassandraStatusUVE: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-266-1.cluster.local  cassandra_compaction_task = <<  pending_compaction_tasks = 0  >>  thread_pool_stats = [ <<  pool_name = ReadStage  active = 0  pending = 0  all_time_blocked = 0  >>, <<  pool_name = CompactionExecutor  active = 0  pending = 0  all_time_blocked = 0  >>, <<  pool_name = MutationStage  active = 0  pending = 0  all_time_blocked = 0  >>, <<  pool_name = MemtableFlushWriter  active = 0  pending = 0  all_time_blocked = 0  >>,  ]  >>  
10/30/2025 01:45:12.040 7f396fc3a900 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-k8s-manifests-266-1.cluster.local  process_status = [ <<  module_id = contrail-config-database-nodemgr  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.241:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>,  ]  description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
10/30/2025 01:45:15.295 7f3976bf1140 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr config_db_uuid'. Error is: Result: 1
10/30/2025 01:45:17.022 7f3976bf1140 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr to_bgp_keyspace'. Error is: Result: 1
10/30/2025 01:45:18.565 7f3976bf1140 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr svc_monitor_keyspace'. Error is: Result: 1
10/30/2025 01:45:19.873 7f3976bf1140 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr DISCOVERY_SERVER'. Error is: Result: 1
10/30/2025 01:45:21.284 7f3976bf1140 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr useragent'. Error is: Result: 1
10/30/2025 01:45:23.800 7f3976bf1140 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr dm_keyspace'. Error is: Result: 1
10/30/2025 01:45:23.813 7f3976bf1140 [contrail-config-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
10/30/2025 01:46:12.879 7f3976bf1140 [contrail-config-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
10/30/2025 01:47:11.245 7f3976bf1140 [contrail-config-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
10/30/2025 01:48:11.370 7f3976bf1140 [contrail-config-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
10/30/2025 01:49:12.699 7f3976bf1140 [contrail-config-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
10/30/2025 01:50:11.564 7f3976bf1140 [contrail-config-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
10/30/2025 01:51:11.468 7f3976bf1140 [contrail-config-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
10/30/2025 01:52:11.565 7f3976bf1140 [contrail-config-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
10/30/2025 01:53:11.599 7f3976bf1140 [contrail-config-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'
10/30/2025 01:54:11.676 7f3976bf1140 [contrail-config-database-nodemgr] [ERROR]: Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'