10/16/2025 01:48:40.478 7f40c096d440 [contrail-config-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 10/16/2025 01:48:40.478 7f40c096d440 [contrail-config-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 10/16/2025 01:48:40.478 7f40c096d440 [contrail-config-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 10/16/2025 01:48:40.498 7f40bde134a0 [contrail-config-database-nodemgr] [ERROR]: Cannot write http_port 8112 to /tmp/contrail-config-database-nodemgr.0.http_port 10/16/2025 01:48:40.499 7f40bde134a0 [contrail-config-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8112 10/16/2025 01:48:40.504 7f40bde13860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 10/16/2025 01:48:40.510 7f40bde13860 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/16/2025 01:48:40.514 7f40bde13860 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 10/16/2025 01:48:40.549 7f40bde13860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 10/16/2025 01:48:40.549 7f40bde13860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:48:40.550 7f40bde13860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1573-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 10/16/2025 01:48:40.550 7f40bde13860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:48:40.550 7f40bde13860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:48:40.550 7f40bde13860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1573-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1760579320512380 collector_name = collector_ip = collector_list = [ 10.0.0.65:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 10/16/2025 01:48:40.550 7f40bde13860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:48:40.778 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 34556457186010011272229012588281694392906136420600751419431242368737454551773, 'start': '1760579315000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'zookeeper', 'group': 'zookeeper', 'pid': 13567962000285319364630629966010651997032870125271298664390589772205318599336, 'start': '1760579315000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-config-database-nodemgr', 'group': 'contrail-config-database-nodemgr', 'pid': 48999768670177403552233175855173073641311561520280225787692768294966021338128, 'start': '1760579315000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 10/16/2025 01:48:40.994 7f40bde13860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:48:40.994 7f40bde13860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1573-1 build_info = {"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-10-16 00:53:58.045224", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11746-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed "}]} system_cpu_info = << num_socket = 1 num_cpu = 4 num_core_per_socket = 4 num_thread_per_core = 1 >> running_package_version = package contrail-lib is not installed installed_package_version = package contrail-lib is not installed >> 10/16/2025 01:48:40.994 7f40bde13860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:48:40.995 7f40c096d440 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-juju-k8s-1573-1', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1760579320995344, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1573-1', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 10/16/2025 01:48:40.995 7f40c096d440 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-juju-k8s-1573-1', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1760579320995666, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1573-1', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1760579315000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1760579315000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1760579315000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-10-16 00:53:58.045224", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11746-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed\n"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 10/16/2025 01:48:40.998 7f40bde13860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:48:40.998 7f40bde13860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1573-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 10/16/2025 01:48:40.999 7f40bde13860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:48:40.999 7f40bde13860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:48:40.999 7f40bde13860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1573-1 deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1760579315000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = zookeeper process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1760579315000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1760579315000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-10-16 00:53:58.045224", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11746-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed "}]} >> 10/16/2025 01:48:40.999 7f40bde13860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:48:41.648 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/16/2025 01:48:42.100 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 10/16/2025 01:48:42.101 7f40bdb1fb80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:48:42.101 7f40bdb1fb80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-juju-k8s-1573-1 database_usage = [ << disk_space_used_1k = 12114028 disk_space_available_1k = 68976456 config_db_size_1k = 4 >>, ] >> 10/16/2025 01:48:42.101 7f40bdb1fb80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:48:42.355 7f40c096d440 [contrail-config-database-nodemgr] [ERROR]: Failed to connect to database by CQL and socket: Result: 1 10/16/2025 01:48:42.356 7f40c096d440 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-juju-k8s-1573-1', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1760579322356038, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1573-1', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 10/16/2025 01:48:42.357 7f40bdb1fb80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:48:42.357 7f40bdb1fb80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1573-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 10/16/2025 01:48:42.357 7f40bdb1fb80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:48:43.473 7f40bdb1fb80 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/16/2025 01:48:44.511 7f40bdb1fb80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/16/2025 01:48:44.512 7f40bdb1fb80 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/16/2025 01:48:44.513 7f40bdb1fb80 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/16/2025 01:48:44.513 7f40bdb1fb80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/16/2025 01:48:44.513 7f40bdb1fb80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1573-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.65:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/16/2025 01:48:44.513 7f40bdb1fb80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/16/2025 01:48:44.514 7f40bdb1fb80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/16/2025 01:48:44.514 7f40bdb1fb80 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/16/2025 01:48:44.514 7f40bdb1fb80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:48:44.514 7f40bdb1fb80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1573-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.65:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/16/2025 01:48:44.515 7f40bdb1fb80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:48:44.515 7f40bdb1fb80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:48:44.515 7f40bdb1fb80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1573-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1760579320512380 collector_name = collector_ip = 10.0.0.65:8086 collector_list = [ 10.0.0.65:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/16/2025 01:48:44.515 7f40bdb1fb80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:48:44.867 7f40c096d440 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 10/16/2025 01:48:46.885 7f40c096d440 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr config_db_uuid'. Error is: Result: 1 10/16/2025 01:48:48.514 7f40bde13f40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/16/2025 01:48:48.515 7f40bde13f40 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/16/2025 01:48:48.515 7f40bde13f40 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/16/2025 01:48:48.515 7f40bde13f40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/16/2025 01:48:48.515 7f40bde13f40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1573-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.65:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/16/2025 01:48:48.516 7f40bde13f40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/16/2025 01:48:48.516 7f40bde13f40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/16/2025 01:48:48.516 7f40bde13f40 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/16/2025 01:48:48.516 7f40bde13f40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:48:48.516 7f40bde13f40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1573-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.65:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/16/2025 01:48:48.516 7f40bde13f40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:48:48.516 7f40bde13f40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:48:48.516 7f40bde13f40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1573-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1760579320512380 collector_name = collector_ip = 10.0.0.65:8086 collector_list = [ 10.0.0.65:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/16/2025 01:48:48.516 7f40bde13f40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:48:48.880 7f40c096d440 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr to_bgp_keyspace'. Error is: Result: 1 10/16/2025 01:48:50.245 7f40c096d440 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr svc_monitor_keyspace'. Error is: Result: 1 10/16/2025 01:48:50.389 7f40bdb1fea0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/16/2025 01:48:51.041 7f40c096d440 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr DISCOVERY_SERVER'. Error is: Result: 1 10/16/2025 01:48:51.802 7f40c096d440 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr useragent'. Error is: Result: 1 10/16/2025 01:48:52.515 7f40bdb1fea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/16/2025 01:48:52.516 7f40bdb1fea0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/16/2025 01:48:52.516 7f40bdb1fea0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/16/2025 01:48:52.516 7f40bdb1fea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/16/2025 01:48:52.517 7f40bdb1fea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1573-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.65:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/16/2025 01:48:52.517 7f40bdb1fea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/16/2025 01:48:52.517 7f40bdb1fea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/16/2025 01:48:52.517 7f40bdb1fea0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/16/2025 01:48:52.518 7f40bdb1fea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:48:52.518 7f40bdb1fea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1573-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.65:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/16/2025 01:48:52.518 7f40bdb1fea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:48:52.518 7f40bdb1fea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:48:52.518 7f40bdb1fea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1573-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1760579320512380 collector_name = collector_ip = 10.0.0.65:8086 collector_list = [ 10.0.0.65:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/16/2025 01:48:52.518 7f40bdb1fea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:48:52.526 7f40c096d440 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr dm_keyspace'. Error is: Result: 1 10/16/2025 01:48:56.517 7f40bdb1fcc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/16/2025 01:48:56.518 7f40bdb1fcc0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/16/2025 01:48:56.519 7f40bdb1fcc0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/16/2025 01:48:56.519 7f40bdb1fcc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/16/2025 01:48:56.519 7f40bdb1fcc0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1573-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.65:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/16/2025 01:48:56.519 7f40bdb1fcc0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/16/2025 01:48:56.519 7f40bdb1fcc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/16/2025 01:48:56.520 7f40bdb1fcc0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/16/2025 01:48:56.520 7f40bdb1fcc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:48:56.520 7f40bdb1fcc0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1573-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.65:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/16/2025 01:48:56.520 7f40bdb1fcc0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:48:56.520 7f40bdb1fcc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:48:56.520 7f40bdb1fcc0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1573-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1760579320512380 collector_name = collector_ip = 10.0.0.65:8086 collector_list = [ 10.0.0.65:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/16/2025 01:48:56.521 7f40bdb1fcc0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:48:57.564 7f40c096d440 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-juju-k8s-1573-1', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1760579337564162, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1573-1', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1760579315000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1760579315000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1760579315000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-10-16 00:53:58.045224", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11746-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed\n"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 10/16/2025 01:48:57.564 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1573-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12119536, partition_space_available_1k=68970948, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1268016, cpu_share=0.0, mem_res=1255832, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53324, cpu_share=0.0, mem_res=48820, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51864, cpu_share=0.0, mem_res=40452, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=5140040, free=238640, buffers=405684, cached=10772016, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.98, five_min_avg=2.58, fifteen_min_avg=1.25, cpu_share=0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/16/2025 01:48:57.564 7f40bdb1fea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:48:57.565 7f40bdb1fea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1573-1 deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1760579315000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = zookeeper process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1760579315000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1760579315000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-10-16 00:53:58.045224", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11746-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed "}]} >> 10/16/2025 01:48:57.565 7f40bdb1fea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:48:57.565 7f40bdb1fea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:48:57.565 7f40bdb1fea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1573-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 12119536 partition_space_available_1k = 68970948 percentage_partition_space_used = 15 >> } all_core_file_list = [ ] process_mem_cpu_usage = { cassandra : << mem_virt = 1268016 cpu_share = 0.0 mem_res = 1255832 >>zookeeper : << mem_virt = 53324 cpu_share = 0.0 mem_res = 48820 >>contrail-config-database-nodemgr : << mem_virt = 51864 cpu_share = 0.0 mem_res = 40452 >> } system_mem_usage = << total = 16184208 used = 5140040 free = 238640 buffers = 405684 cached = 10772016 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 5.98 five_min_avg = 2.58 fifteen_min_avg = 1.25 cpu_share = 0 node_type = config-database-node >> >> 10/16/2025 01:48:57.565 7f40bdb1fea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:49:00.420 7f40bdb1fea0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/16/2025 01:49:00.519 7f40bdb1fea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/16/2025 01:49:00.520 7f40bdb1fea0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 10/16/2025 01:49:00.520 7f40bdb1fea0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/16/2025 01:49:00.520 7f40bdb1fea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/16/2025 01:49:00.520 7f40bdb1fea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1573-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.65:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/16/2025 01:49:00.521 7f40bdb1fea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/16/2025 01:49:00.521 7f40bdb1fea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/16/2025 01:49:00.521 7f40bdb1fea0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/16/2025 01:49:00.521 7f40bdb1fea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:49:00.521 7f40bdb1fea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1573-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.65:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/16/2025 01:49:00.521 7f40bdb1fea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:49:00.521 7f40bdb1fea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:49:00.521 7f40bdb1fea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1573-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1760579320512380 collector_name = collector_ip = 10.0.0.65:8086 collector_list = [ 10.0.0.65:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/16/2025 01:49:00.521 7f40bdb1fea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:49:04.524 7f40bdb1fea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/16/2025 01:49:04.525 7f40bdb1fea0 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected 10/16/2025 01:49:04.526 7f40bdb1fea0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/16/2025 01:49:04.526 7f40bdb1fea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/16/2025 01:49:04.526 7f40bdb1fea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1573-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.65:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/16/2025 01:49:04.526 7f40bdb1fea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/16/2025 01:49:04.526 7f40bdb1fea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 10/16/2025 01:49:04.526 7f40bdb1fea0 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 10/16/2025 01:49:04.527 7f40bdb1fea0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 10/16/2025 01:49:04.527 7f40bdb1fea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 10/16/2025 01:49:04.527 7f40bdb1fea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 10/16/2025 01:49:04.831 7f40bdb66360 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 10/16/2025 01:49:04.831 7f40bdb66e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit] 10/16/2025 01:49:04.832 7f40bdb66e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle] 10/16/2025 01:49:04.832 7f40bdb66e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:49:04.832 7f40bdb66e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1573-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.65:8086, ] status = Down description = ClientInit to Idle on EvTcpClose >>, ] description = Cassandra state detected DOWN. Collector:Collector[ClientInit to Idle on EvTcpClose] connection down >>, ] >> 10/16/2025 01:49:04.832 7f40bdb66e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:49:04.832 7f40bdb66e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:49:04.832 7f40bdb66e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1573-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8112 start_time = 1760579320512380 collector_name = collector_ip = 10.0.0.65:8086 collector_list = [ 10.0.0.65:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/16/2025 01:49:04.833 7f40bdb66e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/16/2025 01:49:08.837 7f40bdb66e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/16/2025 01:49:08.838 7f40bdb66e00 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected 10/16/2025 01:49:08.838 7f40bdb66e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/16/2025 01:49:08.839 7f40bdb66e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/16/2025 01:49:08.839 7f40bdb66e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1573-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.65:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/16/2025 01:49:08.839 7f40bdb66e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/16/2025 01:49:08.839 7f40bdb66e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 10/16/2025 01:49:08.839 7f40bdb66e00 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 10/16/2025 01:49:08.839 7f40bdb66e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 10/16/2025 01:49:08.839 7f40bdb66e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 10/16/2025 01:49:08.840 7f40bdb66e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 10/16/2025 01:49:08.855 7f40bde13b80 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001013 10/16/2025 01:49:08.856 7f40bde13b80 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 10/16/2025 01:49:08.856 7f40bdb1fe00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 10/16/2025 01:49:08.856 7f40bdb1fe00 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 10/16/2025 01:49:08.857 7f40bdb1fe00 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 10/16/2025 01:49:08.857 7f40bdb1fe00 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/16/2025 01:49:08.857 7f40bdb1fe00 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 10/16/2025 01:49:08.858 7f40bdb1fe00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 10/16/2025 01:49:08.858 7f40bdb1fd60 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 10/16/2025 01:49:08.859 7f40bdb1fd60 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 10/16/2025 01:49:26.672 7f40bde13b80 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001435 10/16/2025 01:49:26.673 7f40bde13b80 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 10/16/2025 01:49:26.673 7f40bde13900 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 10/16/2025 01:49:26.674 7f40bde13900 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 3 10/16/2025 01:49:26.674 7f40bde13900 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 10/16/2025 01:49:26.674 7f40bde13900 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/16/2025 01:49:26.676 7f40bde13900 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 10/16/2025 01:49:26.676 7f40bdb66ae0 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 10/16/2025 01:49:26.677 7f40bdb66ae0 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 10/16/2025 01:49:31.684 7f40be7685e0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/16/2025 01:49:39.697 7f40be7685e0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/16/2025 01:49:41.538 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/16/2025 01:49:42.673 7f40c096d440 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 10/16/2025 01:49:42.673 7f40c096d440 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-juju-k8s-1573-1', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1760579382673905, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1573-1', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.65:8086'], status='Up', description='ClientInit to Established on EvSandeshCtrlMessageRecv')]), flag_infos=None, description='')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 10/16/2025 01:49:45.327 7f40c096d440 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 10/16/2025 01:49:49.554 7f40bdb1f5e0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/16/2025 01:49:51.249 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1573-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12899396, partition_space_available_1k=68191088, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1333476, cpu_share=0.34, mem_res=1316836, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=132496, cpu_share=0.14, mem_res=127056, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50948, cpu_share=0.01, mem_res=40384, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7518936, free=261260, buffers=379116, cached=8357648, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=15.58, five_min_avg=5.83, fifteen_min_avg=2.44, cpu_share=21.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/16/2025 01:49:54.015 7f40bdb1f2c0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/16/2025 01:49:57.376 7f40bde13f40 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/16/2025 01:50:02.255 7f40bde13f40 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/16/2025 01:50:06.531 7f40bde13f40 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/16/2025 01:50:15.339 7f40bdb1fcc0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/16/2025 01:50:19.193 7f40bdb1fcc0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/16/2025 01:50:24.752 7f40bdb1fcc0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/16/2025 01:50:38.417 7f40bdb1fb80 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/16/2025 01:50:41.214 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/16/2025 01:50:41.650 7f40c096d440 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 10/16/2025 01:50:42.462 7f40c096d440 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 10/16/2025 01:50:48.200 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1573-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12935244, partition_space_available_1k=68155240, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1333668, cpu_share=0.21, mem_res=1317140, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=132820, cpu_share=0.01, mem_res=127320, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51140, cpu_share=0.01, mem_res=40436, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7714592, free=296996, buffers=379412, cached=8127340, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=9.56, five_min_avg=5.9, fifteen_min_avg=2.67, cpu_share=14.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/16/2025 01:51:41.218 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/16/2025 01:51:41.655 7f40c096d440 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 10/16/2025 01:51:42.460 7f40c096d440 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 10/16/2025 01:51:48.211 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1573-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12943332, partition_space_available_1k=68147152, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1333724, cpu_share=0.03, mem_res=1317284, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=132824, cpu_share=0.0, mem_res=127320, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51108, cpu_share=0.01, mem_res=40596, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7756860, free=272984, buffers=390116, cached=8098768, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.47, five_min_avg=5.68, fifteen_min_avg=2.79, cpu_share=10.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/16/2025 01:52:41.226 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/16/2025 01:52:41.639 7f40c096d440 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 10/16/2025 01:52:42.551 7f40c096d440 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 10/16/2025 01:52:48.228 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1573-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12950232, partition_space_available_1k=68140252, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1333644, cpu_share=0.03, mem_res=1317232, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=132836, cpu_share=0.0, mem_res=127320, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51236, cpu_share=0.01, mem_res=40552, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7763396, free=257120, buffers=390200, cached=8108040, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.31, five_min_avg=4.93, fifteen_min_avg=2.72, cpu_share=8.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/16/2025 01:53:41.230 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/16/2025 01:53:41.634 7f40c096d440 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 10/16/2025 01:53:42.507 7f40c096d440 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 10/16/2025 01:53:48.292 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1573-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12957264, partition_space_available_1k=68133220, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1333700, cpu_share=0.03, mem_res=1317304, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=132836, cpu_share=0.0, mem_res=127320, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51432, cpu_share=0.01, mem_res=40540, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7732944, free=276320, buffers=390284, cached=8119180, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.51, five_min_avg=4.45, fifteen_min_avg=2.69, cpu_share=8.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/16/2025 01:54:41.238 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/16/2025 01:54:41.656 7f40c096d440 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 10/16/2025 01:54:42.489 7f40c096d440 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 10/16/2025 01:54:48.251 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1573-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12965840, partition_space_available_1k=68124644, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1335064, cpu_share=0.04, mem_res=1317704, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=132920, cpu_share=0.0, mem_res=127452, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51388, cpu_share=0.01, mem_res=40352, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7740708, free=256120, buffers=390464, cached=8131452, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.86, five_min_avg=3.93, fifteen_min_avg=2.62, cpu_share=8.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/16/2025 01:55:13.701 7f40bde13f40 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/16/2025 01:55:41.273 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/16/2025 01:55:41.706 7f40c096d440 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 10/16/2025 01:55:42.466 7f40c096d440 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 10/16/2025 01:55:48.264 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1573-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12973520, partition_space_available_1k=68116964, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1334892, cpu_share=0.03, mem_res=1317780, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=133108, cpu_share=0.0, mem_res=127584, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51472, cpu_share=0.01, mem_res=40608, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7746268, free=295200, buffers=390528, cached=8087124, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.63, five_min_avg=3.49, fifteen_min_avg=2.56, cpu_share=8.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/16/2025 01:56:03.474 7f40be768040 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/16/2025 01:56:06.439 7f40be768040 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/16/2025 01:56:32.526 7f40bdb66ea0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/16/2025 01:56:41.286 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/16/2025 01:56:41.759 7f40c096d440 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 10/16/2025 01:56:42.691 7f40c096d440 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 10/16/2025 01:56:48.277 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1573-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12986284, partition_space_available_1k=68104200, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1336584, cpu_share=0.04, mem_res=1319348, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=138136, cpu_share=0.01, mem_res=132600, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51500, cpu_share=0.01, mem_res=40464, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7861792, free=548544, buffers=389892, cached=7722012, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.24, five_min_avg=3.83, fifteen_min_avg=2.72, cpu_share=10.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/16/2025 01:57:41.308 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/16/2025 01:57:41.757 7f40c096d440 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 10/16/2025 01:57:42.815 7f40c096d440 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 10/16/2025 01:57:48.292 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1573-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13986508, partition_space_available_1k=67103976, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1336272, cpu_share=0.03, mem_res=1319368, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=138524, cpu_share=0.0, mem_res=132864, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51256, cpu_share=0.01, mem_res=40344, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7835984, free=166776, buffers=382048, cached=8141128, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.92, five_min_avg=3.76, fifteen_min_avg=2.76, cpu_share=11.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/16/2025 01:58:41.280 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/16/2025 01:58:41.719 7f40c096d440 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 10/16/2025 01:58:42.540 7f40c096d440 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 10/16/2025 01:58:48.308 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1573-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14266684, partition_space_available_1k=66823800, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1340024, cpu_share=0.1, mem_res=1323372, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=139848, cpu_share=0.0, mem_res=134316, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51404, cpu_share=0.01, mem_res=40420, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=8000016, free=273832, buffers=404056, cached=7848644, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.18, five_min_avg=3.56, fifteen_min_avg=2.76, cpu_share=12.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/16/2025 01:59:41.327 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/16/2025 01:59:41.864 7f40c096d440 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 10/16/2025 01:59:43.198 7f40c096d440 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 10/16/2025 01:59:48.304 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1573-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14284060, partition_space_available_1k=66806424, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1342292, cpu_share=0.11, mem_res=1324436, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=142832, cpu_share=0.0, mem_res=137352, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51420, cpu_share=0.01, mem_res=40460, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=8021520, free=231308, buffers=404108, cached=7869608, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.84, five_min_avg=3.81, fifteen_min_avg=2.89, cpu_share=10.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/16/2025 01:59:54.543 7f40be6e1d60 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/16/2025 02:00:41.307 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/16/2025 02:00:41.708 7f40c096d440 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 10/16/2025 02:00:42.550 7f40c096d440 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 10/16/2025 02:00:48.306 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1573-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14290764, partition_space_available_1k=66799720, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1342416, cpu_share=0.04, mem_res=1324780, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=143488, cpu_share=0.0, mem_res=137880, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51264, cpu_share=0.01, mem_res=40352, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=8037300, free=228068, buffers=404228, cached=7857236, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.52, five_min_avg=3.62, fifteen_min_avg=2.88, cpu_share=8.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/16/2025 02:01:41.309 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/16/2025 02:01:41.737 7f40c096d440 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 10/16/2025 02:01:42.528 7f40c096d440 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 10/16/2025 02:01:48.320 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1573-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14303072, partition_space_available_1k=66787412, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1342236, cpu_share=0.03, mem_res=1324836, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=144004, cpu_share=0.0, mem_res=138540, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51284, cpu_share=0.01, mem_res=40460, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=8053180, free=275532, buffers=409680, cached=7789124, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.13, five_min_avg=3.72, fifteen_min_avg=2.95, cpu_share=10.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/16/2025 02:01:54.001 7f40be7685e0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/16/2025 02:01:56.931 7f40be7685e0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/16/2025 02:02:01.400 7f40be7685e0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/16/2025 02:02:41.316 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/16/2025 02:02:41.715 7f40c096d440 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 10/16/2025 02:02:42.534 7f40c096d440 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 10/16/2025 02:02:48.334 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1573-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14310404, partition_space_available_1k=66780080, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1342392, cpu_share=0.04, mem_res=1324864, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=144504, cpu_share=0.0, mem_res=138936, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51316, cpu_share=0.01, mem_res=40464, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=8062904, free=294060, buffers=409168, cached=7761608, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.36, five_min_avg=3.61, fifteen_min_avg=2.97, cpu_share=9.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/16/2025 02:03:41.411 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/16/2025 02:03:41.837 7f40c096d440 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 10/16/2025 02:03:42.624 7f40c096d440 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 10/16/2025 02:03:48.346 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1573-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14316848, partition_space_available_1k=66773636, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1342924, cpu_share=0.05, mem_res=1325136, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=145856, cpu_share=0.0, mem_res=140124, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51388, cpu_share=0.01, mem_res=40528, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=8065448, free=280548, buffers=409216, cached=7770256, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.79, five_min_avg=3.6, fifteen_min_avg=3.0, cpu_share=9.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/16/2025 02:04:41.382 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/16/2025 02:04:41.818 7f40c096d440 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 10/16/2025 02:04:42.857 7f40c096d440 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 10/16/2025 02:04:48.354 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1573-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14706796, partition_space_available_1k=66383688, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1343952, cpu_share=0.05, mem_res=1326008, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=147520, cpu_share=0.0, mem_res=141840, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51292, cpu_share=0.01, mem_res=40424, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7922972, free=247068, buffers=409548, cached=7948664, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.68, five_min_avg=3.9, fifteen_min_avg=3.14, cpu_share=11.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/16/2025 02:05:19.431 7f40bdb66b80 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/16/2025 02:05:22.773 7f40bdb66b80 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/16/2025 02:05:41.356 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/16/2025 02:05:41.769 7f40c096d440 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 10/16/2025 02:05:42.609 7f40c096d440 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 10/16/2025 02:05:48.357 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1573-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14920784, partition_space_available_1k=66169700, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1344164, cpu_share=0.04, mem_res=1325900, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=148028, cpu_share=0.0, mem_res=142368, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51232, cpu_share=0.01, mem_res=40516, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=8084996, free=320448, buffers=405708, cached=7718760, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.56, five_min_avg=4.02, fifteen_min_avg=3.23, cpu_share=10.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/16/2025 02:05:48.528 7f40bdb66a40 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/16/2025 02:06:14.835 7f40bdb662c0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/16/2025 02:06:41.357 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/16/2025 02:06:41.860 7f40c096d440 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 10/16/2025 02:06:42.802 7f40c096d440 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 10/16/2025 02:06:48.361 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1573-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14927824, partition_space_available_1k=66162660, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1344080, cpu_share=0.04, mem_res=1325932, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=148596, cpu_share=0.0, mem_res=143028, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51252, cpu_share=0.01, mem_res=40372, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=8012956, free=398260, buffers=410520, cached=7708484, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.44, five_min_avg=4.43, fifteen_min_avg=3.42, cpu_share=12.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/16/2025 02:07:08.614 7f40bdb66cc0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/16/2025 02:07:13.074 7f40bdb66cc0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/16/2025 02:07:15.884 7f40bdb66cc0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/16/2025 02:07:21.045 7f40bdb1fe00 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 10/16/2025 02:07:41.364 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 10/16/2025 02:07:41.791 7f40c096d440 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 10/16/2025 02:07:42.666 7f40c096d440 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 10/16/2025 02:07:48.381 7f40c096d440 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1573-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14927704, partition_space_available_1k=66162780, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1344284, cpu_share=0.04, mem_res=1326252, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=149296, cpu_share=0.0, mem_res=143688, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51200, cpu_share=0.01, mem_res=40400, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7885760, free=520104, buffers=411080, cached=7713124, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.68, five_min_avg=4.13, fifteen_min_avg=3.39, cpu_share=10.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 10/16/2025 02:08:09.964 7f40bdb66ea0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map