08/18/2025 01:44:53.127 7f3ee4335108 [contrail-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 08/18/2025 01:44:53.127 7f3ee4335108 [contrail-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 08/18/2025 01:44:53.127 7f3ee4335108 [contrail-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 08/18/2025 01:44:53.141 7f3ee184e8d0 [contrail-database-nodemgr] [ERROR]: Cannot write http_port 8103 to /tmp/contrail-database-nodemgr.0.http_port 08/18/2025 01:44:53.141 7f3ee184e8d0 [contrail-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8103 08/18/2025 01:44:53.142 7f3ee1852240 [contrail-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 08/18/2025 01:44:53.147 7f3ee1852240 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 08/18/2025 01:44:53.147 7f3ee1852240 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 08/18/2025 01:44:53.147 7f3ee1852240 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 08/18/2025 01:44:53.148 7f3ee1852240 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/18/2025 01:44:53.148 7f3ee1852240 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1408-1 process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 08/18/2025 01:44:53.151 7f3ee1852240 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/18/2025 01:44:53.151 7f3ee1852240 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/18/2025 01:44:53.152 7f3ee1852240 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1408-1:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1755481493147709 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 08/18/2025 01:44:53.152 7f3ee1852240 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/18/2025 01:44:53.353 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 104132997368757121323595090967921539074826345871197757737629803722198596205231, 'start': '1755481489000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-query-engine', 'group': 'contrail-query-engine', 'pid': 112940067089296217901664615408207999564878514376246893129331773150092634678140, 'start': '1755481489000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-database-nodemgr', 'group': 'contrail-database-nodemgr', 'pid': 19645928604493716434318230586706758425048661081664411298772779754604795215222, 'start': '1755481489000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 08/18/2025 01:44:53.514 7f3ee18edac8 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/18/2025 01:44:53.514 7f3ee18edac8 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1408-1 build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-08-18 00:43:03.682284", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-10395-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 4 num_core_per_socket = 4 num_thread_per_core = 1 >> running_package_version = nightly-nightly.el7 installed_package_version = nightly-nightly.el7 >> 08/18/2025 01:44:53.514 7f3ee18edac8 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/18/2025 01:44:53.515 7f3ee4335108 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-juju-k8s-1408-1', _node_type='Database', _instance_id='0', _timestamp=1755481493514917, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1408-1', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 08/18/2025 01:44:53.515 7f3ee4335108 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-juju-k8s-1408-1', _node_type='Database', _instance_id='0', _timestamp=1755481493515223, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1408-1', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1755481489000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1755481489000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1755481489000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2025-08-18 00:43:03.682284", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-10395-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 08/18/2025 01:44:53.516 7f3ee18edf60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/18/2025 01:44:53.516 7f3ee18edf60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1408-1 process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 08/18/2025 01:44:53.516 7f3ee18edf60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/18/2025 01:44:53.517 7f3ee18edf60 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/18/2025 01:44:53.517 7f3ee18edf60 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1408-1 deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1755481489000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-query-engine process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1755481489000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1755481489000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-08-18 00:43:03.682284", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-10395-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} >> 08/18/2025 01:44:53.517 7f3ee18edf60 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/18/2025 01:44:53.957 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 08/18/2025 01:44:54.400 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:DatabaseUsage] map 08/18/2025 01:44:54.402 7f3ee413f4e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/18/2025 01:44:54.402 7f3ee413f4e0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-juju-k8s-1408-1 database_usage = [ << disk_space_used_1k = 11888560 disk_space_available_1k = 69201924 analytics_db_size_1k = 4 >>, ] >> 08/18/2025 01:44:54.402 7f3ee413f4e0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/18/2025 01:44:54.967 7f3ee4335108 [contrail-database-nodemgr] [ERROR]: Failed to connect to database by CQL and socket: Result: 1 08/18/2025 01:44:54.967 7f3ee4335108 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-juju-k8s-1408-1', _node_type='Database', _instance_id='0', _timestamp=1755481494967766, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1408-1', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 08/18/2025 01:44:54.969 7f3ee0979eb8 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 08/18/2025 01:44:54.969 7f3ee0979eb8 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1408-1 process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 08/18/2025 01:44:54.975 7f3ee0979eb8 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 08/18/2025 01:44:56.699 7f3ee4335108 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/18/2025 01:44:57.143 7f3ee0628c18 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 08/18/2025 01:44:57.144 7f3ee0628c18 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 08/18/2025 01:44:57.144 7f3ee0628c18 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 08/18/2025 01:44:57.144 7f3ee0628c18 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 08/18/2025 01:44:57.145 7f3ee0628c18 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1408-1 process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 08/18/2025 01:44:57.145 7f3ee0628c18 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 08/18/2025 01:44:57.145 7f3ee0628c18 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 08/18/2025 01:44:57.145 7f3ee0628c18 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 08/18/2025 01:44:57.145 7f3ee0628c18 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 08/18/2025 01:44:57.145 7f3ee0628c18 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 08/18/2025 01:44:57.145 7f3ee0628c18 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 08/18/2025 01:44:57.149 7f3ee1853f60 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001014 08/18/2025 01:44:57.149 7f3ee1853f60 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 08/18/2025 01:44:57.150 7f3ee0659b70 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 08/18/2025 01:44:57.150 7f3ee0659b70 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 08/18/2025 01:44:57.150 7f3ee0659b70 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 08/18/2025 01:44:57.150 7f3ee0659b70 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 08/18/2025 01:44:57.150 7f3ee0659b70 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 08/18/2025 01:44:57.150 7f3ee0659b70 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 08/18/2025 01:44:57.151 7f3ee0628c18 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 08/18/2025 01:44:57.151 7f3ee0628c18 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 08/18/2025 01:44:58.000 7f3ee4335108 [contrail-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7200 repair -pr ContrailAnalyticsCql'. Error is: Result: 1 08/18/2025 01:45:03.049 7f3ee4335108 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-juju-k8s-1408-1', _node_type='Database', _instance_id='0', _timestamp=1755481503049577, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1408-1', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1755481489000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1755481489000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1755481489000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2025-08-18 00:43:03.682284", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-10395-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 08/18/2025 01:45:03.050 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1408-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=11896752, partition_space_available_1k=69193732, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1251284, cpu_share=0.0, mem_res=1228764, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=9692, cpu_share=0.0, mem_res=5348, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=55188, cpu_share=0.0, mem_res=40336, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184204, used=6392040, free=244348, buffers=424584, cached=9123232, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.26, five_min_avg=2.48, fifteen_min_avg=1.19, cpu_share=0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 08/18/2025 01:45:48.164 7f3ee3e57630 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 08/18/2025 01:45:52.431 7f3ee3e57630 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 08/18/2025 01:45:53.760 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 08/18/2025 01:45:54.661 7f3ee4335108 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-juju-k8s-1408-1', _node_type='Database', _instance_id='0', _timestamp=1755481554661432, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1408-1', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.50:8086'], status='Up', description='ClientInit to Established on EvSandeshCtrlMessageRecv')]), flag_infos=None, description='')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 08/18/2025 01:45:55.194 7f3ee0623240 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 08/18/2025 01:45:55.629 7f3ee4335108 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/18/2025 01:46:00.658 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1408-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12013276, partition_space_available_1k=69077208, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1319668, cpu_share=0.25, mem_res=1290648, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11872, cpu_share=0.0, mem_res=7196, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=55560, cpu_share=0.01, mem_res=40508, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184204, used=6547220, free=281304, buffers=431840, cached=8923840, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.15, five_min_avg=2.73, fifteen_min_avg=1.36, cpu_share=12.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 08/18/2025 01:46:03.802 7f3ee1fee630 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 08/18/2025 01:46:08.276 7f3ee1fee630 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 08/18/2025 01:46:12.574 7f3ee3e07438 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 08/18/2025 01:46:16.965 7f3ee3e07438 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 08/18/2025 01:46:23.463 7f3ee3e07438 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 08/18/2025 01:46:28.541 7f3ee0637d68 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 08/18/2025 01:46:32.177 7f3ee3e07240 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 08/18/2025 01:46:35.645 7f3ee3e07240 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 08/18/2025 01:46:53.752 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 08/18/2025 01:46:55.249 7f3ee4335108 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/18/2025 01:47:00.289 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1408-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=11944412, partition_space_available_1k=69146072, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1334892, cpu_share=0.19, mem_res=1299580, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12268, cpu_share=0.0, mem_res=7460, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=55540, cpu_share=0.01, mem_res=40540, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184204, used=6619440, free=410560, buffers=429716, cached=8724488, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.0, five_min_avg=3.43, fifteen_min_avg=1.7, cpu_share=14.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 08/18/2025 01:47:22.063 7f3ee3e322e8 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 08/18/2025 01:47:53.750 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 08/18/2025 01:47:55.278 7f3ee4335108 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/18/2025 01:48:00.305 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1408-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=11952088, partition_space_available_1k=69138396, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1334284, cpu_share=0.05, mem_res=1297432, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12344, cpu_share=0.0, mem_res=7592, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=55520, cpu_share=0.01, mem_res=40520, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184204, used=6633068, free=386500, buffers=430008, cached=8734628, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.83, five_min_avg=3.08, fifteen_min_avg=1.68, cpu_share=7.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 08/18/2025 01:48:53.749 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 08/18/2025 01:48:55.290 7f3ee4335108 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/18/2025 01:49:00.325 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1408-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=11960808, partition_space_available_1k=69129676, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1338464, cpu_share=0.04, mem_res=1299864, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12424, cpu_share=0.0, mem_res=7724, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=55660, cpu_share=0.01, mem_res=40444, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184204, used=6661728, free=333316, buffers=432764, cached=8756396, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.42, five_min_avg=2.91, fifteen_min_avg=1.72, cpu_share=9.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 08/18/2025 01:49:03.670 7f3ee1853630 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 08/18/2025 01:49:06.273 7f3ee1853630 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 08/18/2025 01:49:10.205 7f3ee1853630 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 08/18/2025 01:49:53.797 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 08/18/2025 01:49:55.340 7f3ee4335108 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/18/2025 01:50:00.378 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1408-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=11971888, partition_space_available_1k=69118596, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1342888, cpu_share=0.04, mem_res=1302724, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12452, cpu_share=0.0, mem_res=7724, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=55436, cpu_share=0.01, mem_res=40552, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184204, used=6688652, free=290952, buffers=432948, cached=8771652, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.14, five_min_avg=2.73, fifteen_min_avg=1.73, cpu_share=8.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 08/18/2025 01:50:53.778 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 08/18/2025 01:50:55.283 7f3ee4335108 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/18/2025 01:51:00.307 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1408-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=11979024, partition_space_available_1k=69111460, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1350980, cpu_share=0.04, mem_res=1308960, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12536, cpu_share=0.0, mem_res=7856, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=55444, cpu_share=0.01, mem_res=40352, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184204, used=6662732, free=308600, buffers=433000, cached=8779872, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.73, five_min_avg=2.51, fifteen_min_avg=1.71, cpu_share=7.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 08/18/2025 01:51:27.419 7f3ee1fbb8d0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 08/18/2025 01:51:53.807 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 08/18/2025 01:51:55.322 7f3ee4335108 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/18/2025 01:52:00.349 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1408-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=11987992, partition_space_available_1k=69102492, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1360820, cpu_share=0.05, mem_res=1316824, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12568, cpu_share=0.0, mem_res=7856, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=55652, cpu_share=0.01, mem_res=40512, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184204, used=6680408, free=279288, buffers=433104, cached=8791404, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.0, five_min_avg=2.4, fifteen_min_avg=1.72, cpu_share=8.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 08/18/2025 01:52:53.834 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 08/18/2025 01:52:55.508 7f3ee4335108 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/18/2025 01:53:00.539 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1408-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=11995604, partition_space_available_1k=69094880, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1357936, cpu_share=0.05, mem_res=1312216, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12624, cpu_share=0.0, mem_res=7988, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=55724, cpu_share=0.01, mem_res=40476, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184204, used=6754080, free=247080, buffers=429400, cached=8753644, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.06, five_min_avg=2.32, fifteen_min_avg=1.74, cpu_share=8.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 08/18/2025 01:53:53.815 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 08/18/2025 01:53:55.380 7f3ee4335108 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/18/2025 01:54:00.407 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1408-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12004824, partition_space_available_1k=69085660, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1361552, cpu_share=0.05, mem_res=1313368, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12624, cpu_share=0.0, mem_res=7988, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=55696, cpu_share=0.01, mem_res=40540, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184204, used=6714084, free=264348, buffers=430496, cached=8775276, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.58, five_min_avg=2.16, fifteen_min_avg=1.71, cpu_share=8.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 08/18/2025 01:54:53.825 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 08/18/2025 01:54:55.337 7f3ee4335108 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/18/2025 01:55:00.377 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1408-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12011820, partition_space_available_1k=69078664, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1364988, cpu_share=0.05, mem_res=1315040, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12636, cpu_share=0.0, mem_res=7988, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=55568, cpu_share=0.01, mem_res=40724, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184204, used=6743700, free=226420, buffers=430560, cached=8783524, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.35, five_min_avg=1.97, fifteen_min_avg=1.68, cpu_share=7.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 08/18/2025 01:55:02.910 7f3ee43386d8 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 08/18/2025 01:55:05.602 7f3ee43386d8 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 08/18/2025 01:55:09.640 7f3ee43386d8 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 08/18/2025 01:55:53.839 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 08/18/2025 01:55:55.638 7f3ee4335108 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/18/2025 01:56:00.664 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1408-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12023392, partition_space_available_1k=69067092, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1368664, cpu_share=0.05, mem_res=1317340, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12664, cpu_share=0.0, mem_res=7988, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=55512, cpu_share=0.01, mem_res=40624, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184204, used=6723472, free=277756, buffers=429628, cached=8753348, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.25, five_min_avg=1.84, fifteen_min_avg=1.65, cpu_share=8.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 08/18/2025 01:56:19.024 7f3ee1852438 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 08/18/2025 01:56:53.859 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 08/18/2025 01:56:56.069 7f3ee4335108 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/18/2025 01:57:01.105 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1408-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13182184, partition_space_available_1k=67908300, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1367664, cpu_share=0.05, mem_res=1320712, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12664, cpu_share=0.0, mem_res=7988, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54856, cpu_share=0.01, mem_res=40488, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184204, used=6935180, free=299692, buffers=444632, cached=8504700, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.65, five_min_avg=2.69, fifteen_min_avg=1.96, cpu_share=12.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 08/18/2025 01:57:53.875 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 08/18/2025 01:57:55.530 7f3ee4335108 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/18/2025 01:58:00.557 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1408-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13199392, partition_space_available_1k=67891092, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1381160, cpu_share=0.07, mem_res=1329848, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12564, cpu_share=0.0, mem_res=7856, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54800, cpu_share=0.01, mem_res=40376, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184204, used=6955676, free=290712, buffers=458652, cached=8479164, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.86, five_min_avg=2.95, fifteen_min_avg=2.08, cpu_share=10.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 08/18/2025 01:58:53.857 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 08/18/2025 01:58:55.347 7f3ee4335108 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/18/2025 01:59:00.375 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1408-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13220140, partition_space_available_1k=67870344, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1391364, cpu_share=0.08, mem_res=1335416, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12568, cpu_share=0.0, mem_res=7856, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54760, cpu_share=0.01, mem_res=40424, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184204, used=6976532, free=277064, buffers=456804, cached=8473804, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.14, five_min_avg=3.4, fifteen_min_avg=2.29, cpu_share=10.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 08/18/2025 01:59:53.873 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 08/18/2025 01:59:55.349 7f3ee4335108 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/18/2025 02:00:00.379 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1408-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13222556, partition_space_available_1k=67867928, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1402652, cpu_share=0.04, mem_res=1343384, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12604, cpu_share=0.0, mem_res=7856, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54836, cpu_share=0.01, mem_res=40664, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184204, used=7060236, free=178364, buffers=457140, cached=8488464, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.57, five_min_avg=2.98, fifteen_min_avg=2.21, cpu_share=7.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 08/18/2025 02:00:01.255 7f3ee413f4e0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 08/18/2025 02:00:03.914 7f3ee413f4e0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 08/18/2025 02:00:07.805 7f3ee413f4e0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 08/18/2025 02:00:53.882 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 08/18/2025 02:00:55.375 7f3ee4335108 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/18/2025 02:01:00.405 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1408-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13229924, partition_space_available_1k=67860560, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1409280, cpu_share=0.05, mem_res=1347572, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12732, cpu_share=0.0, mem_res=7988, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54656, cpu_share=0.01, mem_res=40568, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184204, used=6992212, free=281712, buffers=457564, cached=8452716, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.95, five_min_avg=2.75, fifteen_min_avg=2.18, cpu_share=8.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 08/18/2025 02:01:53.972 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 08/18/2025 02:01:55.824 7f3ee4335108 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/18/2025 02:02:00.853 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1408-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13234608, partition_space_available_1k=67855876, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1424092, cpu_share=0.05, mem_res=1359632, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12744, cpu_share=0.0, mem_res=7988, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54668, cpu_share=0.01, mem_res=40472, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184204, used=7021404, free=245712, buffers=457596, cached=8459492, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.49, five_min_avg=2.45, fifteen_min_avg=2.12, cpu_share=8.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 08/18/2025 02:02:05.693 7f3ee3e07978 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 08/18/2025 02:02:53.925 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 08/18/2025 02:02:55.448 7f3ee4335108 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/18/2025 02:03:00.478 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1408-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13249100, partition_space_available_1k=67841384, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1421884, cpu_share=0.06, mem_res=1353944, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12748, cpu_share=0.0, mem_res=7988, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54580, cpu_share=0.01, mem_res=40456, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184204, used=6895248, free=346044, buffers=460000, cached=8482912, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.52, five_min_avg=2.31, fifteen_min_avg=2.08, cpu_share=9.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 08/18/2025 02:03:53.235 7f3ee4145438 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 08/18/2025 02:03:54.351 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 08/18/2025 02:03:56.565 7f3ee4335108 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/18/2025 02:04:01.595 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1408-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13613156, partition_space_available_1k=67477328, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1425176, cpu_share=0.05, mem_res=1358320, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12724, cpu_share=0.0, mem_res=7988, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54352, cpu_share=0.01, mem_res=40460, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184204, used=7002200, free=343964, buffers=467904, cached=8370136, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.42, five_min_avg=2.41, fifteen_min_avg=2.13, cpu_share=12.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 08/18/2025 02:04:20.384 7f3ee40f6048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 08/18/2025 02:04:46.282 7f3ee0979b70 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 08/18/2025 02:04:53.914 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 08/18/2025 02:04:55.764 7f3ee4335108 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/18/2025 02:05:00.801 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1408-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13624696, partition_space_available_1k=67465788, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1429036, cpu_share=0.05, mem_res=1360640, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12756, cpu_share=0.0, mem_res=7988, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54640, cpu_share=0.01, mem_res=40640, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184204, used=7119628, free=262560, buffers=467392, cached=8334624, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.75, five_min_avg=2.22, fifteen_min_avg=2.08, cpu_share=10.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 08/18/2025 02:05:12.636 7f3ee3e322e8 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 08/18/2025 02:05:16.466 7f3ee3e322e8 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 08/18/2025 02:05:18.948 7f3ee42de978 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 08/18/2025 02:05:22.895 7f3ee42de978 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 08/18/2025 02:05:53.912 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 08/18/2025 02:05:55.921 7f3ee4335108 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 08/18/2025 02:06:00.946 7f3ee4335108 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1408-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13638312, partition_space_available_1k=67452172, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1433992, cpu_share=0.04, mem_res=1362864, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12824, cpu_share=0.0, mem_res=8120, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54644, cpu_share=0.01, mem_res=40536, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184204, used=6977752, free=388160, buffers=467652, cached=8350640, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.45, five_min_avg=2.39, fifteen_min_avg=2.15, cpu_share=9.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 08/18/2025 02:06:34.403 7f3ee18e7e10 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map