09/29/2025 01:43:51.183 7fefe08d9380 [contrail-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/29/2025 01:43:51.184 7fefe08d9380 [contrail-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/29/2025 01:43:51.184 7fefe08d9380 [contrail-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/29/2025 01:43:51.192 7fefdd6f34a0 [contrail-database-nodemgr] [ERROR]: Cannot write http_port 8103 to /tmp/contrail-database-nodemgr.0.http_port 09/29/2025 01:43:51.192 7fefdd6f34a0 [contrail-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8103 09/29/2025 01:43:51.193 7fefdd6f3860 [contrail-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 09/29/2025 01:43:51.193 7fefdd6f3860 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/29/2025 01:43:51.194 7fefdd6f3860 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/29/2025 01:43:51.194 7fefdd6f3860 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/29/2025 01:43:51.194 7fefdd6f3860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:51.194 7fefdd6f3860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4202-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/29/2025 01:43:51.194 7fefdd6f3860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:51.194 7fefdd6f3860 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:51.194 7fefdd6f3860 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4202-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1759110231194200 collector_name = collector_ip = collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/29/2025 01:43:51.195 7fefdd6f3860 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:51.331 7fefe08d9380 [contrail-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 69200343766812831885718585883197679658295772779914675747694822256931729728192, 'start': '1759110229000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-query-engine', 'group': 'contrail-query-engine', 'pid': 66633146277842171171101070772271310993731638536713825974435205365675833128564, 'start': '1759110229000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-database-nodemgr', 'group': 'contrail-database-nodemgr', 'pid': 114927652970281445523127202015932550291013692839898686920012393711546094664122, 'start': '1759110229000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 09/29/2025 01:43:51.399 7fefdd6f3900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:51.399 7fefdd6f3900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4202-1. build_info = {"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-29 00:11:25.017323", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11319-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed "}]} system_cpu_info = << num_socket = 1 num_cpu = 8 num_core_per_socket = 8 num_thread_per_core = 1 >> running_package_version = package contrail-lib is not installed installed_package_version = package contrail-lib is not installed >> 09/29/2025 01:43:51.399 7fefdd6f3900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:51.400 7fefe08d9380 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-4202-1.', _node_type='Database', _instance_id='0', _timestamp=1759110231400142, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4202-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 09/29/2025 01:43:51.400 7fefe08d9380 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-4202-1.', _node_type='Database', _instance_id='0', _timestamp=1759110231400468, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4202-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1759110229000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1759110229000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1759110229000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-29 00:11:25.017323", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11319-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed\n"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 09/29/2025 01:43:51.402 7fefdd6f3900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:51.402 7fefdd6f3900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4202-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/29/2025 01:43:51.403 7fefdd6f3900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:51.403 7fefdd6f3900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:51.403 7fefdd6f3900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4202-1. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1759110229000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-query-engine process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1759110229000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1759110229000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-29 00:11:25.017323", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11319-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed "}]} >> 09/29/2025 01:43:51.403 7fefdd6f3900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:51.696 7fefe08d9380 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/29/2025 01:43:52.010 7fefe08d9380 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:DatabaseUsage] map 09/29/2025 01:43:52.015 7fefdd6f3900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:52.015 7fefdd6f3900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-4202-1. database_usage = [ << disk_space_used_1k = 16534884 disk_space_available_1k = 64555600 analytics_db_size_1k = 108 >>, ] >> 09/29/2025 01:43:52.016 7fefdd6f3900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:52.162 7fefe08d9380 [contrail-database-nodemgr] [ERROR]: Failed to connect to database by CQL and socket: Result: 1 09/29/2025 01:43:52.162 7fefe08d9380 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-4202-1.', _node_type='Database', _instance_id='0', _timestamp=1759110232162357, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4202-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 09/29/2025 01:43:52.163 7fefdd6c3900 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:52.164 7fefdd6c3900 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4202-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/29/2025 01:43:52.164 7fefdd6c3900 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:53.031 7fefe08d9380 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/29/2025 01:43:54.048 7fefe08d9380 [contrail-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7200 repair -pr ContrailAnalyticsCql'. Error is: Result: 1 09/29/2025 01:43:55.194 7fefdd6c32c0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:43:55.196 7fefdd6c32c0 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:43:55.196 7fefdd6c32c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:43:55.196 7fefdd6c32c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:43:55.196 7fefdd6c32c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4202-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:43:55.196 7fefdd6c32c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:43:55.197 7fefdd6c32c0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:43:55.197 7fefdd6c32c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:43:55.197 7fefdd6c32c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:55.197 7fefdd6c32c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4202-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:43:55.197 7fefdd6c32c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:55.197 7fefdd6c32c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:55.198 7fefdd6c32c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4202-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1759110231194200 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:43:55.198 7fefdd6c32c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:59.081 7fefe08d9380 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-4202-1.', _node_type='Database', _instance_id='0', _timestamp=1759110239081581, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4202-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1759110229000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1759110229000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1759110229000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-29 00:11:25.017323", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11319-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed\n"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 09/29/2025 01:43:59.082 7fefe08d9380 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4202-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=16581852, partition_space_available_1k=64508632, percentage_partition_space_used=20, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1273168, cpu_share=0.0, mem_res=1263036, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=7996, cpu_share=0.0, mem_res=6024, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52096, cpu_share=0.0, mem_res=42640, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=8373672, free=7198696, buffers=664984, cached=16900116, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.66, five_min_avg=2.12, fifteen_min_avg=1.33, cpu_share=0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/29/2025 01:43:59.082 7fefdd6c3c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:59.082 7fefdd6c3c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4202-1. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1759110229000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-query-engine process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1759110229000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1759110229000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-29 00:11:25.017323", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11319-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed "}]} >> 09/29/2025 01:43:59.082 7fefdd6c3c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:59.083 7fefdd6c3c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:59.083 7fefdd6c3c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4202-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 16581852 partition_space_available_1k = 64508632 percentage_partition_space_used = 20 >> } all_core_file_list = [ ] process_mem_cpu_usage = { cassandra : << mem_virt = 1273168 cpu_share = 0.0 mem_res = 1263036 >>contrail-query-engine : << mem_virt = 7996 cpu_share = 0.0 mem_res = 6024 >>contrail-database-nodemgr : << mem_virt = 52096 cpu_share = 0.0 mem_res = 42640 >> } system_mem_usage = << total = 32665120 used = 8373672 free = 7198696 buffers = 664984 cached = 16900116 node_type = database-node >> system_cpu_usage = << one_min_avg = 2.66 five_min_avg = 2.12 fifteen_min_avg = 1.33 cpu_share = 0 node_type = database-node >> >> 09/29/2025 01:43:59.083 7fefdd6c3c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:59.196 7fefdd6c3c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:43:59.197 7fefdd6c3c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:43:59.198 7fefdd6c3c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:43:59.198 7fefdd6c3c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:43:59.198 7fefdd6c3c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4202-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:43:59.198 7fefdd6c3c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:43:59.198 7fefdd6c3c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:43:59.198 7fefdd6c3c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:43:59.198 7fefdd6c3c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:59.198 7fefdd6c3c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4202-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:43:59.199 7fefdd6c3c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:59.199 7fefdd6c3c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:59.199 7fefdd6c3c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4202-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1759110231194200 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:43:59.199 7fefdd6c3c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:03.201 7fefdd6c3c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:44:03.202 7fefdd6c3c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:44:03.202 7fefdd6c3c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:44:03.203 7fefdd6c3c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:44:03.203 7fefdd6c3c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4202-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:44:03.203 7fefdd6c3c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:44:03.203 7fefdd6c3c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:44:03.203 7fefdd6c3c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:44:03.203 7fefdd6c3c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:03.203 7fefdd6c3c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4202-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:44:03.203 7fefdd6c3c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:03.204 7fefdd6c3c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:03.204 7fefdd6c3c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4202-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1759110231194200 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:44:03.204 7fefdd6c3c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:07.203 7fefdd6c3c20 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:44:07.204 7fefdd6c3c20 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:44:07.204 7fefdd6c3c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:44:07.204 7fefdd6c3c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:44:07.205 7fefdd6c3c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4202-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:44:07.205 7fefdd6c3c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:44:07.205 7fefdd6c3c20 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:44:07.205 7fefdd6c3c20 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:44:07.205 7fefdd6c3c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:07.205 7fefdd6c3c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4202-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:44:07.205 7fefdd6c3c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:07.205 7fefdd6c3c20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:07.205 7fefdd6c3c20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4202-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1759110231194200 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:44:07.205 7fefdd6c3c20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:11.208 7fefdd6f3b80 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:44:11.209 7fefdd6f3b80 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:44:11.209 7fefdd6f3b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:44:11.209 7fefdd6f3b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:44:11.209 7fefdd6f3b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4202-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:44:11.209 7fefdd6f3b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:44:11.209 7fefdd6f3b80 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:44:11.210 7fefdd6f3b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:44:11.210 7fefdd6f3b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:11.210 7fefdd6f3b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4202-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:44:11.210 7fefdd6f3b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:11.210 7fefdd6f3b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:11.210 7fefdd6f3b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4202-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1759110231194200 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:44:11.210 7fefdd6f3b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:15.213 7fefdd6f3b80 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:44:15.214 7fefdd6f3b80 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:44:15.214 7fefdd6f3b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:44:15.215 7fefdd6f3b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:44:15.215 7fefdd6f3b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4202-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:44:15.215 7fefdd6f3b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:44:15.215 7fefdd6f3b80 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:44:15.215 7fefdd6f3b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:44:15.215 7fefdd6f3b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:15.215 7fefdd6f3b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4202-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:44:15.215 7fefdd6f3b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:15.215 7fefdd6f3b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:15.216 7fefdd6f3b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4202-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1759110231194200 collector_name = collector_ip = 10.0.0.130:8086 collector_list = [ 10.0.0.130:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:44:15.216 7fefdd6f3b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:19.218 7fefdd6f3b80 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:44:19.219 7fefdd6f3b80 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 09/29/2025 01:44:19.219 7fefdd6f3b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:44:19.220 7fefdd6f3b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:44:19.220 7fefdd6f3b80 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4202-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.130:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:44:19.220 7fefdd6f3b80 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:44:19.220 7fefdd6f3b80 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/29/2025 01:44:19.220 7fefdd6f3b80 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/29/2025 01:44:19.220 7fefdd6f3b80 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/29/2025 01:44:19.220 7fefdd6f3b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/29/2025 01:44:19.220 7fefdd6f3b80 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/29/2025 01:44:27.504 7fefdd6c37c0 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001016 09/29/2025 01:44:27.505 7fefdd6c37c0 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 09/29/2025 01:44:27.505 7fefde3135e0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 09/29/2025 01:44:27.505 7fefde3135e0 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 09/29/2025 01:44:27.506 7fefde3135e0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 09/29/2025 01:44:27.506 7fefde3135e0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/29/2025 01:44:27.506 7fefde3135e0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/29/2025 01:44:27.506 7fefde3135e0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 09/29/2025 01:44:27.507 7fefdd6c3c20 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/29/2025 01:44:27.507 7fefdd6c3c20 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/29/2025 01:44:38.066 7fefdd6c37c0 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001437 09/29/2025 01:44:38.067 7fefdd6c37c0 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 09/29/2025 01:44:38.067 7fefdd6c3360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 09/29/2025 01:44:38.067 7fefdd6c3360 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 3 09/29/2025 01:44:38.067 7fefdd6c3360 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 09/29/2025 01:44:38.067 7fefdd6c3360 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/29/2025 01:44:38.067 7fefdd6c3360 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/29/2025 01:44:38.068 7fefde313040 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/29/2025 01:44:38.068 7fefde313040 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/29/2025 01:44:51.687 7fefe08d9380 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/29/2025 01:44:52.404 7fefe08d9380 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/29/2025 01:44:52.404 7fefe08d9380 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-4202-1.', _node_type='Database', _instance_id='0', _timestamp=1759110292404627, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4202-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.130:8086'], status='Up', description='ClientInit to Established on EvSandeshCtrlMessageRecv')]), flag_infos=None, description='')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 09/29/2025 01:44:53.227 7fefe08d9380 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/29/2025 01:44:58.330 7fefe08d9380 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4202-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=17919520, partition_space_available_1k=63170964, percentage_partition_space_used=22, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1338548, cpu_share=0.27, mem_res=1322992, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11496, cpu_share=0.0, mem_res=9072, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52536, cpu_share=0.02, mem_res=43048, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9926024, free=3980068, buffers=696584, cached=18535664, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.92, five_min_avg=2.61, fifteen_min_avg=1.56, cpu_share=3.88, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/29/2025 01:45:51.698 7fefe08d9380 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/29/2025 01:45:52.301 7fefe08d9380 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/29/2025 01:45:53.254 7fefe08d9380 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/29/2025 01:45:58.434 7fefe08d9380 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4202-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19673588, partition_space_available_1k=61416896, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1335964, cpu_share=0.07, mem_res=1322504, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11440, cpu_share=0.0, mem_res=9388, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45112, cpu_share=0.02, mem_res=43048, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9271056, free=18511652, buffers=707276, cached=4648656, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.57, five_min_avg=3.01, fifteen_min_avg=1.76, cpu_share=5.38, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/29/2025 01:46:30.833 7fefdd6f3900 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/29/2025 01:46:32.884 7fefdd6f3900 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/29/2025 01:46:51.699 7fefe08d9380 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/29/2025 01:46:52.224 7fefe08d9380 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/29/2025 01:46:53.013 7fefe08d9380 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/29/2025 01:46:58.337 7fefe08d9380 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4202-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19876488, partition_space_available_1k=61213996, percentage_partition_space_used=25, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1338020, cpu_share=0.06, mem_res=1322152, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11692, cpu_share=0.0, mem_res=9640, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45152, cpu_share=0.02, mem_res=43048, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=10078672, free=17268096, buffers=716576, cached=5074860, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.76, five_min_avg=2.8, fifteen_min_avg=1.77, cpu_share=3.38, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/29/2025 01:47:42.079 7fefde3135e0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/29/2025 01:47:51.692 7fefe08d9380 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/29/2025 01:47:52.232 7fefe08d9380 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/29/2025 01:47:53.039 7fefe08d9380 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/29/2025 01:47:58.367 7fefe08d9380 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4202-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21172116, partition_space_available_1k=59918368, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1340604, cpu_share=0.07, mem_res=1322536, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=14104, cpu_share=0.0, mem_res=12008, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45156, cpu_share=0.02, mem_res=43048, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=10628932, free=15266124, buffers=751344, cached=6490872, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.85, five_min_avg=2.8, fifteen_min_avg=1.83, cpu_share=3.88, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/29/2025 01:48:51.699 7fefe08d9380 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/29/2025 01:48:52.283 7fefe08d9380 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/29/2025 01:48:53.260 7fefe08d9380 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/29/2025 01:48:58.450 7fefe08d9380 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4202-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21206892, partition_space_available_1k=59883592, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1380984, cpu_share=0.16, mem_res=1355500, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=16552, cpu_share=0.0, mem_res=14412, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45192, cpu_share=0.02, mem_res=43048, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=10737980, free=15114128, buffers=752076, cached=6533132, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.03, five_min_avg=2.87, fifteen_min_avg=1.92, cpu_share=4.38, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/29/2025 01:49:51.699 7fefe08d9380 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/29/2025 01:49:52.236 7fefe08d9380 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/29/2025 01:49:52.990 7fefe08d9380 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/29/2025 01:49:58.356 7fefe08d9380 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4202-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21324892, partition_space_available_1k=59765592, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1385468, cpu_share=0.08, mem_res=1356240, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=16564, cpu_share=0.0, mem_res=14424, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45196, cpu_share=0.02, mem_res=43048, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=11422552, free=14299224, buffers=752160, cached=6663560, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.72, five_min_avg=3.13, fifteen_min_avg=2.07, cpu_share=5.12, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/29/2025 01:50:51.722 7fefe08d9380 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/29/2025 01:50:52.309 7fefe08d9380 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/29/2025 01:50:53.130 7fefe08d9380 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/29/2025 01:50:58.441 7fefe08d9380 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4202-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21356072, partition_space_available_1k=59734412, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1391416, cpu_share=0.12, mem_res=1356804, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=16488, cpu_share=0.0, mem_res=14348, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45200, cpu_share=0.02, mem_res=43048, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=11002080, free=14667048, buffers=752552, cached=6715780, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.7, five_min_avg=3.83, fifteen_min_avg=2.37, cpu_share=4.62, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/29/2025 01:51:51.715 7fefe08d9380 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/29/2025 01:51:52.272 7fefe08d9380 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/29/2025 01:51:53.086 7fefe08d9380 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/29/2025 01:51:58.424 7fefe08d9380 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4202-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21384372, partition_space_available_1k=59706112, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1398104, cpu_share=0.08, mem_res=1359596, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=16580, cpu_share=0.0, mem_res=14440, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45208, cpu_share=0.02, mem_res=43048, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=10914924, free=14728528, buffers=752776, cached=6740904, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.24, five_min_avg=3.64, fifteen_min_avg=2.39, cpu_share=4.38, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/29/2025 01:52:51.734 7fefe08d9380 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/29/2025 01:52:52.329 7fefe08d9380 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/29/2025 01:52:53.241 7fefe08d9380 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/29/2025 01:52:58.338 7fefe08d9380 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4202-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21475184, partition_space_available_1k=59615300, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1403096, cpu_share=0.07, mem_res=1360892, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=16644, cpu_share=0.0, mem_res=14504, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45208, cpu_share=0.02, mem_res=43048, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=12015520, free=13577840, buffers=752844, cached=6790132, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.42, five_min_avg=3.84, fifteen_min_avg=2.54, cpu_share=4.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/29/2025 01:53:51.760 7fefe08d9380 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/29/2025 01:53:52.352 7fefe08d9380 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/29/2025 01:53:53.207 7fefe08d9380 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/29/2025 01:53:58.396 7fefe08d9380 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4202-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21469724, partition_space_available_1k=59620760, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1408816, cpu_share=0.06, mem_res=1363160, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=16740, cpu_share=0.0, mem_res=14600, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45228, cpu_share=0.02, mem_res=43048, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=10977220, free=14587516, buffers=752920, cached=6818820, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.06, five_min_avg=3.55, fifteen_min_avg=2.52, cpu_share=3.62, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/29/2025 01:54:51.747 7fefe08d9380 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/29/2025 01:54:52.259 7fefe08d9380 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/29/2025 01:54:53.059 7fefe08d9380 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/29/2025 01:54:58.394 7fefe08d9380 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4202-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21495808, partition_space_available_1k=59594676, percentage_partition_space_used=27, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1418108, cpu_share=0.07, mem_res=1369048, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=16744, cpu_share=0.0, mem_res=14604, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45224, cpu_share=0.02, mem_res=43048, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=11275508, free=14261336, buffers=752956, cached=6846772, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.08, five_min_avg=3.79, fifteen_min_avg=2.67, cpu_share=4.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/29/2025 01:55:51.739 7fefe08d9380 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/29/2025 01:55:52.299 7fefe08d9380 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/29/2025 01:55:53.124 7fefe08d9380 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/29/2025 01:55:58.754 7fefe08d9380 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4202-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21605408, partition_space_available_1k=59485076, percentage_partition_space_used=27, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1422560, cpu_share=0.06, mem_res=1370528, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=16824, cpu_share=0.0, mem_res=14684, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45220, cpu_share=0.02, mem_res=43048, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=10987264, free=14435308, buffers=753044, cached=6959952, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.75, five_min_avg=3.8, fifteen_min_avg=2.75, cpu_share=4.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/29/2025 01:56:51.755 7fefe08d9380 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/29/2025 01:56:52.272 7fefe08d9380 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/29/2025 01:56:53.018 7fefe08d9380 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/29/2025 01:56:58.415 7fefe08d9380 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4202-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21616660, partition_space_available_1k=59473824, percentage_partition_space_used=27, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1427120, cpu_share=0.06, mem_res=1372272, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=16896, cpu_share=0.0, mem_res=14756, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45228, cpu_share=0.02, mem_res=43048, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=11002544, free=14409228, buffers=753080, cached=6970764, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.77, five_min_avg=3.53, fifteen_min_avg=2.72, cpu_share=2.88, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/29/2025 01:57:40.657 7fefdd6a1e00 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 09/29/2025 01:57:51.751 7fefe08d9380 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 09/29/2025 01:57:52.259 7fefe08d9380 [contrail-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/29/2025 01:57:53.017 7fefe08d9380 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/29/2025 01:57:58.521 7fefe08d9380 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4202-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21629104, partition_space_available_1k=59461380, percentage_partition_space_used=27, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1432860, cpu_share=0.07, mem_res=1374684, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=17076, cpu_share=0.0, mem_res=14936, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45228, cpu_share=0.02, mem_res=43048, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=10817416, free=14537464, buffers=753144, cached=7027460, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.79, five_min_avg=3.61, fifteen_min_avg=2.8, cpu_share=3.12, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 09/29/2025 01:58:29.702 7fefdd6f3c20 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map