03/23/2026 05:37:13.223 7f60e6410df0 [contrail-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 03/23/2026 05:37:13.224 7f60e6410df0 [contrail-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 03/23/2026 05:37:13.224 7f60e6410df0 [contrail-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 03/23/2026 05:37:13.233 7f60e0c83548 [contrail-database-nodemgr] [ERROR]: Cannot write http_port 8103 to /tmp/contrail-database-nodemgr.0.http_port 03/23/2026 05:37:13.234 7f60e0c83548 [contrail-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8103 03/23/2026 05:37:13.235 7f60e0c83d48 [contrail-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 03/23/2026 05:37:13.236 7f60e0c83d48 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/23/2026 05:37:13.237 7f60e0c83d48 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/23/2026 05:37:13.237 7f60e0c83d48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 03/23/2026 05:37:13.238 7f60e0c83d48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:13.238 7f60e0c83d48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/23/2026 05:37:13.238 7f60e0c83d48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:13.238 7f60e0c83d48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:13.238 7f60e0c83d48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1774244233237461 collector_name = collector_ip = collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 03/23/2026 05:37:13.239 7f60e0c83d48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:13.484 7f60e6410df0 [contrail-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 99430542724699331709479873453972484490082645354536070244161137143763100583449, 'start': '1774244230000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-query-engine', 'group': 'contrail-query-engine', 'pid': 11057208290536007534398102479692903237556336124444338531564331186445089128524, 'start': '1774244230000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-database-nodemgr', 'group': 'contrail-database-nodemgr', 'pid': 18492363983684627165209990134390594649914656199270997456059889717054613094126, 'start': '1774244230000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 03/23/2026 05:37:13.661 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:13.662 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. build_info = {"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-10-06 01:00:31.164147", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11344-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 4 num_core_per_socket = 4 num_thread_per_core = 1 >> running_package_version = v0+nightly.r24.1.centos-nightly_r24.1_centos.el7 installed_package_version = v0+nightly.r24.1.centos-nightly_r24.1_centos.el7 >> 03/23/2026 05:37:13.662 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:13.662 7f60e6410df0 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-1.', _node_type='Database', _instance_id='0', _timestamp=1774244233662682, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 03/23/2026 05:37:13.663 7f60e6410df0 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-1.', _node_type='Database', _instance_id='0', _timestamp=1774244233662943, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774244230000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774244230000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774244230000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-10-06 01:00:31.164147", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11344-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 03/23/2026 05:37:13.665 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:13.665 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/23/2026 05:37:13.665 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:13.665 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:13.665 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774244230000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-query-engine process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774244230000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774244230000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-10-06 01:00:31.164147", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11344-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} >> 03/23/2026 05:37:13.665 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:14.030 7f60e6410df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:37:14.508 7f60e6410df0 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:DatabaseUsage] map 03/23/2026 05:37:14.510 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:14.510 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. database_usage = [ << disk_space_used_1k = 13808988 disk_space_available_1k = 67281496 analytics_db_size_1k = 4 >>, ] >> 03/23/2026 05:37:14.510 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:15.292 7f60e6410df0 [contrail-database-nodemgr] [ERROR]: Failed to connect to database by CQL: Result: 1 03/23/2026 05:37:15.292 7f60e6410df0 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-1.', _node_type='Database', _instance_id='0', _timestamp=1774244235292650, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 03/23/2026 05:37:15.300 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:15.300 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/23/2026 05:37:15.300 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:17.236 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:37:17.237 7f60e089a248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:37:17.237 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:37:17.237 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:17.237 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:37:17.237 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:17.237 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:37:17.238 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:37:17.238 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:17.238 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:37:17.238 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:17.238 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:17.238 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1774244233237461 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:37:17.238 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:19.591 7f60e6410df0 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-1.', _node_type='Database', _instance_id='0', _timestamp=1774244239591794, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0)], _table='ObjectDatabaseInfo'), _seqnum=0) 03/23/2026 05:37:19.592 7f60e6410df0 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 03/23/2026 05:37:19.594 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:19.596 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] CassandraStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. cassandra_compaction_task = << pending_compaction_tasks = 0 >> thread_pool_stats = [ << pool_name = ReadStage active = 0 pending = 0 all_time_blocked = 0 >>, << pool_name = CompactionExecutor active = 0 pending = 0 all_time_blocked = 0 >>, << pool_name = MutationStage active = 0 pending = 0 all_time_blocked = 0 >>, << pool_name = MemtableFlushWriter active = 0 pending = 0 all_time_blocked = 0 >>, ] >> 03/23/2026 05:37:19.596 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:21.238 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:37:21.239 7f60e089a248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:37:21.240 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:37:21.240 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:21.240 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:37:21.240 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:21.240 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:37:21.240 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:37:21.240 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:21.241 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:37:21.241 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:21.241 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:21.241 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1774244233237461 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:37:21.241 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:24.621 7f60e6410df0 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-1.', _node_type='Database', _instance_id='0', _timestamp=1774244244621458, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774244230000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774244230000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774244230000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-10-06 01:00:31.164147", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11344-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 03/23/2026 05:37:24.621 7f60e6410df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13810072, partition_space_available_1k=67280412, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1255088, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=7700, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52944, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=7862244, free=367280, buffers=665896, cached=7667136, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.33, five_min_avg=3.04, fifteen_min_avg=1.69, cpu_share=0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:37:24.622 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:24.622 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774244230000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-query-engine process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774244230000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774244230000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-10-06 01:00:31.164147", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11344-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} >> 03/23/2026 05:37:24.622 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:24.622 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:24.622 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 13810072 partition_space_available_1k = 67280412 percentage_partition_space_used = 17 >> } all_core_file_list = [ ] process_mem_cpu_usage = { cassandra : << mem_virt = 1255088 cpu_share = 0 mem_res = 0 >>contrail-query-engine : << mem_virt = 7700 cpu_share = 0 mem_res = 0 >>contrail-database-nodemgr : << mem_virt = 52944 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175640 used = 7862244 free = 367280 buffers = 665896 cached = 7667136 node_type = database-node >> system_cpu_usage = << one_min_avg = 4.33 five_min_avg = 3.04 fifteen_min_avg = 1.69 cpu_share = 0 node_type = database-node >> >> 03/23/2026 05:37:24.622 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:25.240 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:37:25.241 7f60e089a248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:37:25.241 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:37:25.241 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:25.241 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:37:25.242 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:25.242 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:37:25.242 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:37:25.242 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:25.242 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:37:25.242 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:25.242 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:25.242 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1774244233237461 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:37:25.242 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:29.242 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:37:29.243 7f60e089a248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:37:29.243 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:37:29.243 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:29.243 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:37:29.243 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:29.243 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:37:29.244 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:37:29.244 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:29.244 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:37:29.244 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:29.244 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:29.244 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1774244233237461 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:37:29.244 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:33.244 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:37:33.245 7f60e089a248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:37:33.245 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:37:33.245 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:33.246 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:37:33.246 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:33.246 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:37:33.246 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:37:33.246 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:33.246 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:37:33.246 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:33.246 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:33.247 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1774244233237461 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:37:33.247 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:37.249 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:37:37.250 7f60e089a248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:37:37.250 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:37:37.250 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:37.251 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:37:37.251 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:37.251 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:37:37.251 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:37:37.251 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:37.251 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:37:37.251 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:37.251 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:37.252 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1774244233237461 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:37:37.252 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:41.253 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:37:41.254 7f60e089a248 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 03/23/2026 05:37:41.255 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:37:41.255 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:41.255 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:37:41.255 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:41.255 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 03/23/2026 05:37:41.259 7f60e089a248 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 03/23/2026 05:37:41.259 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 03/23/2026 05:37:41.259 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:37:41.260 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:37:50.485 7f60e0d5f248 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001016 03/23/2026 05:37:50.486 7f60e0d5f248 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 03/23/2026 05:37:50.486 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 03/23/2026 05:37:50.486 7f60e089a248 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 03/23/2026 05:37:50.486 7f60e089a248 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 03/23/2026 05:37:50.486 7f60e089a248 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 03/23/2026 05:37:50.487 7f60e089a248 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/23/2026 05:37:50.487 7f60e089a248 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/23/2026 05:37:50.487 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 03/23/2026 05:37:50.487 7f60e0c83e48 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 03/23/2026 05:37:50.489 7f60e0c83e48 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 03/23/2026 05:38:00.822 7f60e0d5f248 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001581 03/23/2026 05:38:00.822 7f60e0d5f248 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 03/23/2026 05:38:00.823 7f60e0c83e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 03/23/2026 05:38:00.824 7f60e0c83e48 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 4 03/23/2026 05:38:00.828 7f60e0c83e48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 03/23/2026 05:38:00.829 7f60e0c83e48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 03/23/2026 05:38:00.829 7f60e0c83e48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/23/2026 05:38:00.830 7f60e0c83e48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/23/2026 05:38:00.830 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 03/23/2026 05:38:00.831 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 03/23/2026 05:38:14.090 7f60e6410df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:38:15.774 7f60e6410df0 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-1.', _node_type='Database', _instance_id='0', _timestamp=1774244295774440, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.38:8086'], status='Up', description='ClientInit to Established on EvSandeshCtrlMessageRecv')]), flag_infos=None, description='')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 03/23/2026 05:38:18.925 7f60e6410df0 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-1.', _node_type='Database', _instance_id='0', _timestamp=1774244298925088, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=0, pending=0, all_time_blocked=1)], _table='ObjectDatabaseInfo'), _seqnum=0) 03/23/2026 05:38:23.948 7f60e6410df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14233084, partition_space_available_1k=66857400, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1343396, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=9804, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52536, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=9008260, free=368124, buffers=680096, cached=6507028, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.26, five_min_avg=3.76, fifteen_min_avg=2.01, cpu_share=18.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:39:14.009 7f60e6410df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:39:23.513 7f60e6410df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14313260, partition_space_available_1k=66777224, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1337332, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=9884, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52560, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=8874248, free=467088, buffers=680216, cached=6541688, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.57, five_min_avg=3.72, fifteen_min_avg=2.11, cpu_share=15.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:40:13.986 7f60e6410df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:40:22.860 7f60e6410df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14339716, partition_space_available_1k=66750768, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1338736, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=9964, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52576, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=8918776, free=392160, buffers=680464, cached=6571820, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.12, five_min_avg=3.73, fifteen_min_avg=2.21, cpu_share=14.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:40:56.610 7f60e0c83e48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/23/2026 05:41:13.997 7f60e6410df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:41:18.808 7f60e089a248 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/23/2026 05:41:23.272 7f60e6410df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14800004, partition_space_available_1k=66290480, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1377444, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10248, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52628, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=8939500, free=413712, buffers=684984, cached=6525084, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.01, five_min_avg=3.74, fifteen_min_avg=2.31, cpu_share=15.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:42:14.130 7f60e6410df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:42:24.920 7f60e6410df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15882676, partition_space_available_1k=65207808, percentage_partition_space_used=20, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1348064, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10328, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52364, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=9171356, free=294064, buffers=751876, cached=6346320, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.34, five_min_avg=4.16, fifteen_min_avg=2.55, cpu_share=17.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:43:14.074 7f60e6410df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:43:22.941 7f60e6410df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=16033992, partition_space_available_1k=65056492, percentage_partition_space_used=20, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1353816, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11024, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52216, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=9358364, free=324072, buffers=759380, cached=6127784, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.94, five_min_avg=4.81, fifteen_min_avg=2.87, cpu_share=18.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:44:14.232 7f60e6410df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:44:27.869 7f60e6410df0 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-1.', _node_type='Database', _instance_id='0', _timestamp=1774244667869677, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=0, pending=0, all_time_blocked=12)], _table='ObjectDatabaseInfo'), _seqnum=0) 03/23/2026 05:44:33.528 7f60e6410df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=17211752, partition_space_available_1k=63878732, percentage_partition_space_used=21, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1380276, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11036, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51700, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=9509352, free=407520, buffers=763108, cached=5887320, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=10.99, five_min_avg=6.11, fifteen_min_avg=3.44, cpu_share=19.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:45:14.113 7f60e6410df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:45:23.863 7f60e6410df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=18193528, partition_space_available_1k=62896956, percentage_partition_space_used=22, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1383496, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11092, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51428, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=9473340, free=332092, buffers=770292, cached=5987488, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=9.19, five_min_avg=6.48, fifteen_min_avg=3.71, cpu_share=19.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:45:55.892 7f60e0d5f248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 03/23/2026 05:45:55.892 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Established] 03/23/2026 05:45:55.894 7f60e089a248 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 03/23/2026 05:45:55.895 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[Established] -> State[Connect] 03/23/2026 05:45:55.895 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:45:55.895 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Established to Connect on EvTcpClose >>, ] description = Collector:Collector[Established to Connect on EvTcpClose] connection down >>, ] >> 03/23/2026 05:45:55.895 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:45:55.895 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 03/23/2026 05:45:55.895 7f60e089a248 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 03/23/2026 05:45:55.896 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 03/23/2026 05:45:55.918 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:45:55.920 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:45:55.961 7f60e0d5f248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 03/23/2026 05:45:55.961 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit] 03/23/2026 05:45:55.962 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle] 03/23/2026 05:45:55.962 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:45:55.962 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = ClientInit to Idle on EvTcpClose >>, ] description = Collector:Collector[ClientInit to Idle on EvTcpClose] connection down >>, ] >> 03/23/2026 05:45:55.962 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:45:55.962 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:45:55.962 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 2 pid = 1 http_port = 8103 start_time = 1774244233237461 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/23/2026 05:45:55.962 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:45:59.964 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:45:59.965 7f60e089a248 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 03/23/2026 05:45:59.965 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:45:59.965 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:45:59.965 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:45:59.965 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:45:59.966 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 03/23/2026 05:45:59.966 7f60e089a248 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 03/23/2026 05:45:59.966 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 03/23/2026 05:45:59.966 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:45:59.966 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:45:59.970 7f60e0d5f248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 03/23/2026 05:45:59.970 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit] 03/23/2026 05:45:59.971 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle] 03/23/2026 05:45:59.971 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:45:59.971 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = ClientInit to Idle on EvTcpClose >>, ] description = Collector:Collector[ClientInit to Idle on EvTcpClose] connection down >>, ] >> 03/23/2026 05:45:59.971 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:45:59.971 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:45:59.971 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 3 pid = 1 http_port = 8103 start_time = 1774244233237461 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/23/2026 05:45:59.971 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:03.974 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:03.975 7f60e089a248 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 03/23/2026 05:46:03.975 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:03.975 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:03.975 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:03.975 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:03.975 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 03/23/2026 05:46:03.976 7f60e089a248 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 03/23/2026 05:46:03.976 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 03/23/2026 05:46:03.976 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:46:03.976 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:46:07.618 7f60e0d5f248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 03/23/2026 05:46:07.619 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit] 03/23/2026 05:46:07.619 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle] 03/23/2026 05:46:07.619 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:07.619 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = ClientInit to Idle on EvTcpClose >>, ] description = Collector:Collector[ClientInit to Idle on EvTcpClose] connection down >>, ] >> 03/23/2026 05:46:07.620 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:07.620 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:07.620 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1774244233237461 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/23/2026 05:46:07.620 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:11.622 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:11.623 7f60e089a248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:46:11.623 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:11.623 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:11.623 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:11.623 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:11.623 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:46:11.624 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:46:11.624 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:11.624 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:46:11.624 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:11.624 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:11.625 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1774244233237461 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/23/2026 05:46:11.625 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:14.007 7f60e6410df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:46:14.413 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:14.413 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. database_usage = [ << disk_space_used_1k = 18771100 disk_space_available_1k = 62319384 analytics_db_size_1k = 1580 >>, ] >> 03/23/2026 05:46:14.413 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:15.623 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:15.626 7f60e089a248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:46:15.626 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:15.626 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:15.626 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:15.626 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:15.626 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:46:15.626 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:46:15.627 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:15.627 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:46:15.627 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:15.627 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:15.627 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1774244233237461 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/23/2026 05:46:15.627 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:19.626 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:19.628 7f60e089a248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:46:19.628 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:19.628 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:19.629 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:19.629 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:19.629 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:46:19.630 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:46:19.630 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:19.631 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:46:19.631 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:19.632 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:19.632 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1774244233237461 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/23/2026 05:46:19.633 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:22.213 7f60e6410df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=18771116, partition_space_available_1k=62319368, percentage_partition_space_used=23, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1395228, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11696, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51424, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=7656108, free=2253448, buffers=772788, cached=5879844, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.3, five_min_avg=6.14, fifteen_min_avg=3.77, cpu_share=15.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:46:22.224 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:22.225 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 18771116 partition_space_available_1k = 62319368 percentage_partition_space_used = 23 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1395228 cpu_share = 0 mem_res = 0 >>contrail-query-engine : << mem_virt = 11696 cpu_share = 0 mem_res = 0 >>contrail-database-nodemgr : << mem_virt = 51424 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175640 used = 7656108 free = 2253448 buffers = 772788 cached = 5879844 node_type = database-node >> system_cpu_usage = << one_min_avg = 6.3 five_min_avg = 6.14 fifteen_min_avg = 3.77 cpu_share = 15.5 node_type = database-node >> >> 03/23/2026 05:46:22.225 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:23.628 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:23.630 7f60e089a248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:46:23.631 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:23.631 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:23.631 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:23.631 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:23.631 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:46:23.633 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:46:23.633 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:23.634 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:46:23.635 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:23.635 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:23.635 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1774244233237461 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/23/2026 05:46:23.635 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:27.631 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:27.633 7f60e089a248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:46:27.633 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:27.633 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:27.633 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:27.633 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:27.633 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:46:27.634 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:46:27.634 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:27.634 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:46:27.634 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:27.634 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:27.634 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1774244233237461 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/23/2026 05:46:27.634 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:31.636 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:31.637 7f60e089a248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:46:31.637 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:31.637 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:31.637 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:31.637 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:31.637 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:46:31.638 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:46:31.638 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:31.638 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:46:31.638 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:31.638 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:31.638 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1774244233237461 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/23/2026 05:46:31.638 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:35.640 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:35.641 7f60e089a248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:46:35.641 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:35.641 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:35.642 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:35.642 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:35.642 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:46:35.642 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:46:35.642 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:35.642 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:46:35.642 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:35.642 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:35.642 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1774244233237461 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/23/2026 05:46:35.643 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:39.642 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:39.644 7f60e089a248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:46:39.644 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:39.644 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:39.644 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:39.644 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:39.645 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:46:39.645 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:46:39.646 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:39.646 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:46:39.647 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:39.647 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:39.647 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1774244233237461 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/23/2026 05:46:39.648 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:43.647 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:43.648 7f60e089a248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:46:43.648 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:43.648 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:43.649 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:43.649 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:43.649 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:46:43.649 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:46:43.649 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:43.649 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:46:43.649 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:43.649 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:43.650 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1774244233237461 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/23/2026 05:46:43.650 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:47.652 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:47.653 7f60e089a248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:46:47.653 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:47.653 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:47.654 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:47.654 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:47.654 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:46:47.654 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:46:47.654 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:47.654 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:46:47.654 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:47.654 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:47.655 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1774244233237461 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/23/2026 05:46:47.655 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:51.657 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:51.658 7f60e089a248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:46:51.658 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:51.658 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:51.659 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:51.659 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:51.659 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:46:51.659 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:46:51.660 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:51.661 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:46:51.661 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:51.662 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:51.662 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1774244233237461 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/23/2026 05:46:51.662 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:55.659 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:55.660 7f60e089a248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:46:55.660 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:55.661 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:55.661 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:55.661 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:55.661 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:46:55.661 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:46:55.662 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:55.662 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:46:55.662 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:55.662 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:55.662 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1774244233237461 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/23/2026 05:46:55.663 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:59.664 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:59.665 7f60e089a248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:46:59.665 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:59.666 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:59.666 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:59.666 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:59.666 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:46:59.667 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:46:59.667 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:59.667 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:46:59.667 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:59.668 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:59.668 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1774244233237461 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/23/2026 05:46:59.668 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:03.668 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:47:03.670 7f60e089a248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:47:03.671 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:47:03.671 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:03.672 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:47:03.672 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:03.673 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:47:03.673 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:47:03.674 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:03.674 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:47:03.674 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:03.675 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:03.675 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1774244233237461 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/23/2026 05:47:03.676 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:07.674 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:47:07.675 7f60e089a248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:47:07.675 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:47:07.675 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:07.675 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:47:07.675 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:07.675 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:47:07.676 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:47:07.676 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:07.676 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:47:07.676 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:07.676 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:07.677 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1774244233237461 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/23/2026 05:47:07.677 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:11.676 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:47:11.677 7f60e089a248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:47:11.677 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:47:11.677 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:11.678 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:47:11.678 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:11.678 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:47:11.678 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:47:11.678 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:11.678 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:47:11.678 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:11.678 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:11.679 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1774244233237461 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/23/2026 05:47:11.679 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:14.111 7f60e6410df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:47:14.591 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:14.591 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. database_usage = [ << disk_space_used_1k = 18709548 disk_space_available_1k = 62380936 analytics_db_size_1k = 1580 >>, ] >> 03/23/2026 05:47:14.591 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:15.678 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:47:15.680 7f60e089a248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:47:15.680 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:47:15.680 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:15.680 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:47:15.680 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:15.680 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:47:15.681 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:47:15.681 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:15.681 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:47:15.681 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:15.681 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:15.681 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1774244233237461 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/23/2026 05:47:15.681 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:19.680 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:47:19.682 7f60e089a248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:47:19.682 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:47:19.682 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:19.682 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:47:19.682 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:19.682 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:47:19.683 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:47:19.683 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:19.683 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:47:19.683 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:19.683 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:19.683 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1774244233237461 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/23/2026 05:47:19.683 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:23.683 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:47:23.685 7f60e089a248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:47:23.685 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:47:23.685 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:23.685 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:47:23.685 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:23.685 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:47:23.685 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:47:23.685 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:23.686 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:47:23.686 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:23.686 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:23.686 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1774244233237461 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/23/2026 05:47:23.686 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:23.798 7f60e6410df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=18710004, partition_space_available_1k=62380480, percentage_partition_space_used=23, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1387948, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12116, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51516, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=8549936, free=1275896, buffers=776816, cached=5932496, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.39, five_min_avg=5.85, fifteen_min_avg=3.82, cpu_share=14.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:47:23.798 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:23.798 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 18710004 partition_space_available_1k = 62380480 percentage_partition_space_used = 23 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1387948 cpu_share = 0 mem_res = 0 >>contrail-query-engine : << mem_virt = 12116 cpu_share = 0 mem_res = 0 >>contrail-database-nodemgr : << mem_virt = 51516 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175640 used = 8549936 free = 1275896 buffers = 776816 cached = 5932496 node_type = database-node >> system_cpu_usage = << one_min_avg = 5.39 five_min_avg = 5.85 fifteen_min_avg = 3.82 cpu_share = 14.25 node_type = database-node >> >> 03/23/2026 05:47:23.798 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:27.689 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:47:27.690 7f60e089a248 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:47:27.691 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:47:27.691 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:27.691 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:47:27.691 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:27.691 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:47:27.692 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:47:27.692 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:27.692 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:47:27.693 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:27.693 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:27.693 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1774244233237461 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 03/23/2026 05:47:27.694 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:31.694 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:47:31.695 7f60e089a248 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 03/23/2026 05:47:31.695 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:47:31.695 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:31.696 7f60e089a248 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:47:31.696 7f60e089a248 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:31.696 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 03/23/2026 05:47:31.696 7f60e089a248 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 03/23/2026 05:47:31.697 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 03/23/2026 05:47:31.697 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:47:31.697 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:47:31.745 7f60e0d5f248 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001016 03/23/2026 05:47:31.745 7f60e0d5f248 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 03/23/2026 05:47:31.746 7f60e089a248 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 03/23/2026 05:47:31.746 7f60e089a248 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 03/23/2026 05:47:31.746 7f60e089a248 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 03/23/2026 05:47:31.747 7f60e089a248 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 03/23/2026 05:47:31.747 7f60e089a248 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/23/2026 05:47:31.747 7f60e089a248 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/23/2026 05:47:31.747 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 03/23/2026 05:47:31.748 7f60e0c83e48 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 03/23/2026 05:47:31.749 7f60e0c83e48 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 03/23/2026 05:47:41.821 7f60e0d5f248 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001583 03/23/2026 05:47:41.822 7f60e0d5f248 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 03/23/2026 05:47:41.822 7f60e0c83e48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 03/23/2026 05:47:41.822 7f60e0c83e48 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 4 03/23/2026 05:47:41.822 7f60e0c83e48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 03/23/2026 05:47:41.822 7f60e0c83e48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 03/23/2026 05:47:41.823 7f60e0c83e48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/23/2026 05:47:41.823 7f60e0c83e48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/23/2026 05:47:41.823 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 03/23/2026 05:47:41.824 7f60e089a248 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 03/23/2026 05:48:14.209 7f60e6410df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:48:17.684 7f60e6410df0 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-1.', _node_type='Database', _instance_id='0', _timestamp=1774244897684213, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=0, pending=0, all_time_blocked=13)], _table='ObjectDatabaseInfo'), _seqnum=0) 03/23/2026 05:48:23.180 7f60e6410df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19071992, partition_space_available_1k=62018492, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1393344, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12376, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51220, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=9877116, free=457132, buffers=740068, cached=5425188, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.94, five_min_avg=5.7, fifteen_min_avg=3.9, cpu_share=15.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:49:14.008 7f60e6410df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:49:23.152 7f60e6410df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19080048, partition_space_available_1k=62010436, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1404184, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12400, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51220, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=9896476, free=428536, buffers=740156, cached=5434144, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.06, five_min_avg=5.01, fifteen_min_avg=3.77, cpu_share=8.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:50:13.992 7f60e6410df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:50:26.294 7f60e6410df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=20203072, partition_space_available_1k=60887412, percentage_partition_space_used=25, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1465264, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=0, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50808, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=9041228, free=1137796, buffers=710396, cached=5655888, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.38, five_min_avg=5.31, fifteen_min_avg=3.94, cpu_share=13.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:50:26.710 7f60e0d5f048 [contrail-database-nodemgr] [DEBUG]: DBG: event_process_state:contrail-query-engine,group:contrail-query-engine,state:PROCESS_STATE_EXITED 03/23/2026 05:50:26.711 7f60e0d5f048 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-1.', _node_type='Database', _instance_id='0', _timestamp=1774245026711286, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774244230000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_EXITED', start_count=1, stop_count=0, exit_count=1, last_start_time='1774244230000000', last_stop_time='', last_exit_time='1774245026711168', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774244230000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-10-06 01:00:31.164147", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11344-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 03/23/2026 05:50:49.161 7fb2f351cf80 [contrail-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 03/23/2026 05:50:49.161 7fb2f351cf80 [contrail-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 03/23/2026 05:50:49.161 7fb2f351cf80 [contrail-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 03/23/2026 05:50:49.170 7fb2f02c9ea0 [contrail-database-nodemgr] [ERROR]: Cannot write http_port 8103 to /tmp/contrail-database-nodemgr.0.http_port 03/23/2026 05:50:49.170 7fb2f02c9ea0 [contrail-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8103 03/23/2026 05:50:49.171 7fb2f03b02c0 [contrail-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 03/23/2026 05:50:49.172 7fb2f03b02c0 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/23/2026 05:50:49.172 7fb2f03b02c0 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/23/2026 05:50:49.172 7fb2f03b02c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 03/23/2026 05:50:49.173 7fb2f03b02c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:50:49.173 7fb2f03b02c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/23/2026 05:50:49.173 7fb2f03b02c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:50:49.173 7fb2f03b02c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:50:49.173 7fb2f03b02c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1774245049172635 collector_name = collector_ip = collector_list = [ 10.0.0.33:8086, 10.0.0.38:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 03/23/2026 05:50:49.173 7fb2f03b02c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:50:49.457 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 81824586784840539639644251922004022213185255093358945749652903911940388494260, 'start': '1774245046000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-query-engine', 'group': 'contrail-query-engine', 'pid': 67541543278346328747259128469975404377351246493572797173318362673451396344365, 'start': '1774245046000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-database-nodemgr', 'group': 'contrail-database-nodemgr', 'pid': 72847205464147838455045283819661095583056039795478946716149806280700069891814, 'start': '1774245046000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 03/23/2026 05:50:49.533 7fb2f03b0360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:50:49.533 7fb2f03b0360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. build_info = {"build-info": [{"build-version": "v0+nightly", "build-time": "2026-03-23 00:50:56.516868", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-14440-1.novalocal", "build-id": "v0+nightly", "build-number": "202603230008"}]} system_cpu_info = << num_socket = 1 num_cpu = 4 num_core_per_socket = 4 num_thread_per_core = 1 >> >> 03/23/2026 05:50:49.533 7fb2f03b0360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:50:49.534 7fb2f351cf80 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-1.', _node_type='Database', _instance_id='0', _timestamp=1774245049534165, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 03/23/2026 05:50:49.534 7fb2f351cf80 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-1.', _node_type='Database', _instance_id='0', _timestamp=1774245049534450, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774245046000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774245046000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774245046000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info": [{"build-version": "v0+nightly", "build-time": "2026-03-23 00:50:56.516868", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-14440-1.novalocal", "build-id": "v0+nightly", "build-number": "202603230008"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 03/23/2026 05:50:49.556 7fb2f03b0360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:50:49.556 7fb2f03b0360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/23/2026 05:50:49.556 7fb2f03b0360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:50:49.556 7fb2f03b0360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:50:49.556 7fb2f03b0360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774245046000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-query-engine process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774245046000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774245046000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info": [{"build-version": "v0+nightly", "build-time": "2026-03-23 00:50:56.516868", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-14440-1.novalocal", "build-id": "v0+nightly", "build-number": "202603230008"}]} >> 03/23/2026 05:50:49.556 7fb2f03b0360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:50:50.100 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:50:50.635 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:DatabaseUsage] map 03/23/2026 05:50:50.636 7fb2f03b0ea0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:50:50.637 7fb2f03b0ea0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. database_usage = [ << disk_space_used_1k = 20176012 disk_space_available_1k = 60914472 analytics_db_size_1k = 15720 >>, ] >> 03/23/2026 05:50:50.637 7fb2f03b0ea0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:50:51.105 7fb2f351cf80 [contrail-database-nodemgr] [ERROR]: Failed to connect to database by CQL and socket: Result: 1 03/23/2026 05:50:51.105 7fb2f351cf80 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-1.', _node_type='Database', _instance_id='0', _timestamp=1774245051105906, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 03/23/2026 05:50:51.107 7fb2f03d8e00 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:50:51.107 7fb2f03d8e00 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/23/2026 05:50:51.107 7fb2f03d8e00 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:50:52.771 7fb2f351cf80 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 2 03/23/2026 05:50:53.172 7fb2f03d8040 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:50:53.174 7fb2f03d8040 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 03/23/2026 05:50:53.174 7fb2f03d8040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:50:53.175 7fb2f03d8040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:50:53.175 7fb2f03d8040 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-1. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:50:53.175 7fb2f03d8040 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:50:53.175 7fb2f03d8040 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 03/23/2026 05:50:53.175 7fb2f03d8040 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 03/23/2026 05:50:53.175 7fb2f03d8040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 03/23/2026 05:50:53.176 7fb2f03d8040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:50:53.176 7fb2f03d8040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:50:53.179 7fb2f0f2f0e0 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001017 03/23/2026 05:50:53.180 7fb2f0f2f0e0 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 03/23/2026 05:50:53.180 7fb2f03d8040 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 03/23/2026 05:50:53.180 7fb2f03d8040 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 03/23/2026 05:50:53.180 7fb2f03d8040 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 03/23/2026 05:50:53.180 7fb2f03d8040 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/23/2026 05:50:53.180 7fb2f03d8040 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/23/2026 05:50:53.181 7fb2f03d8040 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 03/23/2026 05:50:53.181 7fb2f03b0680 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 03/23/2026 05:50:53.182 7fb2f03b0680 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 03/23/2026 05:50:57.821 7fb2f351cf80 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-1.', _node_type='Database', _instance_id='0', _timestamp=1774245057821513, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774245046000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774245046000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774245046000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info": [{"build-version": "v0+nightly", "build-time": "2026-03-23 00:50:56.516868", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-14440-1.novalocal", "build-id": "v0+nightly", "build-number": "202603230008"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 03/23/2026 05:50:57.822 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=20176536, partition_space_available_1k=60913948, percentage_partition_space_used=25, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1268172, cpu_share=0.0, mem_res=1259292, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=9588, cpu_share=0.0, mem_res=7696, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54600, cpu_share=0.0, mem_res=45128, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=9155512, free=1016204, buffers=712820, cached=5666860, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.89, five_min_avg=5.27, fifteen_min_avg=3.97, cpu_share=0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:51:49.798 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:51:50.749 7fb2f351cf80 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-1.', _node_type='Database', _instance_id='0', _timestamp=1774245110749106, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.33:8086'], status='Up', description='ClientInit to Established on EvSandeshCtrlMessageRecv')]), flag_infos=None, description='')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 03/23/2026 05:51:52.831 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-1.', _node_type='Database', _instance_id='0', _timestamp=1774245112831511, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0)], _table='ObjectDatabaseInfo'), _seqnum=0) 03/23/2026 05:51:52.831 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 03/23/2026 05:51:57.876 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=20192180, partition_space_available_1k=60898304, percentage_partition_space_used=25, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1347428, cpu_share=0.29, mem_res=1326960, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10376, cpu_share=0.0, mem_res=8476, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54596, cpu_share=0.02, mem_res=45124, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=9104760, free=1041340, buffers=712880, cached=5685536, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.08, five_min_avg=4.85, fifteen_min_avg=3.91, cpu_share=8.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:52:49.837 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:52:58.470 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=20192404, partition_space_available_1k=60898080, percentage_partition_space_used=25, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1353776, cpu_share=0.17, mem_res=1328944, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10492, cpu_share=0.0, mem_res=8588, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54704, cpu_share=0.02, mem_res=45228, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=9356004, free=776712, buffers=713004, cached=5696128, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.5, five_min_avg=4.5, fifteen_min_avg=3.84, cpu_share=8.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:53:49.839 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:53:58.585 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=20198904, partition_space_available_1k=60891580, percentage_partition_space_used=25, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1376496, cpu_share=0.16, mem_res=1345884, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11348, cpu_share=0.0, mem_res=9436, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54704, cpu_share=0.02, mem_res=45228, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=9474512, free=647468, buffers=713044, cached=5706920, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.38, five_min_avg=4.28, fifteen_min_avg=3.81, cpu_share=10.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:54:49.885 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:54:58.172 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=20244088, partition_space_available_1k=60846396, percentage_partition_space_used=25, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1369528, cpu_share=0.11, mem_res=1337380, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11356, cpu_share=0.0, mem_res=9448, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54704, cpu_share=0.02, mem_res=45228, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=9435348, free=621932, buffers=717612, cached=5766988, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.44, five_min_avg=4.15, fifteen_min_avg=3.79, cpu_share=13.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:55:49.867 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:55:58.811 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=20250640, partition_space_available_1k=60839844, percentage_partition_space_used=25, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1403812, cpu_share=0.11, mem_res=1369244, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11460, cpu_share=0.0, mem_res=9552, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54720, cpu_share=0.02, mem_res=45228, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=9423116, free=545632, buffers=729252, cached=5843044, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.53, five_min_avg=4.02, fifteen_min_avg=3.77, cpu_share=11.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:56:49.944 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:56:58.188 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=20257956, partition_space_available_1k=60832528, percentage_partition_space_used=25, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1367272, cpu_share=0.1, mem_res=1329376, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10836, cpu_share=0.0, mem_res=8928, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54732, cpu_share=0.02, mem_res=45228, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=9541428, free=403116, buffers=731200, cached=5865128, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.6, five_min_avg=3.66, fifteen_min_avg=3.66, cpu_share=12.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:57:49.874 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:57:57.976 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=20262412, partition_space_available_1k=60828072, percentage_partition_space_used=25, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1377760, cpu_share=0.09, mem_res=1337016, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10924, cpu_share=0.0, mem_res=9016, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54728, cpu_share=0.02, mem_res=45228, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=9468584, free=474672, buffers=731364, cached=5866484, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.28, five_min_avg=3.36, fifteen_min_avg=3.55, cpu_share=12.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:58:49.863 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:58:57.500 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=20268076, partition_space_available_1k=60822408, percentage_partition_space_used=25, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1420072, cpu_share=0.1, mem_res=1376740, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10976, cpu_share=0.0, mem_res=9068, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54732, cpu_share=0.02, mem_res=45228, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=9493348, free=441532, buffers=732040, cached=5874212, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.22, five_min_avg=3.15, fifteen_min_avg=3.47, cpu_share=11.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:59:49.925 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:59:58.497 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=20268056, partition_space_available_1k=60822428, percentage_partition_space_used=25, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1400276, cpu_share=0.09, mem_res=1354724, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11024, cpu_share=0.0, mem_res=9116, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54760, cpu_share=0.02, mem_res=45228, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=9467364, free=512072, buffers=726820, cached=5835448, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.18, five_min_avg=2.96, fifteen_min_avg=3.38, cpu_share=11.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:00:49.894 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:00:58.062 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=20290336, partition_space_available_1k=60800148, percentage_partition_space_used=25, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1398780, cpu_share=0.09, mem_res=1349548, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11048, cpu_share=0.0, mem_res=9140, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54764, cpu_share=0.02, mem_res=45228, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=9622472, free=311052, buffers=727816, cached=5883100, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.37, five_min_avg=2.87, fifteen_min_avg=3.32, cpu_share=14.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:01:49.884 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:01:58.055 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=20313008, partition_space_available_1k=60777476, percentage_partition_space_used=25, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1405324, cpu_share=0.1, mem_res=1353160, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11076, cpu_share=0.0, mem_res=9168, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54756, cpu_share=0.02, mem_res=45228, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=9564460, free=338540, buffers=729232, cached=5913764, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.52, five_min_avg=2.81, fifteen_min_avg=3.27, cpu_share=11.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:02:49.886 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:02:57.673 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=20317756, partition_space_available_1k=60772728, percentage_partition_space_used=25, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1434912, cpu_share=0.09, mem_res=1380220, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11072, cpu_share=0.0, mem_res=9164, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54752, cpu_share=0.02, mem_res=45228, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=9521648, free=388172, buffers=728768, cached=5903256, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.22, five_min_avg=2.67, fifteen_min_avg=3.19, cpu_share=9.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:03:49.983 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:03:58.043 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=20319276, partition_space_available_1k=60771208, percentage_partition_space_used=25, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1430852, cpu_share=0.09, mem_res=1373516, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11076, cpu_share=0.0, mem_res=9168, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54748, cpu_share=0.02, mem_res=45228, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=9447020, free=459720, buffers=728796, cached=5906240, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.08, five_min_avg=2.53, fifteen_min_avg=3.11, cpu_share=8.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:04:49.905 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:04:58.051 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=20324552, partition_space_available_1k=60765932, percentage_partition_space_used=25, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1434920, cpu_share=0.09, mem_res=1375240, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11168, cpu_share=0.0, mem_res=9260, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54736, cpu_share=0.02, mem_res=45228, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=9526296, free=421964, buffers=727344, cached=5866448, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.87, five_min_avg=2.38, fifteen_min_avg=3.02, cpu_share=9.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:05:49.875 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:05:57.676 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=20326800, partition_space_available_1k=60763684, percentage_partition_space_used=25, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1425536, cpu_share=0.09, mem_res=1362892, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11200, cpu_share=0.0, mem_res=9292, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54732, cpu_share=0.01, mem_res=45228, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=9452000, free=492760, buffers=727348, cached=5873132, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.65, five_min_avg=2.26, fifteen_min_avg=2.94, cpu_share=7.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:06:49.896 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:06:57.656 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=20331140, partition_space_available_1k=60759344, percentage_partition_space_used=25, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1444136, cpu_share=0.09, mem_res=1379064, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11228, cpu_share=0.0, mem_res=9320, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54736, cpu_share=0.01, mem_res=45228, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=9462408, free=477952, buffers=727356, cached=5874116, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.42, five_min_avg=2.34, fifteen_min_avg=2.92, cpu_share=5.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:07:46.810 7fb2f02c97c0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/23/2026 06:07:49.928 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:07:54.661 7fb2f03d8e00 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/23/2026 06:07:57.772 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=20466372, partition_space_available_1k=60624112, percentage_partition_space_used=25, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1448140, cpu_share=0.09, mem_res=1380696, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11344, cpu_share=0.0, mem_res=9436, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54704, cpu_share=0.02, mem_res=45228, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=9499292, free=351456, buffers=728532, cached=5962264, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.71, five_min_avg=2.13, fifteen_min_avg=2.81, cpu_share=6.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:08:49.949 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:08:58.377 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21699776, partition_space_available_1k=59390708, percentage_partition_space_used=27, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1430768, cpu_share=0.09, mem_res=1375024, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11308, cpu_share=0.0, mem_res=9400, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53848, cpu_share=0.02, mem_res=45228, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=9414352, free=298028, buffers=687548, cached=6155264, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.52, five_min_avg=2.0, fifteen_min_avg=2.72, cpu_share=8.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:09:28.722 7fb2f03b0a40 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/23/2026 06:09:49.942 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:09:58.805 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21758172, partition_space_available_1k=59332312, percentage_partition_space_used=27, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1457660, cpu_share=0.13, mem_res=1400948, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13508, cpu_share=0.0, mem_res=11544, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53652, cpu_share=0.02, mem_res=45228, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=9933740, free=297456, buffers=679004, cached=5648412, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.55, five_min_avg=1.92, fifteen_min_avg=2.64, cpu_share=10.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:10:50.049 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:10:58.870 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21781444, partition_space_available_1k=59309040, percentage_partition_space_used=27, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1481360, cpu_share=0.14, mem_res=1420300, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13580, cpu_share=0.0, mem_res=11624, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53564, cpu_share=0.02, mem_res=45228, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=10022672, free=322876, buffers=671072, cached=5545208, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.27, five_min_avg=2.11, fifteen_min_avg=2.66, cpu_share=10.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:11:49.938 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:11:57.849 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21799216, partition_space_available_1k=59291268, percentage_partition_space_used=27, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1489236, cpu_share=0.16, mem_res=1421692, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13616, cpu_share=0.0, mem_res=11660, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53484, cpu_share=0.02, mem_res=45228, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=10152992, free=366432, buffers=662864, cached=5379620, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.31, five_min_avg=2.22, fifteen_min_avg=2.67, cpu_share=10.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:12:50.052 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:12:58.274 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21815720, partition_space_available_1k=59274764, percentage_partition_space_used=27, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1505396, cpu_share=0.15, mem_res=1430692, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13632, cpu_share=0.0, mem_res=11676, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53428, cpu_share=0.02, mem_res=45228, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=10247520, free=356128, buffers=658264, cached=5300088, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.73, five_min_avg=2.39, fifteen_min_avg=2.7, cpu_share=11.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:13:49.972 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:13:58.656 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21833616, partition_space_available_1k=59256868, percentage_partition_space_used=27, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1521548, cpu_share=0.15, mem_res=1438264, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13680, cpu_share=0.0, mem_res=11724, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53408, cpu_share=0.02, mem_res=45228, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=10224332, free=398748, buffers=657368, cached=5286092, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.66, five_min_avg=2.42, fifteen_min_avg=2.7, cpu_share=10.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:14:50.155 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:15:00.178 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21890620, partition_space_available_1k=59199864, percentage_partition_space_used=27, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1576164, cpu_share=0.21, mem_res=1460108, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13580, cpu_share=0.0, mem_res=11624, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53376, cpu_share=0.02, mem_res=45228, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=10255200, free=431596, buffers=653752, cached=5223204, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.92, five_min_avg=3.19, fifteen_min_avg=2.93, cpu_share=11.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:15:49.954 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:15:57.965 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21900748, partition_space_available_1k=59189736, percentage_partition_space_used=27, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1538176, cpu_share=0.12, mem_res=1416648, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13676, cpu_share=0.0, mem_res=11720, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53376, cpu_share=0.02, mem_res=45228, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=10256896, free=416900, buffers=653776, cached=5236084, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.5, five_min_avg=2.94, fifteen_min_avg=2.86, cpu_share=7.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:16:49.960 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:16:57.845 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21913712, partition_space_available_1k=59176772, percentage_partition_space_used=27, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1564384, cpu_share=0.13, mem_res=1436016, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13676, cpu_share=0.0, mem_res=11720, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53424, cpu_share=0.02, mem_res=45268, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=10258944, free=401720, buffers=653840, cached=5249100, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.99, five_min_avg=2.61, fifteen_min_avg=2.75, cpu_share=8.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:17:49.957 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:17:58.341 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21926688, partition_space_available_1k=59163796, percentage_partition_space_used=27, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1569260, cpu_share=0.12, mem_res=1433336, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13740, cpu_share=0.0, mem_res=11784, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53436, cpu_share=0.02, mem_res=45276, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=10280940, free=364616, buffers=653868, cached=5264108, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.85, five_min_avg=2.74, fifteen_min_avg=2.78, cpu_share=9.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:18:49.991 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:18:58.385 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21937964, partition_space_available_1k=59152520, percentage_partition_space_used=27, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1585460, cpu_share=0.12, mem_res=1447872, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13780, cpu_share=0.0, mem_res=11820, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53392, cpu_share=0.02, mem_res=45276, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=10291344, free=436700, buffers=650528, cached=5186008, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.88, five_min_avg=3.05, fifteen_min_avg=2.89, cpu_share=8.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:19:50.013 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:19:57.907 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21962044, partition_space_available_1k=59128440, percentage_partition_space_used=27, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1615024, cpu_share=0.13, mem_res=1460188, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13780, cpu_share=0.0, mem_res=11820, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53504, cpu_share=0.02, mem_res=45380, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=10342256, free=351528, buffers=650920, cached=5225660, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.39, five_min_avg=3.09, fifteen_min_avg=2.91, cpu_share=9.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:20:49.989 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:20:58.508 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21970296, partition_space_available_1k=59120188, percentage_partition_space_used=27, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1577376, cpu_share=0.12, mem_res=1416020, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13768, cpu_share=0.0, mem_res=11808, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53512, cpu_share=0.02, mem_res=45380, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=10270148, free=410012, buffers=650976, cached=5233596, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.58, five_min_avg=2.87, fifteen_min_avg=2.84, cpu_share=9.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:21:07.847 7fb2f03b07c0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/23/2026 06:21:49.978 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:21:57.885 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21993676, partition_space_available_1k=59096808, percentage_partition_space_used=27, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1584896, cpu_share=0.12, mem_res=1415704, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13852, cpu_share=0.0, mem_res=11892, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53508, cpu_share=0.02, mem_res=45380, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=10128272, free=508120, buffers=652008, cached=5276400, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.2, five_min_avg=2.72, fifteen_min_avg=2.79, cpu_share=9.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:22:50.026 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:22:57.945 7fb2f351cf80 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=22570328, partition_space_available_1k=58520156, percentage_partition_space_used=28, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1563424, cpu_share=0.09, mem_res=1415864, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13860, cpu_share=0.0, mem_res=11900, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53312, cpu_share=0.02, mem_res=45380, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=10067016, free=294740, buffers=640784, cached=5566160, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.99, five_min_avg=2.55, fifteen_min_avg=2.73, cpu_share=6.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:23:20.505 7fb2f11b3c20 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map