03/23/2026 05:37:11.980 7f241a2a4df0 [contrail-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 03/23/2026 05:37:11.980 7f241a2a4df0 [contrail-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 03/23/2026 05:37:11.980 7f241a2a4df0 [contrail-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 03/23/2026 05:37:11.986 7f2414b0a548 [contrail-database-nodemgr] [ERROR]: Cannot write http_port 8103 to /tmp/contrail-database-nodemgr.0.http_port 03/23/2026 05:37:11.987 7f2414b0a548 [contrail-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8103 03/23/2026 05:37:11.987 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 03/23/2026 05:37:11.988 7f2414b0ad48 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/23/2026 05:37:11.988 7f2414b0ad48 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/23/2026 05:37:11.988 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 03/23/2026 05:37:11.988 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:11.989 7f2414b0ad48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/23/2026 05:37:11.989 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:11.989 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:11.989 7f2414b0ad48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1774244231988696 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 03/23/2026 05:37:11.989 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:12.058 7f241a2a4df0 [contrail-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 92108649695506617674075927083111616066868778663496569658491870199641143800355, 'start': '1774244229000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-query-engine', 'group': 'contrail-query-engine', 'pid': 100590882529492409870275793299387409509861673052909748140294711804681891961112, 'start': '1774244230000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-database-nodemgr', 'group': 'contrail-database-nodemgr', 'pid': 108143431439141719412280299433215305206616161463108397688786132718593978470639, 'start': '1774244230000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 03/23/2026 05:37:12.189 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:12.190 7f2414b0ad48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. build_info = {"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-10-06 01:00:31.164147", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11344-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 4 num_core_per_socket = 4 num_thread_per_core = 1 >> running_package_version = v0+nightly.r24.1.centos-nightly_r24.1_centos.el7 installed_package_version = v0+nightly.r24.1.centos-nightly_r24.1_centos.el7 >> 03/23/2026 05:37:12.190 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:12.190 7f241a2a4df0 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-2.', _node_type='Database', _instance_id='0', _timestamp=1774244232190486, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 03/23/2026 05:37:12.190 7f241a2a4df0 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-2.', _node_type='Database', _instance_id='0', _timestamp=1774244232190790, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774244229000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774244230000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774244230000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-10-06 01:00:31.164147", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11344-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 03/23/2026 05:37:12.192 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:12.192 7f2414b0ad48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/23/2026 05:37:12.192 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:12.192 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:12.193 7f2414b0ad48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774244229000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-query-engine process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774244230000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774244230000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-10-06 01:00:31.164147", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11344-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} >> 03/23/2026 05:37:12.193 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:12.461 7f241a2a4df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:37:12.786 7f241a2a4df0 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:DatabaseUsage] map 03/23/2026 05:37:12.789 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:12.789 7f2414b0ad48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. database_usage = [ << disk_space_used_1k = 6663004 disk_space_available_1k = 74427480 analytics_db_size_1k = 4 >>, ] >> 03/23/2026 05:37:12.789 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:13.177 7f241a2a4df0 [contrail-database-nodemgr] [ERROR]: Failed to connect to database by CQL: Result: 1 03/23/2026 05:37:13.178 7f241a2a4df0 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-2.', _node_type='Database', _instance_id='0', _timestamp=1774244233178374, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 03/23/2026 05:37:13.180 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:13.180 7f2414b0ad48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/23/2026 05:37:13.180 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:15.949 7f241a2a4df0 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-2.', _node_type='Database', _instance_id='0', _timestamp=1774244235949906, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=1), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0)], _table='ObjectDatabaseInfo'), _seqnum=0) 03/23/2026 05:37:15.950 7f241a2a4df0 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 03/23/2026 05:37:15.952 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:15.954 7f2414b0ad48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] CassandraStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. cassandra_compaction_task = << pending_compaction_tasks = 1 >> thread_pool_stats = [ << pool_name = ReadStage active = 0 pending = 0 all_time_blocked = 0 >>, << pool_name = CompactionExecutor active = 0 pending = 0 all_time_blocked = 0 >>, << pool_name = MutationStage active = 0 pending = 0 all_time_blocked = 0 >>, << pool_name = MemtableFlushWriter active = 0 pending = 0 all_time_blocked = 0 >>, ] >> 03/23/2026 05:37:15.954 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:15.988 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:37:15.990 7f2414b0ad48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:37:15.990 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:37:15.990 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:15.990 7f2414b0ad48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:37:15.991 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:15.991 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:37:15.991 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:37:15.991 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:15.991 7f2414b0ad48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:37:15.991 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:15.992 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:15.992 7f2414b0ad48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1774244231988696 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:37:15.992 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:19.990 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:37:19.991 7f2414b0ad48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:37:19.992 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:37:19.992 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:19.993 7f2414b0ad48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:37:19.994 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:19.994 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:37:19.994 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:37:19.995 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:19.995 7f2414b0ad48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:37:19.996 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:19.997 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:19.997 7f2414b0ad48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1774244231988696 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:37:19.997 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:20.974 7f241a2a4df0 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-2.', _node_type='Database', _instance_id='0', _timestamp=1774244240974807, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774244229000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774244230000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774244230000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-10-06 01:00:31.164147", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11344-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 03/23/2026 05:37:20.975 7f241a2a4df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6664276, partition_space_available_1k=74426208, percentage_partition_space_used=8, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1215752, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=7660, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51476, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=4733524, free=5567856, buffers=244100, cached=5979628, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.73, five_min_avg=1.9, fifteen_min_avg=0.74, cpu_share=0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:37:20.975 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:20.975 7f2414b0ad48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774244229000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-query-engine process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774244230000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774244230000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-10-06 01:00:31.164147", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11344-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} >> 03/23/2026 05:37:20.975 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:20.975 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:20.976 7f2414b0ad48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 6664276 partition_space_available_1k = 74426208 percentage_partition_space_used = 8 >> } all_core_file_list = [ ] process_mem_cpu_usage = { cassandra : << mem_virt = 1215752 cpu_share = 0 mem_res = 0 >>contrail-query-engine : << mem_virt = 7660 cpu_share = 0 mem_res = 0 >>contrail-database-nodemgr : << mem_virt = 51476 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175636 used = 4733524 free = 5567856 buffers = 244100 cached = 5979628 node_type = database-node >> system_cpu_usage = << one_min_avg = 4.73 five_min_avg = 1.9 fifteen_min_avg = 0.74 cpu_share = 0 node_type = database-node >> >> 03/23/2026 05:37:20.976 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:23.994 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:37:23.995 7f2414b0ad48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:37:23.995 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:37:23.996 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:23.996 7f2414b0ad48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:37:23.996 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:23.996 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:37:23.996 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:37:23.996 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:23.996 7f2414b0ad48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:37:23.996 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:23.997 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:23.997 7f2414b0ad48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1774244231988696 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:37:23.997 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:27.995 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:37:27.997 7f2414b0ad48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:37:27.997 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:37:27.997 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:27.997 7f2414b0ad48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:37:27.997 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:27.997 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:37:27.998 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:37:27.998 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:27.998 7f2414b0ad48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:37:27.998 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:27.998 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:27.998 7f2414b0ad48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1774244231988696 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:37:27.998 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:31.999 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:37:32.001 7f2414b0ad48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:37:32.001 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:37:32.001 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:32.001 7f2414b0ad48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:37:32.001 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:32.001 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:37:32.002 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:37:32.002 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:32.002 7f2414b0ad48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:37:32.002 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:32.002 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:32.002 7f2414b0ad48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1774244231988696 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:37:32.002 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:36.003 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:37:36.005 7f2414b0ad48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:37:36.005 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:37:36.005 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:36.005 7f2414b0ad48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:37:36.005 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:36.005 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:37:36.006 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:37:36.006 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:36.006 7f2414b0ad48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:37:36.006 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:36.006 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:36.006 7f2414b0ad48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1774244231988696 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:37:36.006 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:40.006 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:37:40.007 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 03/23/2026 05:37:40.007 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:37:40.007 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:40.007 7f2414b0ad48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:37:40.008 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:40.008 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 03/23/2026 05:37:40.008 7f2414b0ad48 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 03/23/2026 05:37:40.008 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 03/23/2026 05:37:40.008 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:37:40.008 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:37:49.060 7f2414bc1248 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001016 03/23/2026 05:37:49.060 7f2414bc1248 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 03/23/2026 05:37:49.061 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 03/23/2026 05:37:49.061 7f2414b0ad48 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 03/23/2026 05:37:49.061 7f2414b0ad48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 03/23/2026 05:37:49.061 7f2414b0ad48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 03/23/2026 05:37:49.061 7f2414b0ad48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/23/2026 05:37:49.062 7f2414b0ad48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/23/2026 05:37:49.062 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 03/23/2026 05:37:49.062 7f2414b0ae48 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 03/23/2026 05:37:49.064 7f2414b0ae48 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 03/23/2026 05:38:00.038 7f2414bc1248 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001581 03/23/2026 05:38:00.038 7f2414bc1248 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 03/23/2026 05:38:00.039 7f2414b0ae48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 03/23/2026 05:38:00.039 7f2414b0ae48 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 4 03/23/2026 05:38:00.039 7f2414b0ae48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 03/23/2026 05:38:00.040 7f2414b0ae48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 03/23/2026 05:38:00.040 7f2414b0ae48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/23/2026 05:38:00.040 7f2414b0ae48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/23/2026 05:38:00.040 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 03/23/2026 05:38:00.041 7f2414b0ad48 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 03/23/2026 05:38:12.423 7f241a2a4df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:38:13.321 7f241a2a4df0 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-2.', _node_type='Database', _instance_id='0', _timestamp=1774244293321303, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.50:8086'], status='Up', description='ClientInit to Established on EvSandeshCtrlMessageRecv')]), flag_infos=None, description='')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 03/23/2026 05:38:16.073 7f241a2a4df0 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-2.', _node_type='Database', _instance_id='0', _timestamp=1774244296073532, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=0, pending=0, all_time_blocked=0)], _table='ObjectDatabaseInfo'), _seqnum=0) 03/23/2026 05:38:21.099 7f241a2a4df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7085452, partition_space_available_1k=74005032, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1333972, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=9728, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52108, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=5932008, free=3874200, buffers=266968, cached=6452848, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.17, five_min_avg=2.19, fifteen_min_avg=0.91, cpu_share=10.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:39:12.416 7f241a2a4df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:39:20.817 7f241a2a4df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7163224, partition_space_available_1k=73927260, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1331628, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=9808, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52156, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=5876488, free=3846892, buffers=268504, cached=6534116, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.26, five_min_avg=2.3, fifteen_min_avg=1.03, cpu_share=11.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:40:12.479 7f241a2a4df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:40:21.047 7f241a2a4df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7194344, partition_space_available_1k=73896140, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1333772, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=9932, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52160, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=5932544, free=3760960, buffers=269916, cached=6562564, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.75, five_min_avg=2.62, fifteen_min_avg=1.22, cpu_share=12.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:41:08.583 7f2414b0ae48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/23/2026 05:41:12.466 7f241a2a4df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:41:20.885 7f241a2a4df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7644744, partition_space_available_1k=73445740, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1374428, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10112, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52164, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6006936, free=3203876, buffers=276284, cached=7038692, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.98, five_min_avg=2.9, fifteen_min_avg=1.41, cpu_share=13.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:41:21.480 7f2414b0ae48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/23/2026 05:42:12.478 7f241a2a4df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:42:20.827 7f241a2a4df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7657608, partition_space_available_1k=73432876, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1346524, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10208, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52320, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6004336, free=3204780, buffers=277392, cached=7039412, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.22, five_min_avg=3.13, fifteen_min_avg=1.58, cpu_share=12.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:43:12.473 7f241a2a4df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:43:29.293 7f699fefcf20 [contrail-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 03/23/2026 05:43:29.293 7f699fefcf20 [contrail-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 03/23/2026 05:43:29.293 7f699fefcf20 [contrail-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 03/23/2026 05:43:29.299 7f699a125748 [contrail-database-nodemgr] [ERROR]: Cannot write http_port 8103 to /tmp/contrail-database-nodemgr.0.http_port 03/23/2026 05:43:29.300 7f699a125748 [contrail-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8103 03/23/2026 05:43:29.300 7f6998f43048 [contrail-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 03/23/2026 05:43:29.302 7f6998f43048 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/23/2026 05:43:29.302 7f6998f43048 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/23/2026 05:43:29.303 7f6998f43048 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 03/23/2026 05:43:29.303 7f6998f43048 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:43:29.306 7f6998f43048 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/23/2026 05:43:29.306 7f6998f43048 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:43:29.306 7f6998f43048 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:43:29.306 7f6998f43048 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1774244609302770 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 03/23/2026 05:43:29.306 7f6998f43048 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:43:31.918 7f699fefcf20 [contrail-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 92108649695506617674075927083111616066868778663496569658491870199641143800355, 'start': '1774244607000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-query-engine', 'group': 'contrail-query-engine', 'pid': 100590882529492409870275793299387409509861673052909748140294711804681891961112, 'start': '1774244607000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-database-nodemgr', 'group': 'contrail-database-nodemgr', 'pid': 108143431439141719412280299433215305206616161463108397688786132718593978470639, 'start': '1774244606000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 03/23/2026 05:43:32.296 7f6998f43448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:43:32.297 7f6998f43448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. build_info = {"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-10-06 01:00:31.164147", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11344-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 4 num_core_per_socket = 4 num_thread_per_core = 1 >> running_package_version = v0+nightly.r24.1.centos-nightly_r24.1_centos.el7 installed_package_version = v0+nightly.r24.1.centos-nightly_r24.1_centos.el7 >> 03/23/2026 05:43:32.297 7f6998f43448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:43:32.297 7f699fefcf20 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-2.', _node_type='Database', _instance_id='0', _timestamp=1774244612297761, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 03/23/2026 05:43:32.298 7f699fefcf20 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-2.', _node_type='Database', _instance_id='0', _timestamp=1774244612298114, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774244607000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774244607000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774244606000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-10-06 01:00:31.164147", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11344-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 03/23/2026 05:43:32.328 7f6998f43448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:43:32.328 7f6998f43448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/23/2026 05:43:32.329 7f6998f43448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:43:32.329 7f6998f43448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:43:32.329 7f6998f43448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774244607000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-query-engine process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774244607000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774244606000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-10-06 01:00:31.164147", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11344-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} >> 03/23/2026 05:43:32.329 7f6998f43448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:43:33.223 7f699fefcf20 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:43:33.306 7f6998f43448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:43:33.364 7f6998f43448 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 03/23/2026 05:43:33.374 7f6998f43448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:43:33.374 7f6998f43448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:43:33.374 7f6998f43448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:43:33.374 7f6998f43448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:43:33.374 7f6998f43448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 03/23/2026 05:43:33.375 7f6998f43448 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 03/23/2026 05:43:33.375 7f6998f43448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 03/23/2026 05:43:33.378 7f6998f43448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:43:33.378 7f6998f43448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:43:34.873 7f699fefcf20 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:DatabaseUsage] map 03/23/2026 05:43:34.875 7f6998f43448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:43:37.652 7f699fefcf20 [contrail-database-nodemgr] [ERROR]: Failed to connect to database by CQL: Result: 1 03/23/2026 05:43:37.652 7f699fefcf20 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-2.', _node_type='Database', _instance_id='0', _timestamp=1774244617652685, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.50:8086'], status='Initializing', description='Connect to ClientInit on EvTcpConnected')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[Connect to ClientInit on EvTcpConnected] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 03/23/2026 05:43:37.654 7f6998f43448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:43:41.003 7f6998f03448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 03/23/2026 05:43:41.004 7f6998f43448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit] 03/23/2026 05:43:41.004 7f6998f43448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle] 03/23/2026 05:43:41.004 7f6998f43448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:43:41.004 7f6998f43448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = ClientInit to Idle on EvTcpClose >>, ] description = Cassandra state detected DOWN. Collector:Collector[ClientInit to Idle on EvTcpClose] connection down >>, ] >> 03/23/2026 05:43:41.004 7f6998f43448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:43:41.004 7f6998f43448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:43:41.005 7f6998f43448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8103 start_time = 1774244609302770 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:43:41.005 7f6998f43448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:43:44.796 7f699fefcf20 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 2 03/23/2026 05:43:45.004 7f6998f43448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:43:45.006 7f6998f43448 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 03/23/2026 05:43:45.006 7f6998f43448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:43:45.006 7f6998f43448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:43:45.006 7f6998f43448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:43:45.006 7f6998f43448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:43:45.006 7f6998f43448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 03/23/2026 05:43:45.007 7f6998f43448 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 03/23/2026 05:43:45.007 7f6998f43448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 03/23/2026 05:43:45.007 7f6998f43448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:43:45.007 7f6998f43448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:43:45.010 7f6998f03448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 03/23/2026 05:43:45.010 7f6998f43448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit] 03/23/2026 05:43:45.011 7f6998f43448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle] 03/23/2026 05:43:45.011 7f6998f43448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:43:45.011 7f6998f43448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = ClientInit to Idle on EvTcpClose >>, ] description = Cassandra state detected DOWN. Collector:Collector[ClientInit to Idle on EvTcpClose] connection down >>, ] >> 03/23/2026 05:43:45.011 7f6998f43448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:43:45.011 7f6998f43448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:43:45.011 7f6998f43448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 2 pid = 1 http_port = 8103 start_time = 1774244609302770 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:43:45.011 7f6998f43448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:43:49.011 7f6998f43448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:43:49.013 7f6998f43448 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 03/23/2026 05:43:49.013 7f6998f43448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:43:49.013 7f6998f43448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:43:49.013 7f6998f43448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:43:49.013 7f6998f43448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:43:49.013 7f6998f43448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 03/23/2026 05:43:49.013 7f6998f43448 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 03/23/2026 05:43:49.014 7f6998f43448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 03/23/2026 05:43:49.014 7f6998f43448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:43:49.014 7f6998f43448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:43:49.019 7f6998f03448 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001017 03/23/2026 05:43:49.019 7f6998f03448 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 03/23/2026 05:43:49.020 7f6998f43448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 03/23/2026 05:43:49.020 7f6998f43448 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 03/23/2026 05:43:49.020 7f6998f43448 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 03/23/2026 05:43:49.020 7f6998f43448 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/23/2026 05:43:49.020 7f6998f43448 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/23/2026 05:43:49.020 7f6998f43448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 03/23/2026 05:43:49.021 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 03/23/2026 05:43:49.021 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 03/23/2026 05:43:49.991 7f699fefcf20 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-2.', _node_type='Database', _instance_id='0', _timestamp=1774244629991484, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774244607000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774244607000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774244606000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-10-06 01:00:31.164147", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11344-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 03/23/2026 05:43:49.992 7f699fefcf20 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8055132, partition_space_available_1k=73035352, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1208344, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=9176, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44556, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=5161312, free=3571180, buffers=285688, cached=7507816, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=16.34, five_min_avg=6.7, fifteen_min_avg=2.94, cpu_share=0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:44:32.576 7f699fefcf20 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:44:33.683 7f699fefcf20 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-2.', _node_type='Database', _instance_id='0', _timestamp=1774244673683105, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.38:8086'], status='Up', description='ClientInit to Established on EvSandeshCtrlMessageRecv')]), flag_infos=None, description='')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 03/23/2026 05:44:39.275 7f699fefcf20 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-2.', _node_type='Database', _instance_id='0', _timestamp=1774244679275057, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0)], _table='ObjectDatabaseInfo'), _seqnum=0) 03/23/2026 05:44:39.275 7f699fefcf20 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 03/23/2026 05:44:44.640 7f699fefcf20 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=9024936, partition_space_available_1k=72065548, percentage_partition_space_used=11, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1314688, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=9812, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44600, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=5772388, free=1668640, buffers=337620, cached=8747556, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=16.69, five_min_avg=8.27, fifteen_min_avg=3.68, cpu_share=22.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:45:32.710 7f699fefcf20 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:45:44.689 7f699fefcf20 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=10646224, partition_space_available_1k=70444260, percentage_partition_space_used=13, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1319396, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=9820, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44612, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=5800884, free=542716, buffers=364300, cached=9817756, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=11.55, five_min_avg=8.33, fifteen_min_avg=3.98, cpu_share=17.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:45:55.890 7f6998f03448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 03/23/2026 05:45:55.890 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Established] 03/23/2026 05:45:55.892 7f6998f43148 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 03/23/2026 05:45:55.892 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[Established] -> State[Connect] 03/23/2026 05:45:55.892 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:45:55.892 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Established to Connect on EvTcpClose >>, ] description = Collector:Collector[Established to Connect on EvTcpClose] connection down >>, ] >> 03/23/2026 05:45:55.892 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:45:55.892 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 03/23/2026 05:45:55.893 7f6998f43148 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 03/23/2026 05:45:55.893 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 03/23/2026 05:45:55.893 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:45:55.893 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:45:55.941 7f6998f03448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 03/23/2026 05:45:55.942 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit] 03/23/2026 05:45:55.942 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle] 03/23/2026 05:45:55.942 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:45:55.942 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = ClientInit to Idle on EvTcpClose >>, ] description = Collector:Collector[ClientInit to Idle on EvTcpClose] connection down >>, ] >> 03/23/2026 05:45:55.942 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:45:55.942 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:45:55.943 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1774244609302770 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:45:55.943 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:45:59.945 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:45:59.946 7f6998f43148 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 03/23/2026 05:45:59.947 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:45:59.947 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:45:59.947 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:45:59.947 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:45:59.947 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 03/23/2026 05:45:59.947 7f6998f43148 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 03/23/2026 05:45:59.948 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 03/23/2026 05:45:59.948 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:45:59.948 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:45:59.949 7f6998f03448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 03/23/2026 05:45:59.950 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit] 03/23/2026 05:45:59.950 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle] 03/23/2026 05:45:59.950 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:45:59.950 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = ClientInit to Idle on EvTcpClose >>, ] description = Collector:Collector[ClientInit to Idle on EvTcpClose] connection down >>, ] >> 03/23/2026 05:45:59.950 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:45:59.950 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:45:59.951 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1774244609302770 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:45:59.951 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:03.954 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:03.956 7f6998f43148 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 03/23/2026 05:46:03.956 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:03.956 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:03.956 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:03.956 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:03.956 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 03/23/2026 05:46:03.956 7f6998f43148 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 03/23/2026 05:46:03.957 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 03/23/2026 05:46:03.957 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:46:03.957 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:46:07.618 7f6998f03448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 03/23/2026 05:46:07.619 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit] 03/23/2026 05:46:07.619 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle] 03/23/2026 05:46:07.619 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:07.619 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = ClientInit to Idle on EvTcpClose >>, ] description = Collector:Collector[ClientInit to Idle on EvTcpClose] connection down >>, ] >> 03/23/2026 05:46:07.619 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:07.619 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:07.620 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1774244609302770 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:46:07.620 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:11.621 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:11.622 7f6998f43148 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:46:11.625 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:11.626 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:11.626 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:11.626 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:11.626 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:46:11.626 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:46:11.627 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:11.627 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:46:11.627 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:11.627 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:11.627 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1774244609302770 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:46:11.627 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:15.623 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:15.624 7f6998f43148 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:46:15.624 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:15.624 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:15.624 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:15.624 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:15.624 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:46:15.624 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:46:15.625 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:15.625 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:46:15.625 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:15.625 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:15.625 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1774244609302770 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:46:15.625 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:19.627 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:19.628 7f6998f43148 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:46:19.629 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:19.629 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:19.629 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:19.629 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:19.629 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:46:19.630 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:46:19.630 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:19.630 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:46:19.630 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:19.630 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:19.630 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1774244609302770 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:46:19.630 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:23.632 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:23.632 7f6998f43148 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:46:23.633 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:23.633 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:23.633 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:23.633 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:23.633 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:46:23.633 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:46:23.634 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:23.634 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:46:23.634 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:23.634 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:23.634 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1774244609302770 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:46:23.634 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:27.634 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:27.635 7f6998f43148 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:46:27.635 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:27.635 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:27.636 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:27.636 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:27.636 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:46:27.636 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:46:27.636 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:27.636 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:46:27.637 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:27.637 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:27.637 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1774244609302770 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:46:27.637 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:31.638 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:31.639 7f6998f43148 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:46:31.640 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:31.640 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:31.640 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:31.640 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:31.640 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:46:31.640 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:46:31.641 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:31.641 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:46:31.641 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:31.641 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:31.641 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1774244609302770 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:46:31.641 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:32.496 7f699fefcf20 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:46:32.695 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:32.695 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. database_usage = [ << disk_space_used_1k = 10481952 disk_space_available_1k = 70608532 analytics_db_size_1k = 5280 >>, ] >> 03/23/2026 05:46:32.695 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:35.640 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:35.643 7f6998f43148 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:46:35.645 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:35.645 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:35.645 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:35.645 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:35.645 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:46:35.646 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:46:35.647 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:35.647 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:46:35.647 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:35.647 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:35.647 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1774244609302770 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:46:35.648 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:39.644 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:39.645 7f6998f43148 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:46:39.646 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:39.646 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:39.646 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:39.646 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:39.646 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:46:39.647 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:46:39.647 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:39.647 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:46:39.647 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:39.647 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:39.647 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1774244609302770 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:46:39.648 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:40.730 7f699fefcf20 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=10481928, partition_space_available_1k=70608556, percentage_partition_space_used=13, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1319800, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10812, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44824, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=4524512, free=1909216, buffers=370848, cached=9719764, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.15, five_min_avg=7.09, fifteen_min_avg=3.83, cpu_share=5.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:46:40.730 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:40.731 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 10481928 partition_space_available_1k = 70608556 percentage_partition_space_used = 13 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1319800 cpu_share = 0 mem_res = 0 >>contrail-query-engine : << mem_virt = 10812 cpu_share = 0 mem_res = 0 >>contrail-database-nodemgr : << mem_virt = 44824 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175636 used = 4524512 free = 1909216 buffers = 370848 cached = 9719764 node_type = database-node >> system_cpu_usage = << one_min_avg = 5.15 five_min_avg = 7.09 fifteen_min_avg = 3.83 cpu_share = 5.25 node_type = database-node >> >> 03/23/2026 05:46:40.731 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:43.648 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:43.649 7f6998f43148 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:46:43.649 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:43.649 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:43.650 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:43.650 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:43.650 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:46:43.650 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:46:43.650 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:43.650 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:46:43.650 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:43.650 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:43.650 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1774244609302770 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:46:43.651 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:47.650 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:47.651 7f6998f43148 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:46:47.651 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:47.651 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:47.651 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:47.651 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:47.651 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:46:47.652 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:46:47.652 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:47.652 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:46:47.652 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:47.652 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:47.652 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1774244609302770 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:46:47.652 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:51.652 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:51.653 7f6998f43148 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:46:51.653 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:51.653 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:51.653 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:51.653 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:51.653 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:46:51.654 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:46:51.654 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:51.654 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:46:51.654 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:51.654 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:51.654 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1774244609302770 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:46:51.654 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:55.655 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:55.657 7f6998f43148 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:46:55.657 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:55.657 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:55.657 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:55.657 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:55.657 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:46:55.658 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:46:55.658 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:55.658 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:46:55.658 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:55.658 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:55.658 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1774244609302770 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:46:55.658 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:59.658 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:59.659 7f6998f43148 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:46:59.659 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:59.660 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:59.660 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:59.660 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:59.660 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:46:59.661 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:46:59.661 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:59.661 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:46:59.661 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:59.661 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:59.661 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1774244609302770 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:46:59.661 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:03.662 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:47:03.663 7f6998f43148 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:47:03.663 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:47:03.663 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:03.663 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:47:03.664 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:03.664 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:47:03.664 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:47:03.664 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:03.664 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:47:03.664 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:03.664 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:03.665 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1774244609302770 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:47:03.665 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:07.665 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:47:07.666 7f6998f43148 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:47:07.667 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:47:07.667 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:07.667 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:47:07.667 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:07.667 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:47:07.668 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:47:07.668 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:07.668 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:47:07.668 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:07.668 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:07.668 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1774244609302770 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:47:07.668 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:11.667 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:47:11.668 7f6998f43148 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:47:11.669 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:47:11.669 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:11.669 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:47:11.669 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:11.670 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:47:11.670 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:47:11.670 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:11.671 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:47:11.671 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:11.671 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:11.671 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1774244609302770 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:47:11.672 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:15.670 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:47:15.672 7f6998f43148 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:47:15.672 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:47:15.672 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:15.672 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:47:15.672 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:15.672 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:47:15.673 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:47:15.673 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:15.673 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:47:15.673 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:15.673 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:15.673 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1774244609302770 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:47:15.673 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:19.674 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:47:19.675 7f6998f43148 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:47:19.675 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:47:19.675 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:19.676 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:47:19.676 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:19.676 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:47:19.676 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:47:19.676 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:19.677 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:47:19.677 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:19.677 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:19.677 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1774244609302770 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:47:19.677 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:23.678 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:47:23.679 7f6998f43148 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:47:23.679 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:47:23.679 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:23.679 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:47:23.679 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:23.679 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:47:23.680 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:47:23.680 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:23.680 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:47:23.680 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:23.680 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:23.680 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1774244609302770 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:47:23.680 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:27.682 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:47:27.683 7f6998f43148 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:47:27.683 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:47:27.683 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:27.683 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:47:27.683 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:27.684 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:47:27.684 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:47:27.684 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:27.684 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:47:27.684 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:27.684 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:27.684 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 6 pid = 1 http_port = 8103 start_time = 1774244609302770 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.50:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:47:27.684 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:31.686 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:47:31.687 7f6998f43148 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 03/23/2026 05:47:31.687 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:47:31.688 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:31.688 7f6998f43148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:47:31.688 7f6998f43148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:31.688 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 03/23/2026 05:47:31.688 7f6998f43148 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 03/23/2026 05:47:31.688 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 03/23/2026 05:47:31.688 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:47:31.689 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:47:31.723 7f6998f03448 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001016 03/23/2026 05:47:31.724 7f6998f03448 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 03/23/2026 05:47:31.724 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 03/23/2026 05:47:31.724 7f6998f43148 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 03/23/2026 05:47:31.725 7f6998f43148 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 03/23/2026 05:47:31.725 7f6998f43148 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 03/23/2026 05:47:31.725 7f6998f43148 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/23/2026 05:47:31.725 7f6998f43148 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/23/2026 05:47:31.725 7f6998f43148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 03/23/2026 05:47:31.726 7f6998f43448 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 03/23/2026 05:47:31.727 7f6998f43448 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 03/23/2026 05:47:32.541 7f699fefcf20 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:47:41.825 7f6998f03448 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001582 03/23/2026 05:47:41.826 7f6998f03448 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 03/23/2026 05:47:41.826 7f6998f43148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 03/23/2026 05:47:41.827 7f6998f43148 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 4 03/23/2026 05:47:41.827 7f6998f43148 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 03/23/2026 05:47:41.827 7f6998f43148 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 03/23/2026 05:47:41.827 7f6998f43148 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/23/2026 05:47:41.827 7f6998f43148 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/23/2026 05:47:41.828 7f6998f43448 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 03/23/2026 05:47:41.828 7f6998f43448 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 03/23/2026 05:47:45.509 7f699fefcf20 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=10702644, partition_space_available_1k=70387840, percentage_partition_space_used=13, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1327892, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11276, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44992, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7009904, free=201956, buffers=373820, cached=8899360, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.43, five_min_avg=6.4, fifteen_min_avg=3.81, cpu_share=11.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:48:32.547 7f699fefcf20 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:48:42.483 7f699fefcf20 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=10834388, partition_space_available_1k=70256096, percentage_partition_space_used=13, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1369864, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11324, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45044, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6945236, free=277452, buffers=378832, cached=8852776, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.65, five_min_avg=5.54, fifteen_min_avg=3.66, cpu_share=9.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:49:32.562 7f699fefcf20 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:49:41.501 7f699fefcf20 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=10884572, partition_space_available_1k=70205912, percentage_partition_space_used=13, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1353796, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11428, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45248, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7044684, free=316076, buffers=380456, cached=8715320, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.26, five_min_avg=4.91, fifteen_min_avg=3.56, cpu_share=8.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:50:32.553 7f699fefcf20 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:50:40.488 7f699fefcf20 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=10899804, partition_space_available_1k=70190680, percentage_partition_space_used=13, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1357012, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11816, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45260, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7267900, free=235592, buffers=380360, cached=8574080, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.12, five_min_avg=4.38, fifteen_min_avg=3.46, cpu_share=7.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:51:32.561 7f699fefcf20 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:51:40.515 7f699fefcf20 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=11905368, partition_space_available_1k=69185116, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1389404, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12060, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45256, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6045336, free=1685016, buffers=405596, cached=8336320, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.35, five_min_avg=4.49, fifteen_min_avg=3.55, cpu_share=11.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:51:52.589 7f6998f03248 [contrail-database-nodemgr] [DEBUG]: DBG: event_process_state:contrail-query-engine,group:contrail-query-engine,state:PROCESS_STATE_EXITED 03/23/2026 05:51:52.590 7f6998f03248 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-2.', _node_type='Database', _instance_id='0', _timestamp=1774245112590278, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774244607000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_EXITED', start_count=1, stop_count=0, exit_count=1, last_start_time='1774244607000000', last_stop_time='', last_exit_time='1774245112590162', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774244606000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-10-06 01:00:31.164147", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11344-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 03/23/2026 05:52:11.183 7fb114bc42c0 [contrail-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 03/23/2026 05:52:11.183 7fb114bc42c0 [contrail-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 03/23/2026 05:52:11.183 7fb114bc42c0 [contrail-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 03/23/2026 05:52:11.192 7fb112695ea0 [contrail-database-nodemgr] [ERROR]: Cannot write http_port 8103 to /tmp/contrail-database-nodemgr.0.http_port 03/23/2026 05:52:11.192 7fb112695ea0 [contrail-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8103 03/23/2026 05:52:11.193 7fb111afd2c0 [contrail-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 03/23/2026 05:52:11.194 7fb111afd2c0 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/23/2026 05:52:11.195 7fb111afd2c0 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/23/2026 05:52:11.195 7fb111afd2c0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 03/23/2026 05:52:11.195 7fb111afd2c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:52:11.195 7fb111afd2c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/23/2026 05:52:11.196 7fb111afd2c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:52:11.196 7fb111afd2c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:52:11.196 7fb111afd2c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1774245131195387 collector_name = collector_ip = collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 03/23/2026 05:52:11.196 7fb111afd2c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:52:11.293 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 3717405285892000432597807460057202528391712680429882219236216860113644153524, 'start': '1774245128000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-query-engine', 'group': 'contrail-query-engine', 'pid': 782750979036690746659459284965604153403911091453226841463743299941589495040, 'start': '1774245129000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-database-nodemgr', 'group': 'contrail-database-nodemgr', 'pid': 96543133415266656244745239289907852640933622171924347667024364045700501027423, 'start': '1774245129000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 03/23/2026 05:52:11.330 7fb111afd360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:52:11.330 7fb111afd360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. build_info = {"build-info": [{"build-version": "v0+nightly", "build-time": "2026-03-23 00:50:56.516868", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-14440-1.novalocal", "build-id": "v0+nightly", "build-number": "202603230008"}]} system_cpu_info = << num_socket = 1 num_cpu = 4 num_core_per_socket = 4 num_thread_per_core = 1 >> >> 03/23/2026 05:52:11.330 7fb111afd360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:52:11.331 7fb114bc42c0 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-2.', _node_type='Database', _instance_id='0', _timestamp=1774245131331013, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 03/23/2026 05:52:11.331 7fb114bc42c0 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-2.', _node_type='Database', _instance_id='0', _timestamp=1774245131331274, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774245128000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774245129000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774245129000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info": [{"build-version": "v0+nightly", "build-time": "2026-03-23 00:50:56.516868", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-14440-1.novalocal", "build-id": "v0+nightly", "build-number": "202603230008"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 03/23/2026 05:52:11.337 7fb111afd360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:52:11.337 7fb111afd360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/23/2026 05:52:11.337 7fb111afd360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:52:11.337 7fb111afd360 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:52:11.337 7fb111afd360 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774245128000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-query-engine process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774245129000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774245129000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info": [{"build-version": "v0+nightly", "build-time": "2026-03-23 00:50:56.516868", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-14440-1.novalocal", "build-id": "v0+nightly", "build-number": "202603230008"}]} >> 03/23/2026 05:52:11.337 7fb111afd360 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:52:11.564 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:52:11.973 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:DatabaseUsage] map 03/23/2026 05:52:11.974 7fb111afd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:52:11.975 7fb111afd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. database_usage = [ << disk_space_used_1k = 12000052 disk_space_available_1k = 69090432 analytics_db_size_1k = 15976 >>, ] >> 03/23/2026 05:52:11.975 7fb111afd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:52:12.327 7fb114bc42c0 [contrail-database-nodemgr] [ERROR]: Failed to connect to database by CQL and socket: Result: 1 03/23/2026 05:52:12.327 7fb114bc42c0 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-2.', _node_type='Database', _instance_id='0', _timestamp=1774245132327819, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 03/23/2026 05:52:12.329 7fb111afd7c0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:52:12.329 7fb111afd7c0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/23/2026 05:52:12.329 7fb111afd7c0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:52:14.116 7fb114bc42c0 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 2 03/23/2026 05:52:15.195 7fb111de19a0 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:52:15.196 7fb111de19a0 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 03/23/2026 05:52:15.197 7fb111de19a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:52:15.197 7fb111de19a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:52:15.197 7fb111de19a0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:52:15.197 7fb111de19a0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:52:15.197 7fb111de19a0 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 03/23/2026 05:52:15.198 7fb111de19a0 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 03/23/2026 05:52:15.198 7fb111de19a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 03/23/2026 05:52:15.199 7fb111de19a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:52:15.199 7fb111de19a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:52:15.203 7fb111add0e0 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001017 03/23/2026 05:52:15.204 7fb111add0e0 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 03/23/2026 05:52:15.204 7fb111de19a0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 03/23/2026 05:52:15.204 7fb111de19a0 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 03/23/2026 05:52:15.205 7fb111de19a0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 03/23/2026 05:52:15.205 7fb111de19a0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/23/2026 05:52:15.205 7fb111de19a0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/23/2026 05:52:15.206 7fb111de19a0 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 03/23/2026 05:52:15.207 7fb111afd680 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 03/23/2026 05:52:15.208 7fb111afd680 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 03/23/2026 05:52:19.155 7fb114bc42c0 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-2.', _node_type='Database', _instance_id='0', _timestamp=1774245139155532, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774245128000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774245129000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774245129000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info": [{"build-version": "v0+nightly", "build-time": "2026-03-23 00:50:56.516868", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-14440-1.novalocal", "build-id": "v0+nightly", "build-number": "202603230008"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 03/23/2026 05:52:19.156 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12000724, partition_space_available_1k=69089760, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1283124, cpu_share=0.0, mem_res=1274680, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=9504, cpu_share=0.0, mem_res=7608, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54624, cpu_share=0.0, mem_res=45132, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6384268, free=1182668, buffers=410000, cached=8519864, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.26, five_min_avg=4.73, fifteen_min_avg=3.67, cpu_share=0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:53:11.532 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:53:12.338 7fb114bc42c0 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-2.', _node_type='Database', _instance_id='0', _timestamp=1774245192337980, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.38:8086'], status='Up', description='ClientInit to Established on EvSandeshCtrlMessageRecv')]), flag_infos=None, description='')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 03/23/2026 05:53:14.470 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-2.', _node_type='Database', _instance_id='0', _timestamp=1774245194470167, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0)], _table='ObjectDatabaseInfo'), _seqnum=0) 03/23/2026 05:53:14.470 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 03/23/2026 05:53:19.507 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12017944, partition_space_available_1k=69072540, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1348440, cpu_share=0.31, mem_res=1328768, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10980, cpu_share=0.0, mem_res=9076, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54608, cpu_share=0.01, mem_res=45128, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6611912, free=944768, buffers=410060, cached=8529644, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.82, five_min_avg=4.14, fifteen_min_avg=3.53, cpu_share=9.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:54:11.526 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:54:19.166 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12014908, partition_space_available_1k=69075576, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1349712, cpu_share=0.16, mem_res=1326468, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11192, cpu_share=0.0, mem_res=9284, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54612, cpu_share=0.01, mem_res=45128, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6669792, free=882384, buffers=410076, cached=8534124, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.75, five_min_avg=3.58, fifteen_min_avg=3.37, cpu_share=7.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:55:11.536 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:55:19.053 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12019876, partition_space_available_1k=69070608, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1352320, cpu_share=0.1, mem_res=1326968, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11204, cpu_share=0.0, mem_res=9296, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54620, cpu_share=0.01, mem_res=45128, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6635728, free=910368, buffers=410080, cached=8539256, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.19, five_min_avg=3.08, fifteen_min_avg=3.21, cpu_share=7.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:56:11.578 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:56:19.143 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12020524, partition_space_available_1k=69069960, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1356384, cpu_share=0.13, mem_res=1328240, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11232, cpu_share=0.0, mem_res=9324, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54620, cpu_share=0.01, mem_res=45128, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6620188, free=925164, buffers=410092, cached=8539952, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.03, five_min_avg=2.68, fifteen_min_avg=3.06, cpu_share=7.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:57:11.559 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:57:19.124 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12028280, partition_space_available_1k=69062204, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1399376, cpu_share=0.11, mem_res=1368592, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11260, cpu_share=0.0, mem_res=9352, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54640, cpu_share=0.01, mem_res=45128, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6675324, free=862208, buffers=410092, cached=8547720, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.96, five_min_avg=2.35, fifteen_min_avg=2.92, cpu_share=7.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:58:11.577 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:58:19.111 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12029908, partition_space_available_1k=69060576, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1376084, cpu_share=0.11, mem_res=1342356, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10624, cpu_share=0.0, mem_res=8716, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54640, cpu_share=0.01, mem_res=45128, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6702048, free=833472, buffers=410100, cached=8549736, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.27, five_min_avg=2.2, fifteen_min_avg=2.84, cpu_share=7.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 05:59:11.582 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 05:59:19.198 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12035368, partition_space_available_1k=69055116, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1374768, cpu_share=0.1, mem_res=1338340, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10728, cpu_share=0.0, mem_res=8820, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54640, cpu_share=0.01, mem_res=45128, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6669876, free=860148, buffers=410104, cached=8555252, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.22, five_min_avg=2.02, fifteen_min_avg=2.74, cpu_share=6.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:00:11.609 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:00:19.251 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12035776, partition_space_available_1k=69054708, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1398620, cpu_share=0.1, mem_res=1359956, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10728, cpu_share=0.0, mem_res=8820, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54644, cpu_share=0.01, mem_res=45128, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6728860, free=800708, buffers=410108, cached=8555676, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.93, five_min_avg=1.8, fifteen_min_avg=2.61, cpu_share=7.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:01:11.630 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:01:19.131 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12037172, partition_space_available_1k=69053312, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1391580, cpu_share=0.1, mem_res=1349968, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10728, cpu_share=0.0, mem_res=8820, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54652, cpu_share=0.01, mem_res=45128, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6741852, free=786316, buffers=410108, cached=8557092, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.24, five_min_avg=1.75, fifteen_min_avg=2.55, cpu_share=6.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:02:11.637 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:02:19.056 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12042620, partition_space_available_1k=69047864, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1394396, cpu_share=0.09, mem_res=1350264, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10752, cpu_share=0.0, mem_res=8844, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54652, cpu_share=0.01, mem_res=45128, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6733216, free=789360, buffers=410108, cached=8562756, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.38, five_min_avg=1.68, fifteen_min_avg=2.47, cpu_share=6.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:03:11.647 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:03:19.148 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12044188, partition_space_available_1k=69046296, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1396488, cpu_share=0.09, mem_res=1349672, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10752, cpu_share=0.0, mem_res=8844, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54656, cpu_share=0.01, mem_res=45128, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6748528, free=770780, buffers=410108, cached=8565904, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.14, five_min_avg=1.55, fifteen_min_avg=2.37, cpu_share=7.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:04:11.647 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:04:19.467 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12049464, partition_space_available_1k=69041020, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1419700, cpu_share=0.1, mem_res=1370368, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10824, cpu_share=0.0, mem_res=8916, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54656, cpu_share=0.01, mem_res=45128, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6781788, free=729348, buffers=410156, cached=8574096, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.79, five_min_avg=1.64, fifteen_min_avg=2.35, cpu_share=7.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:05:11.654 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:05:19.301 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12049912, partition_space_available_1k=69040572, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1420192, cpu_share=0.09, mem_res=1368508, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10872, cpu_share=0.0, mem_res=8964, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54664, cpu_share=0.01, mem_res=45128, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6784144, free=726536, buffers=410156, cached=8574552, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.25, five_min_avg=1.51, fifteen_min_avg=2.26, cpu_share=6.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:06:11.664 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:06:19.286 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12055084, partition_space_available_1k=69035400, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1424296, cpu_share=0.09, mem_res=1370044, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10904, cpu_share=0.0, mem_res=8996, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54668, cpu_share=0.01, mem_res=45128, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6753688, free=751736, buffers=410156, cached=8579820, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.09, five_min_avg=1.41, fifteen_min_avg=2.18, cpu_share=6.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:07:11.688 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:07:19.422 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12055776, partition_space_available_1k=69034708, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1413396, cpu_share=0.09, mem_res=1356464, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10968, cpu_share=0.0, mem_res=9060, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54668, cpu_share=0.01, mem_res=45128, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6756456, free=748132, buffers=410164, cached=8580644, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.32, five_min_avg=1.41, fifteen_min_avg=2.13, cpu_share=6.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:07:52.114 7fb11281aa40 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/23/2026 06:07:57.135 7fb11281aa40 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/23/2026 06:08:11.771 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:08:19.263 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12201808, partition_space_available_1k=68888676, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1443964, cpu_share=0.1, mem_res=1383616, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11088, cpu_share=0.0, mem_res=9180, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54668, cpu_share=0.01, mem_res=45128, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6814624, free=532648, buffers=413116, cached=8734032, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.43, five_min_avg=1.44, fifteen_min_avg=2.09, cpu_share=7.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:09:11.685 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:09:19.171 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12207400, partition_space_available_1k=68883084, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1432436, cpu_share=0.09, mem_res=1369224, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11164, cpu_share=0.0, mem_res=9256, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54672, cpu_share=0.01, mem_res=45128, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6782480, free=557972, buffers=413140, cached=8740808, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.41, five_min_avg=1.43, fifteen_min_avg=2.05, cpu_share=6.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:09:28.826 7fb11281aa40 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/23/2026 06:10:11.740 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:10:19.843 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12213692, partition_space_available_1k=68876792, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1462404, cpu_share=0.13, mem_res=1393848, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13156, cpu_share=0.0, mem_res=11216, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54676, cpu_share=0.01, mem_res=45128, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6881744, free=449932, buffers=413148, cached=8749364, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.27, five_min_avg=1.59, fifteen_min_avg=2.05, cpu_share=8.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:11:11.762 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:11:20.488 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12230552, partition_space_available_1k=68859932, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1504616, cpu_share=0.17, mem_res=1423064, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13208, cpu_share=0.0, mem_res=11272, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54688, cpu_share=0.01, mem_res=45128, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6932208, free=383000, buffers=413152, cached=8765948, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.23, five_min_avg=1.88, fifteen_min_avg=2.12, cpu_share=9.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:12:11.770 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:12:19.998 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12239284, partition_space_available_1k=68851200, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1504568, cpu_share=0.13, mem_res=1416168, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13260, cpu_share=0.0, mem_res=11324, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54684, cpu_share=0.01, mem_res=45128, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6941860, free=362328, buffers=413156, cached=8776000, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.71, five_min_avg=2.17, fifteen_min_avg=2.19, cpu_share=8.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:13:11.727 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:13:20.066 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12262116, partition_space_available_1k=68828368, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1518532, cpu_share=0.17, mem_res=1415312, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13408, cpu_share=0.0, mem_res=11468, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54716, cpu_share=0.01, mem_res=45128, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6956816, free=357908, buffers=412372, cached=8765724, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.06, five_min_avg=1.98, fifteen_min_avg=2.13, cpu_share=9.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:14:11.726 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:14:19.982 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12278720, partition_space_available_1k=68811764, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1538260, cpu_share=0.13, mem_res=1426936, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13500, cpu_share=0.0, mem_res=11560, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54712, cpu_share=0.01, mem_res=45128, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6990652, free=349280, buffers=412180, cached=8740768, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.84, five_min_avg=1.99, fifteen_min_avg=2.13, cpu_share=8.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:15:11.734 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:15:19.664 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12306744, partition_space_available_1k=68783740, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1636688, cpu_share=0.28, mem_res=1488852, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13560, cpu_share=0.0, mem_res=11620, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54688, cpu_share=0.01, mem_res=45128, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7070428, free=346100, buffers=411356, cached=8666052, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.13, five_min_avg=2.1, fifteen_min_avg=2.16, cpu_share=9.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:16:11.738 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:16:19.709 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12318036, partition_space_available_1k=68772448, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1613228, cpu_share=0.14, mem_res=1458060, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13564, cpu_share=0.0, mem_res=11624, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54684, cpu_share=0.01, mem_res=45128, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7091172, free=343116, buffers=410312, cached=8649520, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.01, five_min_avg=2.65, fifteen_min_avg=2.34, cpu_share=8.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:17:11.762 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:17:19.628 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12322948, partition_space_available_1k=68767536, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1620692, cpu_share=0.12, mem_res=1459912, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13604, cpu_share=0.0, mem_res=11664, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54688, cpu_share=0.01, mem_res=45128, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7074908, free=352432, buffers=410312, cached=8656352, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.59, five_min_avg=2.47, fifteen_min_avg=2.29, cpu_share=7.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:18:11.748 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:18:19.270 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12336024, partition_space_available_1k=68754460, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1644320, cpu_share=0.11, mem_res=1478476, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13656, cpu_share=0.0, mem_res=11708, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54676, cpu_share=0.01, mem_res=45128, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7110400, free=350832, buffers=408952, cached=8624332, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.65, five_min_avg=2.24, fifteen_min_avg=2.23, cpu_share=7.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:19:11.759 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:19:19.363 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12338676, partition_space_available_1k=68751808, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1650340, cpu_share=0.12, mem_res=1478336, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13656, cpu_share=0.0, mem_res=11708, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54672, cpu_share=0.01, mem_res=45128, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7114216, free=358392, buffers=408880, cached=8612936, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.31, five_min_avg=2.48, fifteen_min_avg=2.3, cpu_share=7.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:20:11.799 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:20:19.476 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12359896, partition_space_available_1k=68730588, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1735116, cpu_share=0.15, mem_res=1543588, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13664, cpu_share=0.0, mem_res=11716, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54640, cpu_share=0.01, mem_res=45128, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7176228, free=357900, buffers=408092, cached=8553140, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.45, five_min_avg=2.38, fifteen_min_avg=2.28, cpu_share=7.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:21:07.976 7fb111de15e0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 03/23/2026 06:21:11.788 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:21:19.330 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12368964, partition_space_available_1k=68721520, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1695288, cpu_share=0.12, mem_res=1497428, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13748, cpu_share=0.0, mem_res=11800, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54640, cpu_share=0.01, mem_res=45128, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7198636, free=345436, buffers=408780, cached=8542552, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.61, five_min_avg=2.44, fifteen_min_avg=2.3, cpu_share=8.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:22:11.754 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:22:19.314 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12375104, partition_space_available_1k=68715380, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1711328, cpu_share=0.11, mem_res=1508476, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13688, cpu_share=0.0, mem_res=11740, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54648, cpu_share=0.01, mem_res=45128, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7196736, free=338800, buffers=409032, cached=8551016, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.48, five_min_avg=2.47, fifteen_min_avg=2.32, cpu_share=6.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:23:11.768 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 03/23/2026 06:23:19.360 7fb114bc42c0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12382616, partition_space_available_1k=68707868, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1705240, cpu_share=0.09, mem_res=1499312, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=13704, cpu_share=0.0, mem_res=11756, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54644, cpu_share=0.01, mem_res=45128, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7208116, free=342200, buffers=409040, cached=8536348, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.93, five_min_avg=2.32, fifteen_min_avg=2.28, cpu_share=6.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 03/23/2026 06:23:41.947 7fb111afdc20 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map