09/29/2025 01:42:36.438 7fb3293f3b80 [contrail-config-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/29/2025 01:42:36.438 7fb3293f3b80 [contrail-config-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/29/2025 01:42:36.438 7fb3293f3b80 [contrail-config-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/29/2025 01:42:36.446 7fb3264904a0 [contrail-config-database-nodemgr] [ERROR]: Cannot write http_port 8112 to /tmp/contrail-config-database-nodemgr.0.http_port 09/29/2025 01:42:36.446 7fb3264904a0 [contrail-config-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8112 09/29/2025 01:42:36.447 7fb326490860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 09/29/2025 01:42:36.448 7fb326490860 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/29/2025 01:42:36.448 7fb326490860 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/29/2025 01:42:36.448 7fb326490860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/29/2025 01:42:36.448 7fb326490860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:42:36.448 7fb326490860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/29/2025 01:42:36.448 7fb326490860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:42:36.448 7fb326490860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:42:36.448 7fb326490860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1759110156448298 collector_name = collector_ip = collector_list = [ 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 09/29/2025 01:42:36.448 7fb326490860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:42:36.566 7fb3293f3b80 [contrail-config-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 25443398105828583780549240361589921887691238637190543780208572130935551250177, 'start': '1759110154000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'zookeeper', 'group': 'zookeeper', 'pid': 38073235472587093846299243980135533558400554681681335097068489088890961403624, 'start': '1759110154000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-config-database-nodemgr', 'group': 'contrail-config-database-nodemgr', 'pid': 107160102983135029937231455051368795331405237412388215930185367129325608444330, 'start': '1759110155000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 09/29/2025 01:42:36.638 7fb3267b90e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:42:36.638 7fb3267b90e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. build_info = {"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-29 00:11:25.017323", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11319-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed "}]} system_cpu_info = << num_socket = 1 num_cpu = 8 num_core_per_socket = 8 num_thread_per_core = 1 >> running_package_version = package contrail-lib is not installed installed_package_version = package contrail-lib is not installed >> 09/29/2025 01:42:36.639 7fb3267b90e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:42:36.639 7fb3293f3b80 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-4200-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1759110156639626, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4200-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 09/29/2025 01:42:36.640 7fb3293f3b80 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-4200-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1759110156640044, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4200-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1759110154000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1759110154000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1759110155000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-29 00:11:25.017323", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11319-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed\n"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 09/29/2025 01:42:36.643 7fb3267b90e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:42:36.643 7fb3267b90e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/29/2025 01:42:36.643 7fb3267b90e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:42:36.643 7fb3267b90e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:42:36.643 7fb3267b90e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1759110154000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = zookeeper process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1759110154000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1759110155000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-29 00:11:25.017323", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11319-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed "}]} >> 09/29/2025 01:42:36.643 7fb3267b90e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:42:36.878 7fb3293f3b80 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/29/2025 01:42:37.141 7fb3293f3b80 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 09/29/2025 01:42:37.143 7fb3267b9220 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:42:37.143 7fb3267b9220 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. database_usage = [ << disk_space_used_1k = 13668352 disk_space_available_1k = 67422132 config_db_size_1k = 4 >>, ] >> 09/29/2025 01:42:37.143 7fb3267b9220 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:42:37.280 7fb3293f3b80 [contrail-config-database-nodemgr] [ERROR]: Failed to connect to database by CQL and socket: Result: 1 09/29/2025 01:42:37.281 7fb3293f3b80 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-4200-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1759110157281163, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4200-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 09/29/2025 01:42:37.282 7fb3267b9220 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:42:37.282 7fb3267b9220 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/29/2025 01:42:37.282 7fb3267b9220 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:42:38.082 7fb3293f3b80 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/29/2025 01:42:39.122 7fb3293f3b80 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr config_db_uuid'. Error is: Result: 1 09/29/2025 01:42:40.001 7fb3293f3b80 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr to_bgp_keyspace'. Error is: Result: 1 09/29/2025 01:42:40.449 7fb3267b94a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:42:40.450 7fb3267b94a0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:42:40.450 7fb3267b94a0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:42:40.450 7fb3267b94a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:42:40.451 7fb3267b94a0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:42:40.451 7fb3267b94a0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:42:40.451 7fb3267b94a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:42:40.451 7fb3267b94a0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:42:40.451 7fb3267b94a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:42:40.452 7fb3267b94a0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:42:40.452 7fb3267b94a0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:42:40.452 7fb3267b94a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:42:40.452 7fb3267b94a0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1759110156448298 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:42:40.452 7fb3267b94a0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:42:40.784 7fb3293f3b80 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr svc_monitor_keyspace'. Error is: Result: 1 09/29/2025 01:42:41.501 7fb3293f3b80 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr DISCOVERY_SERVER'. Error is: Result: 1 09/29/2025 01:42:42.213 7fb3293f3b80 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr useragent'. Error is: Result: 1 09/29/2025 01:42:42.981 7fb3293f3b80 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr dm_keyspace'. Error is: Result: 1 09/29/2025 01:42:44.451 7fb3267a0540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:42:44.452 7fb3267a0540 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:42:44.452 7fb3267a0540 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:42:44.453 7fb3267a0540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:42:44.453 7fb3267a0540 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:42:44.453 7fb3267a0540 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:42:44.453 7fb3267a0540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:42:44.453 7fb3267a0540 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:42:44.453 7fb3267a0540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:42:44.453 7fb3267a0540 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:42:44.454 7fb3267a0540 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:42:44.454 7fb3267a0540 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:42:44.454 7fb3267a0540 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1759110156448298 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:42:44.454 7fb3267a0540 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:42:48.009 7fb3293f3b80 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-4200-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1759110168009522, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4200-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1759110154000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1759110154000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1759110155000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-29 00:11:25.017323", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11319-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed\n"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 09/29/2025 01:42:48.009 7fb3293f3b80 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4200-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13847976, partition_space_available_1k=67242508, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1275376, cpu_share=0.0, mem_res=1264456, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54688, cpu_share=0.0, mem_res=52400, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52664, cpu_share=0.0, mem_res=42672, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=5303712, free=13336456, buffers=611732, cached=13955776, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.4, five_min_avg=1.72, fifteen_min_avg=1.13, cpu_share=0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/29/2025 01:42:48.010 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:42:48.010 7fb326490c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1759110154000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = zookeeper process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1759110154000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1759110155000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-09-29 00:11:25.017323", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11319-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed "}]} >> 09/29/2025 01:42:48.010 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:42:48.010 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:42:48.010 7fb326490c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 13847976 partition_space_available_1k = 67242508 percentage_partition_space_used = 17 >> } all_core_file_list = [ ] process_mem_cpu_usage = { cassandra : << mem_virt = 1275376 cpu_share = 0.0 mem_res = 1264456 >>zookeeper : << mem_virt = 54688 cpu_share = 0.0 mem_res = 52400 >>contrail-config-database-nodemgr : << mem_virt = 52664 cpu_share = 0.0 mem_res = 42672 >> } system_mem_usage = << total = 32665120 used = 5303712 free = 13336456 buffers = 611732 cached = 13955776 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 2.4 five_min_avg = 1.72 fifteen_min_avg = 1.13 cpu_share = 0 node_type = config-database-node >> >> 09/29/2025 01:42:48.010 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:42:48.453 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:42:48.454 7fb326490c20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:42:48.454 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:42:48.454 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:42:48.454 7fb326490c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:42:48.454 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:42:48.455 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:42:48.455 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:42:48.455 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:42:48.455 7fb326490c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:42:48.455 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:42:48.455 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:42:48.455 7fb326490c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1759110156448298 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:42:48.455 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:42:52.455 7fb3267b9720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:42:52.456 7fb3267b9720 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:42:52.457 7fb3267b9720 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:42:52.457 7fb3267b9720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:42:52.457 7fb3267b9720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:42:52.457 7fb3267b9720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:42:52.457 7fb3267b9720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:42:52.457 7fb3267b9720 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:42:52.458 7fb3267b9720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:42:52.458 7fb3267b9720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:42:52.458 7fb3267b9720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:42:52.458 7fb3267b9720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:42:52.458 7fb3267b9720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1759110156448298 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:42:52.458 7fb3267b9720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:42:56.461 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:42:56.462 7fb326490d60 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:42:56.462 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:42:56.463 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:42:56.463 7fb326490d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:42:56.463 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:42:56.463 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:42:56.463 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:42:56.463 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:42:56.463 7fb326490d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:42:56.463 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:42:56.463 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:42:56.464 7fb326490d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1759110156448298 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:42:56.464 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:00.467 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:43:00.468 7fb326490d60 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:43:00.468 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:43:00.468 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:43:00.468 7fb326490d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:43:00.468 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:43:00.468 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:43:00.469 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:43:00.469 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:00.469 7fb326490d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:43:00.469 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:00.469 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:00.469 7fb326490d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1759110156448298 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:43:00.469 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:04.472 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:43:04.473 7fb326490d60 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:43:04.473 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:43:04.473 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:43:04.473 7fb326490d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:43:04.473 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:43:04.473 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:43:04.473 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:43:04.473 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:04.474 7fb326490d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:43:04.474 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:04.474 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:04.474 7fb326490d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1759110156448298 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:43:04.474 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:08.473 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:43:08.474 7fb326490e00 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:43:08.475 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:43:08.475 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:43:08.475 7fb326490e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:43:08.475 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:43:08.475 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:43:08.475 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:43:08.475 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:08.476 7fb326490e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:43:08.476 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:08.476 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:08.476 7fb326490e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1759110156448298 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:43:08.476 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:12.478 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:43:12.479 7fb326490e00 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:43:12.480 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:43:12.480 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:43:12.480 7fb326490e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:43:12.480 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:43:12.480 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:43:12.480 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:43:12.480 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:12.480 7fb326490e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:43:12.481 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:12.481 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:12.481 7fb326490e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1759110156448298 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:43:12.481 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:16.484 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:43:16.485 7fb326490e00 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:43:16.485 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:43:16.486 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:43:16.486 7fb326490e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:43:16.486 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:43:16.486 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:43:16.487 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:43:16.487 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:16.487 7fb326490e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:43:16.487 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:16.487 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:16.487 7fb326490e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1759110156448298 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:43:16.487 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:20.486 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:43:20.487 7fb326490e00 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:43:20.487 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:43:20.487 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:43:20.488 7fb326490e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:43:20.488 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:43:20.488 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:43:20.488 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:43:20.488 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:20.488 7fb326490e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:43:20.488 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:20.488 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:20.488 7fb326490e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1759110156448298 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:43:20.489 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:24.490 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:43:24.491 7fb326490c20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:43:24.491 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:43:24.491 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:43:24.491 7fb326490c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:43:24.492 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:43:24.492 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:43:24.492 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:43:24.492 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:24.492 7fb326490c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:43:24.493 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:24.493 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:24.493 7fb326490c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1759110156448298 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:43:24.493 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:28.495 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:43:28.496 7fb326490c20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:43:28.496 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:43:28.496 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:43:28.496 7fb326490c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:43:28.497 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:43:28.497 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:43:28.497 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:43:28.497 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:28.497 7fb326490c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:43:28.497 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:28.497 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:28.497 7fb326490c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1759110156448298 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:43:28.497 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:32.498 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:43:32.499 7fb326490c20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:43:32.499 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:43:32.499 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:43:32.500 7fb326490c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:43:32.500 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:43:32.500 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:43:32.500 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:43:32.500 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:32.500 7fb326490c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:43:32.500 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:32.500 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:32.500 7fb326490c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1759110156448298 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:43:32.501 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:36.502 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:43:36.503 7fb326490c20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:43:36.504 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:43:36.504 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:43:36.504 7fb326490c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:43:36.504 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:43:36.504 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:43:36.504 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:43:36.504 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:36.504 7fb326490c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:43:36.505 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:36.505 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:36.505 7fb326490c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1759110156448298 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:43:36.505 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:36.810 7fb3293f3b80 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/29/2025 01:43:37.049 7fb3267b9860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:37.050 7fb3267b9860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. database_usage = [ << disk_space_used_1k = 14786628 disk_space_available_1k = 66303856 config_db_size_1k = 956 >>, ] >> 09/29/2025 01:43:37.050 7fb3267b9860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:37.199 7fb3293f3b80 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/29/2025 01:43:37.200 7fb3293f3b80 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-4200-1.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1759110217200196, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4200-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.49:8086'], status='Down', description='Connect to Idle on EvTcpConnectFail')]), flag_infos=None, description='Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 09/29/2025 01:43:37.201 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:37.201 7fb326490c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:43:37.201 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:37.882 7fb3293f3b80 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/29/2025 01:43:40.504 7fb3267a0360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:43:40.505 7fb3267a0360 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:43:40.505 7fb3267a0360 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:43:40.505 7fb3267a0360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:43:40.506 7fb3267a0360 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:43:40.506 7fb3267a0360 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:43:40.506 7fb3267a0360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:43:40.506 7fb3267a0360 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:43:40.506 7fb3267a0360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:40.506 7fb3267a0360 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:43:40.506 7fb3267a0360 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:40.506 7fb3267a0360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:40.506 7fb3267a0360 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1759110156448298 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:43:40.507 7fb3267a0360 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:42.913 7fb3293f3b80 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4200-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14786408, partition_space_available_1k=66304076, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1344780, cpu_share=0.26, mem_res=1330920, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=73340, cpu_share=0.01, mem_res=70220, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52768, cpu_share=0.01, mem_res=42672, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=6291288, free=11603936, buffers=651772, cached=14661148, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.28, five_min_avg=2.08, fifteen_min_avg=1.28, cpu_share=3.12, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/29/2025 01:43:42.913 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:42.914 7fb326490c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 14786408 partition_space_available_1k = 66304076 percentage_partition_space_used = 18 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1344780 cpu_share = 0.26 mem_res = 1330920 >>zookeeper : << mem_virt = 73340 cpu_share = 0.01 mem_res = 70220 >>contrail-config-database-nodemgr : << mem_virt = 52768 cpu_share = 0.01 mem_res = 42672 >> } system_mem_usage = << total = 32665120 used = 6291288 free = 11603936 buffers = 651772 cached = 14661148 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 3.28 five_min_avg = 2.08 fifteen_min_avg = 1.28 cpu_share = 3.12 node_type = config-database-node >> >> 09/29/2025 01:43:42.914 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:44.506 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:43:44.508 7fb326490c20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:43:44.508 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:43:44.508 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:43:44.508 7fb326490c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:43:44.508 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:43:44.510 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:43:44.510 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:43:44.511 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:44.511 7fb326490c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:43:44.511 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:44.511 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:44.511 7fb326490c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1759110156448298 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:43:44.511 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:48.512 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:43:48.513 7fb326490c20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:43:48.514 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:43:48.514 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:43:48.514 7fb326490c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:43:48.514 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:43:48.514 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:43:48.515 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:43:48.515 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:48.515 7fb326490c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:43:48.515 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:48.515 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:48.515 7fb326490c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1759110156448298 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:43:48.515 7fb326490c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:52.514 7fb3267b97c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:43:52.515 7fb3267b97c0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:43:52.515 7fb3267b97c0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:43:52.515 7fb3267b97c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:43:52.515 7fb3267b97c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:43:52.516 7fb3267b97c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:43:52.516 7fb3267b97c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:43:52.516 7fb3267b97c0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:43:52.516 7fb3267b97c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:52.516 7fb3267b97c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:43:52.516 7fb3267b97c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:52.516 7fb3267b97c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:52.516 7fb3267b97c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1759110156448298 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:43:52.516 7fb3267b97c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:56.518 7fb3267b97c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:43:56.519 7fb3267b97c0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:43:56.519 7fb3267b97c0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:43:56.520 7fb3267b97c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:43:56.520 7fb3267b97c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:43:56.520 7fb3267b97c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:43:56.520 7fb3267b97c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:43:56.520 7fb3267b97c0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:43:56.520 7fb3267b97c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:56.520 7fb3267b97c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:43:56.520 7fb3267b97c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:56.521 7fb3267b97c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:43:56.521 7fb3267b97c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1759110156448298 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:43:56.521 7fb3267b97c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:00.522 7fb3267b97c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:44:00.523 7fb3267b97c0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:44:00.523 7fb3267b97c0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:44:00.524 7fb3267b97c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:44:00.524 7fb3267b97c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:44:00.524 7fb3267b97c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:44:00.524 7fb3267b97c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:44:00.524 7fb3267b97c0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:44:00.524 7fb3267b97c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:00.524 7fb3267b97c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:44:00.524 7fb3267b97c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:00.524 7fb3267b97c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:00.525 7fb3267b97c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1759110156448298 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:44:00.525 7fb3267b97c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:04.526 7fb3267b97c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:44:04.527 7fb3267b97c0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:44:04.527 7fb3267b97c0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:44:04.528 7fb3267b97c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:44:04.528 7fb3267b97c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:44:04.528 7fb3267b97c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:44:04.528 7fb3267b97c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:44:04.528 7fb3267b97c0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:44:04.528 7fb3267b97c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:04.528 7fb3267b97c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:44:04.528 7fb3267b97c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:04.528 7fb3267b97c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:04.529 7fb3267b97c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1759110156448298 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:44:04.529 7fb3267b97c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:08.529 7fb3267b9c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:44:08.530 7fb3267b9c20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:44:08.530 7fb3267b9c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:44:08.530 7fb3267b9c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:44:08.530 7fb3267b9c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:44:08.531 7fb3267b9c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:44:08.531 7fb3267b9c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:44:08.531 7fb3267b9c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:44:08.531 7fb3267b9c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:08.531 7fb3267b9c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:44:08.531 7fb3267b9c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:08.531 7fb3267b9c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:08.531 7fb3267b9c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1759110156448298 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:44:08.531 7fb3267b9c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:12.534 7fb3267b9c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:44:12.535 7fb3267b9c20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:44:12.536 7fb3267b9c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:44:12.536 7fb3267b9c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:44:12.536 7fb3267b9c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:44:12.536 7fb3267b9c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:44:12.536 7fb3267b9c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:44:12.536 7fb3267b9c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:44:12.536 7fb3267b9c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:12.536 7fb3267b9c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:44:12.537 7fb3267b9c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:12.537 7fb3267b9c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:12.537 7fb3267b9c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1759110156448298 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:44:12.537 7fb3267b9c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:16.538 7fb3267b9c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:44:16.539 7fb3267b9c20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:44:16.539 7fb3267b9c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:44:16.540 7fb3267b9c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:44:16.540 7fb3267b9c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:44:16.540 7fb3267b9c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:44:16.540 7fb3267b9c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:44:16.540 7fb3267b9c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:44:16.540 7fb3267b9c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:16.540 7fb3267b9c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:44:16.540 7fb3267b9c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:16.541 7fb3267b9c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:16.541 7fb3267b9c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1759110156448298 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:44:16.541 7fb3267b9c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:20.543 7fb3267b9c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:44:20.543 7fb3267b9c20 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:44:20.544 7fb3267b9c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:44:20.544 7fb3267b9c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:44:20.544 7fb3267b9c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:44:20.544 7fb3267b9c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:44:20.544 7fb3267b9c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:44:20.544 7fb3267b9c20 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:44:20.544 7fb3267b9c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:20.544 7fb3267b9c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:44:20.545 7fb3267b9c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:20.545 7fb3267b9c20 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:20.545 7fb3267b9c20 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1759110156448298 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:44:20.545 7fb3267b9c20 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:24.545 7fb3267b95e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:44:24.546 7fb3267b95e0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:44:24.546 7fb3267b95e0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:44:24.546 7fb3267b95e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:44:24.546 7fb3267b95e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:44:24.546 7fb3267b95e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:44:24.546 7fb3267b95e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:44:24.547 7fb3267b95e0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:44:24.547 7fb3267b95e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:24.547 7fb3267b95e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:44:24.547 7fb3267b95e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:24.547 7fb3267b95e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:24.547 7fb3267b95e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1759110156448298 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:44:24.547 7fb3267b95e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:28.548 7fb3267b95e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:44:28.549 7fb3267b95e0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:44:28.549 7fb3267b95e0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:44:28.549 7fb3267b95e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:44:28.549 7fb3267b95e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:44:28.549 7fb3267b95e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:44:28.549 7fb3267b95e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:44:28.550 7fb3267b95e0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:44:28.550 7fb3267b95e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:28.550 7fb3267b95e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:44:28.550 7fb3267b95e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:28.550 7fb3267b95e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:28.550 7fb3267b95e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1759110156448298 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:44:28.550 7fb3267b95e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:32.550 7fb3267b95e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:44:32.551 7fb3267b95e0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:44:32.552 7fb3267b95e0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:44:32.552 7fb3267b95e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:44:32.552 7fb3267b95e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:44:32.552 7fb3267b95e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:44:32.552 7fb3267b95e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:44:32.552 7fb3267b95e0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:44:32.552 7fb3267b95e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:32.553 7fb3267b95e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:44:32.553 7fb3267b95e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:32.553 7fb3267b95e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:32.553 7fb3267b95e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1759110156448298 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:44:32.553 7fb3267b95e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:36.555 7fb3267b95e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:44:36.556 7fb3267b95e0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:44:36.556 7fb3267b95e0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:44:36.556 7fb3267b95e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:44:36.556 7fb3267b95e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:44:36.556 7fb3267b95e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:44:36.556 7fb3267b95e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:44:36.556 7fb3267b95e0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:44:36.557 7fb3267b95e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:36.557 7fb3267b95e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:44:36.557 7fb3267b95e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:36.557 7fb3267b95e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:36.557 7fb3267b95e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1759110156448298 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:44:36.557 7fb3267b95e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:36.843 7fb3293f3b80 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/29/2025 01:44:37.098 7fb3267b9680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:37.099 7fb3267b9680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. database_usage = [ << disk_space_used_1k = 16495812 disk_space_available_1k = 64594672 config_db_size_1k = 956 >>, ] >> 09/29/2025 01:44:37.099 7fb3267b9680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:37.221 7fb3293f3b80 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/29/2025 01:44:37.905 7fb3293f3b80 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/29/2025 01:44:40.558 7fb3267a0cc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:44:40.559 7fb3267a0cc0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:44:40.559 7fb3267a0cc0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:44:40.560 7fb3267a0cc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:44:40.560 7fb3267a0cc0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:44:40.560 7fb3267a0cc0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:44:40.560 7fb3267a0cc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:44:40.560 7fb3267a0cc0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:44:40.560 7fb3267a0cc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:40.560 7fb3267a0cc0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:44:40.561 7fb3267a0cc0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:40.561 7fb3267a0cc0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:40.561 7fb3267a0cc0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1759110156448298 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:44:40.561 7fb3267a0cc0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:42.935 7fb3293f3b80 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4200-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=16495684, partition_space_available_1k=64594800, percentage_partition_space_used=20, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1342336, cpu_share=0.04, mem_res=1329228, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=73376, cpu_share=0.0, mem_res=70260, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52860, cpu_share=0.01, mem_res=42672, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=7842880, free=8173816, buffers=659344, cached=16466696, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.99, five_min_avg=2.2, fifteen_min_avg=1.37, cpu_share=2.62, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/29/2025 01:44:42.935 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:42.936 7fb326490d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 16495684 partition_space_available_1k = 64594800 percentage_partition_space_used = 20 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1342336 cpu_share = 0.04 mem_res = 1329228 >>zookeeper : << mem_virt = 73376 cpu_share = 0.0 mem_res = 70260 >>contrail-config-database-nodemgr : << mem_virt = 52860 cpu_share = 0.01 mem_res = 42672 >> } system_mem_usage = << total = 32665120 used = 7842880 free = 8173816 buffers = 659344 cached = 16466696 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 2.99 five_min_avg = 2.2 fifteen_min_avg = 1.37 cpu_share = 2.62 node_type = config-database-node >> >> 09/29/2025 01:44:42.936 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:44.561 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:44:44.562 7fb326490d60 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:44:44.562 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:44:44.563 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:44:44.563 7fb326490d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:44:44.563 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:44:44.563 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:44:44.563 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:44:44.563 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:44.563 7fb326490d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:44:44.563 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:44.563 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:44.564 7fb326490d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1759110156448298 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:44:44.564 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:48.567 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:44:48.568 7fb326490d60 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:44:48.568 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:44:48.568 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:44:48.568 7fb326490d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:44:48.568 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:44:48.568 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:44:48.569 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:44:48.569 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:48.569 7fb326490d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:44:48.569 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:48.569 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:48.569 7fb326490d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1759110156448298 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:44:48.569 7fb326490d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:52.569 7fb3267a07c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:44:52.570 7fb3267a07c0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:44:52.570 7fb3267a07c0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:44:52.570 7fb3267a07c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:44:52.571 7fb3267a07c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:44:52.571 7fb3267a07c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:44:52.571 7fb3267a07c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:44:52.571 7fb3267a07c0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:44:52.571 7fb3267a07c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:52.571 7fb3267a07c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:44:52.571 7fb3267a07c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:52.571 7fb3267a07c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:52.572 7fb3267a07c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1759110156448298 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:44:52.572 7fb3267a07c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:56.575 7fb3267b9680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:44:56.576 7fb3267b9680 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:44:56.576 7fb3267b9680 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:44:56.576 7fb3267b9680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:44:56.576 7fb3267b9680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:44:56.576 7fb3267b9680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:44:56.576 7fb3267b9680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:44:56.577 7fb3267b9680 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:44:56.577 7fb3267b9680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:56.577 7fb3267b9680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:44:56.577 7fb3267b9680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:56.577 7fb3267b9680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:44:56.577 7fb3267b9680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1759110156448298 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:44:56.577 7fb3267b9680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:45:00.579 7fb3267b9680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:45:00.580 7fb3267b9680 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:45:00.580 7fb3267b9680 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:45:00.581 7fb3267b9680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:45:00.581 7fb3267b9680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:45:00.581 7fb3267b9680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:45:00.581 7fb3267b9680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:45:00.581 7fb3267b9680 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:45:00.581 7fb3267b9680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:45:00.581 7fb3267b9680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:45:00.582 7fb3267b9680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:45:00.582 7fb3267b9680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:45:00.582 7fb3267b9680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1759110156448298 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:45:00.582 7fb3267b9680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:45:04.583 7fb3267b9680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:45:04.584 7fb3267b9680 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 09/29/2025 01:45:04.584 7fb3267b9680 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:45:04.584 7fb3267b9680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:45:04.585 7fb3267b9680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:45:04.585 7fb3267b9680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:45:04.585 7fb3267b9680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/29/2025 01:45:04.585 7fb3267b9680 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/29/2025 01:45:04.585 7fb3267b9680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:45:04.585 7fb3267b9680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/29/2025 01:45:04.585 7fb3267b9680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:45:04.585 7fb3267b9680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:45:04.586 7fb3267b9680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1759110156448298 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:45:04.586 7fb3267b9680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:45:08.586 7fb326490900 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:45:08.587 7fb326490900 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected 09/29/2025 01:45:08.587 7fb326490900 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:45:08.587 7fb326490900 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:45:08.587 7fb326490900 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:45:08.587 7fb326490900 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:45:08.587 7fb326490900 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/29/2025 01:45:08.587 7fb326490900 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/29/2025 01:45:08.588 7fb326490900 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/29/2025 01:45:08.588 7fb326490900 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/29/2025 01:45:08.588 7fb326490900 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/29/2025 01:45:17.008 7fb3267b90e0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 09/29/2025 01:45:17.008 7fb3272125e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit] 09/29/2025 01:45:17.009 7fb3272125e0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle] 09/29/2025 01:45:17.009 7fb3272125e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:45:17.009 7fb3272125e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Down description = ClientInit to Idle on EvTcpClose >>, ] description = Collector:Collector[ClientInit to Idle on EvTcpClose] connection down >>, ] >> 09/29/2025 01:45:17.009 7fb3272125e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:45:17.009 7fb3272125e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:45:17.009 7fb3272125e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8112 start_time = 1759110156448298 collector_name = collector_ip = 10.0.0.49:8086 collector_list = [ 10.0.0.49:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 09/29/2025 01:45:17.009 7fb3272125e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/29/2025 01:45:21.010 7fb3272125e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/29/2025 01:45:21.012 7fb3272125e0 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected 09/29/2025 01:45:21.012 7fb3272125e0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/29/2025 01:45:21.012 7fb3272125e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:45:21.012 7fb3272125e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4200-1. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.49:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/29/2025 01:45:21.012 7fb3272125e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/29/2025 01:45:21.012 7fb3272125e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 09/29/2025 01:45:21.013 7fb3272125e0 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 09/29/2025 01:45:21.013 7fb3272125e0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 09/29/2025 01:45:21.013 7fb3272125e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/29/2025 01:45:21.013 7fb3272125e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 09/29/2025 01:45:21.020 7fb3267b9360 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001017 09/29/2025 01:45:21.021 7fb3267b9360 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 09/29/2025 01:45:21.021 7fb3272125e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 09/29/2025 01:45:21.021 7fb3272125e0 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 09/29/2025 01:45:21.022 7fb3272125e0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 09/29/2025 01:45:21.022 7fb3272125e0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/29/2025 01:45:21.022 7fb3272125e0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/29/2025 01:45:21.022 7fb3272125e0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 09/29/2025 01:45:21.022 7fb3267a0540 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/29/2025 01:45:21.023 7fb3267a0540 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/29/2025 01:45:28.213 7fb3267b9360 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001438 09/29/2025 01:45:28.214 7fb3267b9360 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 09/29/2025 01:45:28.214 7fb326490e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 09/29/2025 01:45:28.214 7fb326490e00 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 3 09/29/2025 01:45:28.214 7fb326490e00 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 09/29/2025 01:45:28.214 7fb326490e00 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/29/2025 01:45:28.215 7fb326490e00 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/29/2025 01:45:28.215 7fb3267a0180 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 09/29/2025 01:45:28.215 7fb3267a0180 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 09/29/2025 01:45:36.889 7fb3293f3b80 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/29/2025 01:45:37.393 7fb3293f3b80 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/29/2025 01:45:38.238 7fb3293f3b80 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/29/2025 01:45:43.544 7fb3293f3b80 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4200-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=17751620, partition_space_available_1k=63338864, percentage_partition_space_used=22, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1344888, cpu_share=0.04, mem_res=1331696, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=111972, cpu_share=0.04, mem_res=108324, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52888, cpu_share=0.01, mem_res=42668, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9493084, free=5049436, buffers=688884, cached=17911960, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.09, five_min_avg=2.64, fifteen_min_avg=1.57, cpu_share=3.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/29/2025 01:46:36.959 7fb3293f3b80 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/29/2025 01:46:37.503 7fb3293f3b80 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/29/2025 01:46:38.303 7fb3293f3b80 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/29/2025 01:46:43.573 7fb3293f3b80 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4200-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19578220, partition_space_available_1k=61512264, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1342940, cpu_share=0.04, mem_res=1331840, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=116528, cpu_share=0.01, mem_res=113192, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44928, cpu_share=0.02, mem_res=42668, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=9471476, free=18406016, buffers=704316, cached=4567852, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.12, five_min_avg=2.63, fifteen_min_avg=1.64, cpu_share=4.12, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/29/2025 01:47:17.793 7fb326490d60 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/29/2025 01:47:19.868 7fb326490d60 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 09/29/2025 01:47:36.902 7fb3293f3b80 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/29/2025 01:47:37.373 7fb3293f3b80 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/29/2025 01:47:38.169 7fb3293f3b80 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/29/2025 01:47:43.565 7fb3293f3b80 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4200-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=19765756, partition_space_available_1k=61324728, percentage_partition_space_used=24, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1343040, cpu_share=0.04, mem_res=1331948, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=116880, cpu_share=0.0, mem_res=113540, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=48196, cpu_share=0.02, mem_res=42668, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=10075752, free=17429580, buffers=713456, cached=4924872, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.47, five_min_avg=2.54, fifteen_min_avg=1.67, cpu_share=3.12, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/29/2025 01:48:36.908 7fb3293f3b80 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/29/2025 01:48:37.432 7fb3293f3b80 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/29/2025 01:48:38.265 7fb3293f3b80 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/29/2025 01:48:43.582 7fb3293f3b80 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4200-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21052400, partition_space_available_1k=60038084, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1344248, cpu_share=0.06, mem_res=1332496, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=117676, cpu_share=0.0, mem_res=114336, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=48208, cpu_share=0.02, mem_res=42668, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=10643560, free=15542484, buffers=748284, cached=6209228, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.9, five_min_avg=2.69, fifteen_min_avg=1.78, cpu_share=3.88, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/29/2025 01:49:36.953 7fb3293f3b80 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/29/2025 01:49:37.473 7fb3293f3b80 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/29/2025 01:49:38.245 7fb3293f3b80 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/29/2025 01:49:43.595 7fb3293f3b80 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4200-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21084992, partition_space_available_1k=60005492, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1347984, cpu_share=0.21, mem_res=1335896, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=130900, cpu_share=0.01, mem_res=127492, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=48212, cpu_share=0.02, mem_res=42668, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=10699840, free=15422348, buffers=748996, cached=6271092, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.66, five_min_avg=3.15, fifteen_min_avg=2.0, cpu_share=4.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/29/2025 01:50:36.946 7fb3293f3b80 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/29/2025 01:50:37.470 7fb3293f3b80 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/29/2025 01:50:38.264 7fb3293f3b80 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/29/2025 01:50:43.607 7fb3293f3b80 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4200-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21199408, partition_space_available_1k=59891076, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1342708, cpu_share=0.08, mem_res=1330552, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=126500, cpu_share=0.0, mem_res=123072, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=48212, cpu_share=0.02, mem_res=42668, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=11339988, free=14658728, buffers=749056, cached=6394748, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.09, five_min_avg=3.75, fifteen_min_avg=2.27, cpu_share=5.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/29/2025 01:51:36.954 7fb3293f3b80 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/29/2025 01:51:37.484 7fb3293f3b80 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/29/2025 01:51:38.247 7fb3293f3b80 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/29/2025 01:51:43.621 7fb3293f3b80 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4200-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21231012, partition_space_available_1k=59859472, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1341228, cpu_share=0.14, mem_res=1328600, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=131492, cpu_share=0.01, mem_res=128004, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=48220, cpu_share=0.02, mem_res=42668, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=11105348, free=14843704, buffers=749452, cached=6446704, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.45, five_min_avg=4.05, fifteen_min_avg=2.47, cpu_share=4.62, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/29/2025 01:52:36.950 7fb3293f3b80 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/29/2025 01:52:37.436 7fb3293f3b80 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/29/2025 01:52:38.141 7fb3293f3b80 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/29/2025 01:52:43.614 7fb3293f3b80 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4200-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21259656, partition_space_available_1k=59830828, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1341116, cpu_share=0.09, mem_res=1328316, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=142836, cpu_share=0.01, mem_res=139264, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=48220, cpu_share=0.02, mem_res=42668, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=10943540, free=14978480, buffers=749672, cached=6470996, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.48, five_min_avg=4.04, fifteen_min_avg=2.57, cpu_share=4.12, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/29/2025 01:53:36.961 7fb3293f3b80 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/29/2025 01:53:37.457 7fb3293f3b80 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/29/2025 01:53:38.234 7fb3293f3b80 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/29/2025 01:53:43.627 7fb3293f3b80 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4200-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21350156, partition_space_available_1k=59740328, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1341524, cpu_share=0.07, mem_res=1328140, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=144628, cpu_share=0.01, mem_res=141036, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=48232, cpu_share=0.02, mem_res=42668, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=10889952, free=14984876, buffers=749764, cached=6517896, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.58, five_min_avg=3.85, fifteen_min_avg=2.59, cpu_share=4.12, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/29/2025 01:54:36.952 7fb3293f3b80 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/29/2025 01:54:37.420 7fb3293f3b80 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/29/2025 01:54:38.222 7fb3293f3b80 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/29/2025 01:54:43.640 7fb3293f3b80 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4200-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21343324, partition_space_available_1k=59747160, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1342012, cpu_share=0.06, mem_res=1328388, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=146652, cpu_share=0.0, mem_res=143048, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=48232, cpu_share=0.01, mem_res=42668, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=10913952, free=14935900, buffers=749812, cached=6542220, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.38, five_min_avg=3.85, fifteen_min_avg=2.68, cpu_share=3.38, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/29/2025 01:55:36.964 7fb3293f3b80 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/29/2025 01:55:37.482 7fb3293f3b80 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/29/2025 01:55:38.228 7fb3293f3b80 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/29/2025 01:55:43.650 7fb3293f3b80 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4200-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21369400, partition_space_available_1k=59721084, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1352452, cpu_share=0.08, mem_res=1338696, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=150060, cpu_share=0.0, mem_res=146440, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=48236, cpu_share=0.02, mem_res=42668, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=10955464, free=14866340, buffers=749856, cached=6570156, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.63, five_min_avg=3.55, fifteen_min_avg=2.65, cpu_share=4.12, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/29/2025 01:56:36.963 7fb3293f3b80 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/29/2025 01:56:37.490 7fb3293f3b80 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/29/2025 01:56:38.363 7fb3293f3b80 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/29/2025 01:56:43.661 7fb3293f3b80 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4200-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21480696, partition_space_available_1k=59609788, percentage_partition_space_used=26, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1352716, cpu_share=0.07, mem_res=1338716, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=152068, cpu_share=0.0, mem_res=148424, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=48240, cpu_share=0.02, mem_res=42668, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=11056228, free=14648668, buffers=749972, cached=6687088, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.98, five_min_avg=3.99, fifteen_min_avg=2.85, cpu_share=5.12, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/29/2025 01:57:36.983 7fb3293f3b80 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/29/2025 01:57:37.480 7fb3293f3b80 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/29/2025 01:57:38.207 7fb3293f3b80 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/29/2025 01:57:43.653 7fb3293f3b80 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4200-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21489736, partition_space_available_1k=59600748, percentage_partition_space_used=27, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1352444, cpu_share=0.04, mem_res=1338584, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=152544, cpu_share=0.0, mem_res=148900, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=48248, cpu_share=0.02, mem_res=42668, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=10930560, free=14768224, buffers=749976, cached=6693012, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.6, five_min_avg=3.7, fifteen_min_avg=2.82, cpu_share=2.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/29/2025 01:58:37.018 7fb3293f3b80 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 09/29/2025 01:58:37.513 7fb3293f3b80 [contrail-config-database-nodemgr] [WARNING]: CQLSH failed, but Cassandra native transport port is open 09/29/2025 01:58:38.243 7fb3293f3b80 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 09/29/2025 01:58:43.656 7fb3293f3b80 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-4200-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=21547676, partition_space_available_1k=59542808, percentage_partition_space_used=27, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1353732, cpu_share=0.08, mem_res=1339136, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=155316, cpu_share=0.0, mem_res=151648, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=48248, cpu_share=0.02, mem_res=42668, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665120, used=10734332, free=14892308, buffers=752336, cached=6761700, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.55, five_min_avg=3.37, fifteen_min_avg=2.76, cpu_share=3.12, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 09/29/2025 01:58:59.268 7fb326490b80 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map