03/23/2026 05:34:29.117 7f5b8e9c6df0 [contrail-config-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 03/23/2026 05:34:29.117 7f5b8e9c6df0 [contrail-config-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 03/23/2026 05:34:29.117 7f5b8e9c6df0 [contrail-config-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 03/23/2026 05:34:29.128 7f5b89264548 [contrail-config-database-nodemgr] [ERROR]: Cannot write http_port 8112 to /tmp/contrail-config-database-nodemgr.0.http_port 03/23/2026 05:34:29.129 7f5b89264548 [contrail-config-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8112 03/23/2026 05:34:29.129 7f5b89264d48 [contrail-config-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 03/23/2026 05:34:29.130 7f5b89264d48 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 03/23/2026 05:34:29.130 7f5b89264d48 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/23/2026 05:34:29.130 7f5b89264d48 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 03/23/2026 05:34:29.131 7f5b89264d48 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:29.131 7f5b89264d48 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/23/2026 05:34:29.131 7f5b89264d48 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:29.131 7f5b89264d48 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:29.131 7f5b89264d48 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 03/23/2026 05:34:29.131 7f5b89264d48 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:29.188 7f5b8e9c6df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 49272100671639752793538942992921442278240362946750721112866467363431621646583, 'start': '1774244066000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'zookeeper', 'group': 'zookeeper', 'pid': 61653878596493082782633898596964876048579072582723600998390400976263827948208, 'start': '1774244066000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-config-database-nodemgr', 'group': 'contrail-config-database-nodemgr', 'pid': 114954805924595068410342993938399120294239317146604628856750655849275989431086, 'start': '1774244067000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 03/23/2026 05:34:29.319 7f5b89103048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:29.319 7f5b89103048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. build_info = {"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-10-06 01:00:31.164147", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11344-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 4 num_core_per_socket = 4 num_thread_per_core = 1 >> running_package_version = v0+nightly.r24.1.centos-nightly_r24.1_centos.el7 installed_package_version = v0+nightly.r24.1.centos-nightly_r24.1_centos.el7 >> 03/23/2026 05:34:29.319 7f5b89103048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:29.320 7f5b8e9c6df0 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1774244069320608, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 03/23/2026 05:34:29.321 7f5b8e9c6df0 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1774244069320926, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774244066000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774244066000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774244067000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-10-06 01:00:31.164147", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11344-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 03/23/2026 05:34:29.322 7f5b89103048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:29.323 7f5b89103048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/23/2026 05:34:29.323 7f5b89103048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:29.323 7f5b89103048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:29.323 7f5b89103048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774244066000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = zookeeper process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774244066000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774244067000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-10-06 01:00:31.164147", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11344-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} >> 03/23/2026 05:34:29.323 7f5b89103048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:29.527 7f5b8e9c6df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 05:34:29.699 7f5b8e9c6df0 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 03/23/2026 05:34:29.701 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:29.701 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. database_usage = [ << disk_space_used_1k = 4558368 disk_space_available_1k = 76532116 config_db_size_1k = 4 >>, ] >> 03/23/2026 05:34:29.701 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:30.149 7f5b8e9c6df0 [contrail-config-database-nodemgr] [ERROR]: Failed to connect to database by CQL: Result: 1 03/23/2026 05:34:30.149 7f5b8e9c6df0 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1774244070149531, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 03/23/2026 05:34:30.151 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:30.151 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/23/2026 05:34:30.151 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:31.807 7f5b8e9c6df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 05:34:33.130 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:34:33.131 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:34:33.132 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:34:33.132 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:34:33.132 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:34:33.132 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:34:33.132 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:34:33.133 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:34:33.133 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:33.133 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:34:33.133 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:33.133 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:33.134 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:34:33.134 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:33.932 7f5b8e9c6df0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr config_db_uuid'. Error is: Result: 1 03/23/2026 05:34:35.099 7f5b8e9c6df0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr to_bgp_keyspace'. Error is: Result: 1 03/23/2026 05:34:35.860 7f5b8e9c6df0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr svc_monitor_keyspace'. Error is: Result: 1 03/23/2026 05:34:36.774 7f5b8e9c6df0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr DISCOVERY_SERVER'. Error is: Result: 1 03/23/2026 05:34:37.132 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:34:37.134 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:34:37.134 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:34:37.134 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:34:37.134 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:34:37.134 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:34:37.134 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:34:37.135 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:34:37.135 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:37.135 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:34:37.135 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:37.135 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:37.135 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:34:37.135 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:37.683 7f5b8e9c6df0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr useragent'. Error is: Result: 1 03/23/2026 05:34:38.561 7f5b8e9c6df0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr dm_keyspace'. Error is: Result: 1 03/23/2026 05:34:41.134 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:34:41.136 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:34:41.136 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:34:41.136 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:34:41.136 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:34:41.136 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:34:41.137 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:34:41.137 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:34:41.137 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:41.137 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:34:41.137 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:41.137 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:41.138 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:34:41.138 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:43.585 7f5b8e9c6df0 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1774244083585691, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774244066000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774244066000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774244067000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-10-06 01:00:31.164147", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11344-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 03/23/2026 05:34:43.586 7f5b8e9c6df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=4910444, partition_space_available_1k=76180040, percentage_partition_space_used=6, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1236656, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57984, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51084, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=2075924, free=9816116, buffers=102844, cached=4528720, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.86, five_min_avg=0.54, fifteen_min_avg=0.19, cpu_share=0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 05:34:43.586 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:43.586 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774244066000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = zookeeper process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774244066000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774244067000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-10-06 01:00:31.164147", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11344-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} >> 03/23/2026 05:34:43.586 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:43.587 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:43.587 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 4910444 partition_space_available_1k = 76180040 percentage_partition_space_used = 6 >> } all_core_file_list = [ ] process_mem_cpu_usage = { cassandra : << mem_virt = 1236656 cpu_share = 0 mem_res = 0 >>zookeeper : << mem_virt = 57984 cpu_share = 0 mem_res = 0 >>contrail-config-database-nodemgr : << mem_virt = 51084 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175636 used = 2075924 free = 9816116 buffers = 102844 cached = 4528720 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 1.86 five_min_avg = 0.54 fifteen_min_avg = 0.19 cpu_share = 0 node_type = config-database-node >> >> 03/23/2026 05:34:43.587 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:45.137 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:34:45.138 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:34:45.138 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:34:45.138 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:34:45.139 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:34:45.139 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:34:45.139 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:34:45.139 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:34:45.139 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:45.139 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:34:45.139 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:45.140 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:45.140 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:34:45.140 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:49.139 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:34:49.140 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:34:49.141 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:34:49.141 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:34:49.141 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:34:49.141 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:34:49.141 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:34:49.141 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:34:49.141 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:49.141 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:34:49.142 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:49.142 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:49.142 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:34:49.142 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:53.142 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:34:53.143 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:34:53.143 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:34:53.143 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:34:53.144 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:34:53.144 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:34:53.144 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:34:53.144 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:34:53.144 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:53.144 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:34:53.144 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:53.144 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:53.145 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:34:53.145 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:57.147 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:34:57.149 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:34:57.149 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:34:57.149 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:34:57.149 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:34:57.149 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:34:57.149 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:34:57.149 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:34:57.150 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:57.150 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:34:57.150 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:57.150 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:34:57.150 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:34:57.150 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:01.150 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:35:01.151 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:35:01.153 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:35:01.153 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:35:01.153 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:35:01.153 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:35:01.154 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:35:01.155 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:35:01.155 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:01.155 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:35:01.155 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:01.155 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:01.155 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:35:01.155 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:05.155 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:35:05.156 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:35:05.156 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:35:05.156 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:35:05.157 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:35:05.157 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:35:05.157 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:35:05.157 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:35:05.157 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:05.157 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:35:05.157 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:05.157 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:05.158 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:35:05.158 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:09.160 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:35:09.161 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:35:09.162 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:35:09.162 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:35:09.162 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:35:09.162 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:35:09.162 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:35:09.162 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:35:09.162 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:09.163 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:35:09.163 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:09.163 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:09.163 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:35:09.163 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:13.162 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:35:13.163 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:35:13.163 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:35:13.164 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:35:13.164 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:35:13.164 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:35:13.164 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:35:13.164 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:35:13.164 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:13.164 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:35:13.164 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:13.165 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:13.165 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:35:13.165 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:17.166 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:35:17.168 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:35:17.169 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:35:17.169 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:35:17.169 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:35:17.169 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:35:17.169 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:35:17.169 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:35:17.170 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:17.170 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:35:17.170 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:17.170 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:17.170 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:35:17.170 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:21.170 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:35:21.172 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:35:21.172 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:35:21.172 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:35:21.172 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:35:21.172 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:35:21.172 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:35:21.173 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:35:21.173 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:21.173 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:35:21.173 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:21.173 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:21.173 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:35:21.174 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:25.174 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:35:25.176 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:35:25.176 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:35:25.176 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:35:25.176 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:35:25.177 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:35:25.177 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:35:25.177 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:35:25.178 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:25.178 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:35:25.178 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:25.178 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:25.178 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:35:25.178 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:29.179 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:35:29.180 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:35:29.180 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:35:29.180 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:35:29.180 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:35:29.180 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:35:29.181 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:35:29.181 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:35:29.181 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:29.181 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:35:29.181 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:29.181 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:29.181 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:35:29.182 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:29.510 7f5b8e9c6df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 05:35:29.834 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:29.834 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. database_usage = [ << disk_space_used_1k = 6009416 disk_space_available_1k = 75081068 config_db_size_1k = 1576 >>, ] >> 03/23/2026 05:35:29.834 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:30.775 7f5b8e9c6df0 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1774244130775804, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.50:8086'], status='Down', description='Connect to Idle on EvTcpConnectFail')]), flag_infos=None, description='Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 03/23/2026 05:35:30.777 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:30.777 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:35:30.777 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:32.601 7f5b8e9c6df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 05:35:33.180 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:35:33.182 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:35:33.182 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:35:33.182 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:35:33.182 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:35:33.182 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:35:33.183 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:35:33.183 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:35:33.183 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:33.183 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:35:33.183 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:33.183 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:33.183 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:35:33.183 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:37.183 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:35:37.184 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:35:37.184 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:35:37.185 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:35:37.185 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:35:37.185 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:35:37.185 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:35:37.185 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:35:37.185 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:37.185 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:35:37.185 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:37.186 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:37.186 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:35:37.186 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:37.756 7f5b8e9c6df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6106308, partition_space_available_1k=74984176, percentage_partition_space_used=8, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1318644, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=62044, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51556, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=2629488, free=8311544, buffers=206716, cached=5376520, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.42, five_min_avg=0.93, fifteen_min_avg=0.34, cpu_share=9.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 05:35:37.758 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:37.761 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 6106308 partition_space_available_1k = 74984176 percentage_partition_space_used = 8 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1318644 cpu_share = 0 mem_res = 0 >>zookeeper : << mem_virt = 62044 cpu_share = 0 mem_res = 0 >>contrail-config-database-nodemgr : << mem_virt = 51556 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175636 used = 2629488 free = 8311544 buffers = 206716 cached = 5376520 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 2.42 five_min_avg = 0.93 fifteen_min_avg = 0.34 cpu_share = 9.0 node_type = config-database-node >> >> 03/23/2026 05:35:37.761 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:41.186 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:35:41.187 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:35:41.187 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:35:41.187 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:35:41.187 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:35:41.187 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:35:41.188 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:35:41.188 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:35:41.188 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:41.188 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:35:41.188 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:41.188 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:41.188 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:35:41.188 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:45.188 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:35:45.189 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:35:45.189 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:35:45.190 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:35:45.190 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:35:45.190 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:35:45.190 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:35:45.190 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:35:45.190 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:45.191 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:35:45.191 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:45.191 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:45.191 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:35:45.191 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:49.190 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:35:49.192 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:35:49.192 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:35:49.192 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:35:49.192 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:35:49.192 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:35:49.192 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:35:49.193 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:35:49.193 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:49.193 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:35:49.193 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:49.193 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:49.193 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:35:49.193 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:53.194 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:35:53.195 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:35:53.196 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:35:53.196 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:35:53.196 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:35:53.196 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:35:53.196 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:35:53.196 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:35:53.196 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:53.197 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:35:53.197 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:53.197 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:53.197 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:35:53.197 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:57.199 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:35:57.200 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:35:57.201 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:35:57.201 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:35:57.201 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:35:57.201 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:35:57.201 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:35:57.202 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:35:57.202 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:57.202 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:35:57.202 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:57.202 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:35:57.202 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:35:57.202 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:01.201 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:36:01.202 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:36:01.202 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:36:01.202 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:36:01.202 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:36:01.203 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:36:01.203 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:36:01.203 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:36:01.203 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:01.203 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:36:01.203 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:01.203 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:01.203 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:36:01.204 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:05.203 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:36:05.204 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:36:05.204 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:36:05.204 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:36:05.205 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:36:05.205 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:36:05.206 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:36:05.206 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:36:05.206 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:05.206 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:36:05.206 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:05.206 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:05.206 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:36:05.206 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:09.206 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:36:09.207 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:36:09.208 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:36:09.208 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:36:09.208 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:36:09.208 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:36:09.208 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:36:09.208 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:36:09.208 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:09.209 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:36:09.209 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:09.209 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:09.209 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:36:09.209 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:13.211 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:36:13.212 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:36:13.213 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:36:13.213 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:36:13.213 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:36:13.213 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:36:13.213 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:36:13.213 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:36:13.213 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:13.214 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:36:13.214 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:13.214 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:13.214 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:36:13.214 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:17.214 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:36:17.215 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:36:17.216 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:36:17.216 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:36:17.216 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:36:17.216 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:36:17.216 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:36:17.217 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:36:17.217 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:17.217 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:36:17.217 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:17.217 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:17.217 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:36:17.217 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:21.218 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:36:21.219 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:36:21.220 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:36:21.220 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:36:21.220 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:36:21.220 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:36:21.220 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:36:21.220 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:36:21.220 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:21.221 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:36:21.221 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:21.221 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:21.221 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:36:21.221 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:25.222 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:36:25.224 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:36:25.224 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:36:25.224 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:36:25.224 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:36:25.224 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:36:25.225 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:36:25.225 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:36:25.225 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:25.225 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:36:25.225 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:25.226 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:25.226 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:36:25.226 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:29.226 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:36:29.228 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:36:29.228 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:36:29.228 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:36:29.228 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:36:29.228 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:36:29.228 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:36:29.229 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:36:29.229 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:29.229 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:36:29.229 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:29.229 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:29.229 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:36:29.229 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:29.572 7f5b8e9c6df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 05:36:30.472 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:30.472 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. database_usage = [ << disk_space_used_1k = 6253096 disk_space_available_1k = 74837388 config_db_size_1k = 1728 >>, ] >> 03/23/2026 05:36:30.472 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:32.498 7f5b8e9c6df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 05:36:33.228 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:36:33.230 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:36:33.230 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:36:33.230 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:36:33.231 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:36:33.231 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:36:33.231 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:36:33.231 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:36:33.231 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:33.231 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:36:33.232 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:33.232 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:33.232 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:36:33.232 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:37.231 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:36:37.232 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:36:37.233 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:36:37.233 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:36:37.233 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:36:37.233 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:36:37.233 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:36:37.234 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:36:37.234 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:37.234 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:36:37.234 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:37.234 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:37.235 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:36:37.235 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:37.755 7f5b8e9c6df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6259588, partition_space_available_1k=74830896, percentage_partition_space_used=8, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1687712, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=74044, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52060, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=3487604, free=6934920, buffers=219704, cached=5882616, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.06, five_min_avg=1.08, fifteen_min_avg=0.43, cpu_share=6.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 05:36:37.755 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:37.755 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 6259588 partition_space_available_1k = 74830896 percentage_partition_space_used = 8 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1687712 cpu_share = 0 mem_res = 0 >>zookeeper : << mem_virt = 74044 cpu_share = 0 mem_res = 0 >>contrail-config-database-nodemgr : << mem_virt = 52060 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175636 used = 3487604 free = 6934920 buffers = 219704 cached = 5882616 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 2.06 five_min_avg = 1.08 fifteen_min_avg = 0.43 cpu_share = 6.75 node_type = config-database-node >> >> 03/23/2026 05:36:37.756 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:41.235 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:36:41.236 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:36:41.237 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:36:41.237 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:36:41.237 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:36:41.237 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:36:41.237 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:36:41.238 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:36:41.238 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:41.238 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:36:41.238 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:41.238 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:41.238 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:36:41.238 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:45.237 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:36:45.238 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:36:45.238 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:36:45.239 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:36:45.239 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:36:45.239 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:36:45.239 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:36:45.239 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:36:45.239 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:45.239 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:36:45.239 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:45.239 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:45.240 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:36:45.240 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:49.242 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:36:49.243 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:36:49.243 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:36:49.244 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:36:49.244 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:36:49.244 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:36:49.244 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:36:49.244 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:36:49.244 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:49.244 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:36:49.244 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:49.245 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:49.245 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:36:49.245 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:53.244 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:36:53.245 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:36:53.246 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:36:53.246 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:36:53.246 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:36:53.246 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:36:53.246 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:36:53.247 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:36:53.247 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:53.247 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:36:53.247 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:53.247 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:53.247 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:36:53.247 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:57.249 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:36:57.251 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:36:57.251 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:36:57.251 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:36:57.251 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:36:57.251 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:36:57.251 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:36:57.252 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:36:57.252 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:57.252 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:36:57.252 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:57.252 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:36:57.252 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:36:57.252 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:01.252 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:37:01.253 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:37:01.253 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:37:01.253 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:01.254 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:37:01.254 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:01.254 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:37:01.254 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:37:01.254 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:01.254 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:37:01.254 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:01.255 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:01.255 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:37:01.255 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:05.254 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:37:05.255 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:37:05.255 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:37:05.256 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:05.256 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:37:05.256 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:05.256 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:37:05.256 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:37:05.256 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:05.257 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:37:05.257 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:05.257 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:05.257 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:37:05.257 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:09.258 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:37:09.260 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:37:09.260 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:37:09.260 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:09.260 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:37:09.260 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:09.260 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:37:09.260 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:37:09.261 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:09.261 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:37:09.261 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:09.261 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:09.261 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:37:09.261 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:13.262 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:37:13.263 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:37:13.263 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:37:13.263 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:13.264 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:37:13.264 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:13.264 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:37:13.264 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:37:13.264 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:13.264 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:37:13.264 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:13.264 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:13.264 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:37:13.264 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:17.263 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:37:17.264 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:37:17.265 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:37:17.265 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:17.265 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:37:17.265 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:17.265 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:37:17.265 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:37:17.265 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:17.266 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:37:17.266 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:17.266 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:17.266 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:37:17.266 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:21.266 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:37:21.267 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:37:21.269 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:37:21.269 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:21.269 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:37:21.270 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:21.270 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:37:21.270 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:37:21.270 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:21.270 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:37:21.270 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:21.270 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:21.271 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:37:21.271 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:25.270 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:37:25.272 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:37:25.272 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:37:25.272 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:25.272 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:37:25.272 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:25.273 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:37:25.273 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:37:25.273 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:25.273 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:37:25.273 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:25.274 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:25.274 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:37:25.274 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:29.274 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:37:29.276 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:37:29.276 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:37:29.276 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:29.276 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:37:29.276 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:29.276 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:37:29.277 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:37:29.277 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:29.277 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:37:29.277 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:29.277 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:29.277 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:37:29.277 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:29.558 7f5b8e9c6df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 05:37:29.804 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:29.804 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. database_usage = [ << disk_space_used_1k = 6666516 disk_space_available_1k = 74423968 config_db_size_1k = 1692 >>, ] >> 03/23/2026 05:37:29.804 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:31.287 7f5b8e9c6df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 05:37:33.276 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:37:33.278 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:37:33.278 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:37:33.278 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:33.278 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:37:33.278 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:33.278 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:37:33.279 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:37:33.279 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:33.279 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:37:33.279 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:33.279 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:33.279 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:37:33.279 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:36.749 7f5b8e9c6df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6666864, partition_space_available_1k=74423620, percentage_partition_space_used=8, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1712156, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=78548, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52340, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=4782808, free=5516080, buffers=244708, cached=5981500, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.76, five_min_avg=1.82, fifteen_min_avg=0.73, cpu_share=8.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 05:37:36.749 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:36.749 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 6666864 partition_space_available_1k = 74423620 percentage_partition_space_used = 8 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1712156 cpu_share = 0 mem_res = 0 >>zookeeper : << mem_virt = 78548 cpu_share = 0 mem_res = 0 >>contrail-config-database-nodemgr : << mem_virt = 52340 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175636 used = 4782808 free = 5516080 buffers = 244708 cached = 5981500 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 3.76 five_min_avg = 1.82 fifteen_min_avg = 0.73 cpu_share = 8.25 node_type = config-database-node >> >> 03/23/2026 05:37:36.750 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:37.278 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:37:37.280 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:37:37.280 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:37:37.280 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:37.280 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:37:37.280 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:37.280 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:37:37.281 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:37:37.281 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:37.281 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:37:37.281 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:37.281 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:37.281 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244069130716 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.33:8086, 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/23/2026 05:37:37.281 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:37:41.283 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:37:41.286 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected 03/23/2026 05:37:41.287 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:37:41.287 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:41.288 7f5b88e08248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:37:41.288 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:37:41.289 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 03/23/2026 05:37:41.290 7f5b88e08248 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 03/23/2026 05:37:41.291 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 03/23/2026 05:37:41.291 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:37:41.292 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:37:49.079 7f5b89103048 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001016 03/23/2026 05:37:49.080 7f5b89103048 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 03/23/2026 05:37:49.081 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 03/23/2026 05:37:49.081 7f5b88e08248 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 03/23/2026 05:37:49.082 7f5b88e08248 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 03/23/2026 05:37:49.082 7f5b88e08248 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 03/23/2026 05:37:49.082 7f5b88e08248 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/23/2026 05:37:49.082 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 03/23/2026 05:37:49.083 7f5b89264e48 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 03/23/2026 05:37:49.087 7f5b89264e48 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 03/23/2026 05:38:00.032 7f5b89103048 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001439 03/23/2026 05:38:00.032 7f5b89103048 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 03/23/2026 05:38:00.032 7f5b89264e48 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 03/23/2026 05:38:00.033 7f5b89264e48 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 3 03/23/2026 05:38:00.033 7f5b89264e48 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 03/23/2026 05:38:00.033 7f5b89264e48 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 03/23/2026 05:38:00.033 7f5b89264e48 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/23/2026 05:38:00.033 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 03/23/2026 05:38:00.034 7f5b88e08248 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 03/23/2026 05:38:29.570 7f5b8e9c6df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 05:38:31.590 7f5b8e9c6df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 05:38:36.776 7f5b8e9c6df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7106460, partition_space_available_1k=73984024, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1706208, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=156724, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52452, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=5940080, free=3843288, buffers=267512, cached=6475088, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.84, five_min_avg=2.22, fifteen_min_avg=0.94, cpu_share=12.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 05:39:29.580 7f5b8e9c6df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 05:39:32.029 7f5b8e9c6df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 05:39:37.783 7f5b8e9c6df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7171544, partition_space_available_1k=73918940, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1704948, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=131956, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52464, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=5917672, free=3801264, buffers=268812, cached=6538256, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.26, five_min_avg=2.35, fifteen_min_avg=1.06, cpu_share=11.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 05:40:29.585 7f5b8e9c6df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 05:40:31.606 7f5b8e9c6df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 05:40:36.800 7f5b8e9c6df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7198000, partition_space_available_1k=73892484, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1713792, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=152092, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52496, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=5940528, free=3749276, buffers=270236, cached=6565908, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.85, five_min_avg=2.7, fifteen_min_avg=1.27, cpu_share=12.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 05:41:08.644 7f5b89264e48 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 03/23/2026 05:41:21.528 7f5b89264e48 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 03/23/2026 05:41:29.580 7f5b8e9c6df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 05:41:31.465 7f5b8e9c6df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 05:41:36.795 7f5b8e9c6df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7656948, partition_space_available_1k=73433536, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1710488, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=172120, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52524, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=5996856, free=3218460, buffers=276580, cached=7034048, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.47, five_min_avg=2.84, fifteen_min_avg=1.41, cpu_share=12.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 05:42:29.573 7f5b8e9c6df0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 05:42:32.103 7f5b8e9c6df0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 05:42:37.808 7f5b8e9c6df0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7662380, partition_space_available_1k=73428104, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1741404, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=194220, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52544, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6086468, free=3121728, buffers=277916, cached=7039840, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.86, five_min_avg=3.32, fifteen_min_avg=1.66, cpu_share=12.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 05:43:24.750 7f5b89103048 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 03/23/2026 05:43:24.751 7f5b89264e48 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Established] 03/23/2026 05:43:24.754 7f5b89264e48 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected 03/23/2026 05:43:24.754 7f5b89264e48 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[Established] -> State[Connect] 03/23/2026 05:43:24.754 7f5b89264e48 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:43:24.754 7f5b89264e48 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Established to Connect on EvTcpClose >>, ] description = Collector:Collector[Established to Connect on EvTcpClose] connection down >>, ] >> 03/23/2026 05:43:24.755 7f5b89264e48 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:43:24.755 7f5b89264e48 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 03/23/2026 05:43:24.755 7f5b89264e48 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 03/23/2026 05:43:24.755 7f5b89264e48 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 03/23/2026 05:43:24.755 7f5b89264e48 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:43:24.755 7f5b89264e48 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:43:33.059 7f77c117ef20 [contrail-config-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 03/23/2026 05:43:33.059 7f77c117ef20 [contrail-config-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 03/23/2026 05:43:33.059 7f77c117ef20 [contrail-config-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 03/23/2026 05:43:33.072 7f77bb39c748 [contrail-config-database-nodemgr] [ERROR]: Cannot write http_port 8112 to /tmp/contrail-config-database-nodemgr.0.http_port 03/23/2026 05:43:33.082 7f77bb39c748 [contrail-config-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8112 03/23/2026 05:43:33.083 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 03/23/2026 05:43:33.086 7f77ba1b7048 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 03/23/2026 05:43:33.104 7f77ba1b7048 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/23/2026 05:43:33.104 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 03/23/2026 05:43:33.104 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:43:33.110 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/23/2026 05:43:33.111 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:43:33.111 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:43:33.111 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774244613086459 collector_name = collector_ip = collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 03/23/2026 05:43:33.111 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:43:33.736 7f77c117ef20 [contrail-config-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 49272100671639752793538942992921442278240362946750721112866467363431621646583, 'start': '1774244607000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'zookeeper', 'group': 'zookeeper', 'pid': 61653878596493082782633898596964876048579072582723600998390400976263827948208, 'start': '1774244607000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-config-database-nodemgr', 'group': 'contrail-config-database-nodemgr', 'pid': 114954805924595068410342993938399120294239317146604628856750655849275989431086, 'start': '1774244607000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 03/23/2026 05:43:34.235 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:43:34.236 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. build_info = {"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-10-06 01:00:31.164147", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11344-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 4 num_core_per_socket = 4 num_thread_per_core = 1 >> running_package_version = v0+nightly.r24.1.centos-nightly_r24.1_centos.el7 installed_package_version = v0+nightly.r24.1.centos-nightly_r24.1_centos.el7 >> 03/23/2026 05:43:34.236 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:43:34.236 7f77c117ef20 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1774244614236425, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 03/23/2026 05:43:34.236 7f77c117ef20 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1774244614236720, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774244607000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774244607000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774244607000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-10-06 01:00:31.164147", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11344-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 03/23/2026 05:43:34.239 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:43:34.239 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/23/2026 05:43:34.239 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:43:34.239 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:43:34.240 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774244607000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = zookeeper process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774244607000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774244607000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-10-06 01:00:31.164147", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11344-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} >> 03/23/2026 05:43:34.240 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:43:35.704 7f77c117ef20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 05:43:37.013 7f77c117ef20 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 03/23/2026 05:43:37.015 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:43:37.015 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. database_usage = [ << disk_space_used_1k = 7660032 disk_space_available_1k = 73430452 config_db_size_1k = 3064 >>, ] >> 03/23/2026 05:43:37.015 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:43:37.086 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:43:37.087 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected 03/23/2026 05:43:37.088 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:43:37.088 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:43:37.088 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:43:37.088 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:43:37.088 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 03/23/2026 05:43:37.088 7f77ba1b7048 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 03/23/2026 05:43:37.088 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 03/23/2026 05:43:37.088 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:43:37.088 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:43:37.094 7f77ba172448 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001017 03/23/2026 05:43:37.094 7f77ba172448 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 03/23/2026 05:43:37.094 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 03/23/2026 05:43:37.095 7f77ba1b7048 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 03/23/2026 05:43:37.095 7f77ba1b7048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 03/23/2026 05:43:37.095 7f77ba1b7048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 03/23/2026 05:43:37.095 7f77ba1b7048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/23/2026 05:43:37.095 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 03/23/2026 05:43:37.095 7f77ba1b7448 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 03/23/2026 05:43:37.096 7f77ba1b7448 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 03/23/2026 05:43:39.491 7f77c117ef20 [contrail-config-database-nodemgr] [ERROR]: Failed to connect to database by CQL: Result: 1 03/23/2026 05:43:39.491 7f77c117ef20 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1774244619491761, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.38:8086'], status='Up', description='ClientInit to Established on EvSandeshCtrlMessageRecv')]), flag_infos=None, description='Cassandra state detected DOWN. ')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 03/23/2026 05:43:46.174 7f77c117ef20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 05:43:52.812 7f77c117ef20 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr config_db_uuid'. Error is: Result: 1 03/23/2026 05:43:55.830 7f77c117ef20 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr to_bgp_keyspace'. Error is: Result: 1 03/23/2026 05:43:57.038 7f77c117ef20 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr svc_monitor_keyspace'. Error is: Result: 1 03/23/2026 05:44:00.750 7f77c117ef20 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr DISCOVERY_SERVER'. Error is: Result: 1 03/23/2026 05:44:05.264 7f77c117ef20 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr useragent'. Error is: Result: 1 03/23/2026 05:44:09.678 7f77c117ef20 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr dm_keyspace'. Error is: Result: 1 03/23/2026 05:44:14.725 7f77c117ef20 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1774244654724909, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774244607000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774244607000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774244607000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-10-06 01:00:31.164147", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11344-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 03/23/2026 05:44:14.725 7f77c117ef20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8505112, partition_space_available_1k=72585372, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1311436, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=265340, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44328, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=5581644, free=2628440, buffers=301696, cached=8013968, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=16.74, five_min_avg=7.42, fifteen_min_avg=3.26, cpu_share=0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 05:44:34.594 7f77c117ef20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 05:44:35.793 7f77c117ef20 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1774244675793465, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.38:8086'], status='Up', description='ClientInit to Established on EvSandeshCtrlMessageRecv')]), flag_infos=None, description='')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 03/23/2026 05:44:39.048 7f77c117ef20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 05:44:44.641 7f77c117ef20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=9021668, partition_space_available_1k=72068816, percentage_partition_space_used=11, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1325612, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=415588, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44344, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=5772628, free=1668400, buffers=337620, cached=8747556, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=16.69, five_min_avg=8.27, fifteen_min_avg=3.68, cpu_share=20.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 05:45:34.884 7f77c117ef20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 05:45:40.809 7f77c117ef20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 05:45:46.692 7f77c117ef20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=10599344, partition_space_available_1k=70491140, percentage_partition_space_used=13, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1638324, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=427540, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44372, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=5813008, free=526504, buffers=364516, cached=9821580, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=10.78, five_min_avg=8.22, fifteen_min_avg=3.97, cpu_share=17.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 05:45:55.891 7f77ba172448 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 03/23/2026 05:45:55.892 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Established] 03/23/2026 05:45:55.897 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected 03/23/2026 05:45:55.900 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[Established] -> State[Connect] 03/23/2026 05:45:55.901 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:45:55.901 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Established to Connect on EvTcpClose >>, ] description = Collector:Collector[Established to Connect on EvTcpClose] connection down >>, ] >> 03/23/2026 05:45:55.901 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:45:55.901 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 03/23/2026 05:45:55.901 7f77ba1b7048 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 03/23/2026 05:45:55.902 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 03/23/2026 05:45:55.903 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:45:55.906 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:45:55.974 7f77ba172448 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 03/23/2026 05:45:55.974 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit] 03/23/2026 05:45:55.975 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle] 03/23/2026 05:45:55.975 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:45:55.975 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = ClientInit to Idle on EvTcpClose >>, ] description = Collector:Collector[ClientInit to Idle on EvTcpClose] connection down >>, ] >> 03/23/2026 05:45:55.975 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:45:55.975 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:45:55.975 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 2 pid = 1 http_port = 8112 start_time = 1774244613086459 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:45:55.975 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:45:59.977 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:45:59.978 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected 03/23/2026 05:45:59.978 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:45:59.978 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:45:59.978 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:45:59.978 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:45:59.978 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 03/23/2026 05:45:59.979 7f77ba1b7048 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 03/23/2026 05:45:59.979 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 03/23/2026 05:45:59.979 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:45:59.979 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:45:59.985 7f77ba172448 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 03/23/2026 05:45:59.985 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit] 03/23/2026 05:45:59.985 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle] 03/23/2026 05:45:59.985 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:45:59.986 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = ClientInit to Idle on EvTcpClose >>, ] description = Collector:Collector[ClientInit to Idle on EvTcpClose] connection down >>, ] >> 03/23/2026 05:45:59.986 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:45:59.986 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:45:59.986 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 3 pid = 1 http_port = 8112 start_time = 1774244613086459 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:45:59.986 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:03.988 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:03.990 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected 03/23/2026 05:46:03.991 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:03.991 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:03.991 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:03.991 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:03.991 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 03/23/2026 05:46:03.991 7f77ba1b7048 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 03/23/2026 05:46:03.991 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 03/23/2026 05:46:03.992 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:46:03.992 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:46:07.622 7f77ba172448 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 03/23/2026 05:46:07.622 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit] 03/23/2026 05:46:07.623 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle] 03/23/2026 05:46:07.623 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:07.623 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = ClientInit to Idle on EvTcpClose >>, ] description = Collector:Collector[ClientInit to Idle on EvTcpClose] connection down >>, ] >> 03/23/2026 05:46:07.624 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:07.624 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:07.624 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774244613086459 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:46:07.624 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:11.627 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:11.629 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:46:11.629 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:11.629 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:11.629 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:11.629 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:11.629 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:46:11.629 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:46:11.630 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:11.630 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:46:11.630 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:11.630 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:11.630 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774244613086459 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:46:11.630 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:15.632 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:15.633 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:46:15.633 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:15.633 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:15.634 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:15.634 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:15.634 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:46:15.634 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:46:15.634 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:15.634 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:46:15.634 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:15.634 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:15.634 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774244613086459 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:46:15.634 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:19.634 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:19.635 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:46:19.635 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:19.635 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:19.636 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:19.636 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:19.636 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:46:19.636 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:46:19.636 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:19.636 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:46:19.636 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:19.637 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:19.637 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774244613086459 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:46:19.637 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:23.635 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:23.637 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:46:23.637 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:23.638 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:23.638 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:23.638 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:23.638 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:46:23.639 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:46:23.641 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:23.641 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:46:23.641 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:23.641 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:23.641 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774244613086459 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:46:23.641 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:27.638 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:27.639 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:46:27.639 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:27.639 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:27.639 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:27.640 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:27.640 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:46:27.640 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:46:27.640 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:27.640 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:46:27.640 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:27.640 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:27.640 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774244613086459 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:46:27.640 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:31.642 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:31.643 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:46:31.644 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:31.644 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:31.644 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:31.644 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:31.644 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:46:31.644 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:46:31.645 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:31.645 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:46:31.645 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:31.645 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:31.645 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774244613086459 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:46:31.645 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:34.461 7f77c117ef20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 05:46:34.753 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:34.753 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. database_usage = [ << disk_space_used_1k = 10481724 disk_space_available_1k = 70608760 config_db_size_1k = 2888 >>, ] >> 03/23/2026 05:46:34.753 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:35.644 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:35.646 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:46:35.646 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:35.646 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:35.646 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:35.646 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:35.646 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:46:35.646 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:46:35.646 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:35.647 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:46:35.647 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:35.647 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:35.647 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774244613086459 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:46:35.647 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:36.143 7f77c117ef20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 05:46:39.646 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:39.648 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:46:39.648 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:39.649 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:39.649 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:39.649 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:39.649 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:46:39.649 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:46:39.649 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:39.650 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:46:39.650 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:39.650 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:39.650 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774244613086459 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:46:39.650 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:41.734 7f77c117ef20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=10482152, partition_space_available_1k=70608332, percentage_partition_space_used=13, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1753004, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=423988, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44424, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=4523636, free=1910004, buffers=370848, cached=9719852, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.15, five_min_avg=7.09, fifteen_min_avg=3.83, cpu_share=5.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 05:46:41.734 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:41.734 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 10482152 partition_space_available_1k = 70608332 percentage_partition_space_used = 13 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1753004 cpu_share = 0 mem_res = 0 >>zookeeper : << mem_virt = 423988 cpu_share = 0 mem_res = 0 >>contrail-config-database-nodemgr : << mem_virt = 44424 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175636 used = 4523636 free = 1910004 buffers = 370848 cached = 9719852 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 5.15 five_min_avg = 7.09 fifteen_min_avg = 3.83 cpu_share = 5.0 node_type = config-database-node >> >> 03/23/2026 05:46:41.735 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:43.649 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:43.651 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:46:43.651 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:43.652 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:43.652 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:43.652 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:43.652 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:46:43.653 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:46:43.653 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:43.653 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:46:43.653 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:43.653 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:43.653 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774244613086459 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:46:43.654 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:47.654 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:47.655 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:46:47.655 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:47.655 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:47.655 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:47.655 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:47.655 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:46:47.656 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:46:47.656 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:47.656 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:46:47.656 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:47.656 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:47.656 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774244613086459 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:46:47.656 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:51.656 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:51.657 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:46:51.657 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:51.657 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:51.657 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:51.657 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:51.658 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:46:51.658 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:46:51.658 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:51.658 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:46:51.658 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:51.658 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:51.658 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774244613086459 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:46:51.658 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:55.659 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:55.661 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:46:55.661 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:55.661 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:55.662 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:55.662 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:55.662 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:46:55.662 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:46:55.662 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:55.662 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:46:55.662 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:55.662 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:55.663 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774244613086459 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:46:55.663 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:59.663 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:46:59.664 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:46:59.664 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:46:59.664 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:59.664 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:46:59.664 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:46:59.664 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:46:59.665 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:46:59.665 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:59.665 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:46:59.665 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:59.665 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:46:59.665 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774244613086459 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:46:59.665 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:03.667 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:47:03.670 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:47:03.671 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:47:03.671 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:03.671 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:47:03.671 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:03.671 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:47:03.671 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:47:03.672 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:03.672 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:47:03.672 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:03.672 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:03.672 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774244613086459 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:47:03.672 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:07.671 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:47:07.672 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:47:07.672 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:47:07.672 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:07.672 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:47:07.672 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:07.672 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:47:07.673 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:47:07.673 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:07.673 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:47:07.673 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:07.673 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:07.673 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774244613086459 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:47:07.673 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:11.675 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:47:11.676 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:47:11.676 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:47:11.676 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:11.676 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:47:11.676 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:11.677 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:47:11.677 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:47:11.677 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:11.677 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:47:11.677 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:11.678 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:11.678 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774244613086459 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:47:11.678 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:15.678 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:47:15.679 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:47:15.680 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:47:15.680 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:15.680 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:47:15.680 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:15.680 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:47:15.681 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:47:15.681 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:15.681 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:47:15.681 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:15.681 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:15.681 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774244613086459 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:47:15.681 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:19.682 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:47:19.684 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:47:19.684 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:47:19.685 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:19.685 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:47:19.685 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:19.685 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:47:19.685 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:47:19.685 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:19.685 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:47:19.685 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:19.685 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:19.685 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774244613086459 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:47:19.686 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:23.685 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:47:23.686 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:47:23.686 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:47:23.686 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:23.686 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:47:23.686 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:23.687 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:47:23.687 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:47:23.687 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:23.687 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:47:23.687 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:23.687 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:23.688 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774244613086459 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:47:23.688 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:27.689 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:47:27.690 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/23/2026 05:47:27.690 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:47:27.691 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:27.691 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:47:27.691 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:27.691 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/23/2026 05:47:27.691 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/23/2026 05:47:27.691 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:27.691 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/23/2026 05:47:27.691 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:27.691 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:27.691 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774244613086459 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/23/2026 05:47:27.692 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:47:31.691 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:47:31.692 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected 03/23/2026 05:47:31.692 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:47:31.692 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:31.692 7f77ba1b7048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:47:31.692 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:47:31.693 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 03/23/2026 05:47:31.693 7f77ba1b7048 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 03/23/2026 05:47:31.693 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 03/23/2026 05:47:31.693 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:47:31.693 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:47:31.741 7f77ba172448 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001016 03/23/2026 05:47:31.741 7f77ba172448 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 03/23/2026 05:47:31.741 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 03/23/2026 05:47:31.742 7f77ba1b7048 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 03/23/2026 05:47:31.742 7f77ba1b7048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 03/23/2026 05:47:31.742 7f77ba1b7048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 03/23/2026 05:47:31.742 7f77ba1b7048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/23/2026 05:47:31.742 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 03/23/2026 05:47:31.742 7f77ba1b7448 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 03/23/2026 05:47:31.743 7f77ba1b7448 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 03/23/2026 05:47:34.854 7f77c117ef20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 05:47:39.002 7f77c117ef20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 05:47:41.817 7f77ba172448 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001438 03/23/2026 05:47:41.818 7f77ba172448 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 03/23/2026 05:47:41.818 7f77ba1b7048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 03/23/2026 05:47:41.818 7f77ba1b7048 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 3 03/23/2026 05:47:41.819 7f77ba1b7048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 03/23/2026 05:47:41.819 7f77ba1b7048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 03/23/2026 05:47:41.819 7f77ba1b7048 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/23/2026 05:47:41.819 7f77ba1b7448 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 03/23/2026 05:47:41.820 7f77ba1b7448 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 03/23/2026 05:47:44.507 7f77c117ef20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=10695176, partition_space_available_1k=70395308, percentage_partition_space_used=13, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1808516, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=424264, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44532, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6934840, free=171844, buffers=372504, cached=9004924, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.29, five_min_avg=6.41, fifteen_min_avg=3.8, cpu_share=11.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 05:48:34.608 7f77c117ef20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 05:48:36.834 7f77c117ef20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 05:48:42.480 7f77c117ef20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=10834560, partition_space_available_1k=70255924, percentage_partition_space_used=13, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1837868, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=424180, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44556, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6946172, free=276516, buffers=378832, cached=8852776, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.65, five_min_avg=5.54, fifteen_min_avg=3.66, cpu_share=9.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 05:49:34.569 7f77c117ef20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 05:49:36.375 7f77c117ef20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 05:49:41.505 7f77c117ef20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=10884904, partition_space_available_1k=70205580, percentage_partition_space_used=13, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1872272, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=423944, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44592, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7044684, free=316076, buffers=380456, cached=8715320, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.26, five_min_avg=4.91, fifteen_min_avg=3.56, cpu_share=8.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 05:50:34.513 7f77c117ef20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 05:50:36.255 7f77c117ef20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 05:50:41.505 7f77c117ef20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=10900192, partition_space_available_1k=70190292, percentage_partition_space_used=13, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2074428, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=408980, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44612, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7269112, free=234344, buffers=380360, cached=8574112, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.12, five_min_avg=4.38, fifteen_min_avg=3.46, cpu_share=7.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 05:51:24.815 7f77ba172248 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_process_state:zookeeper,group:zookeeper,state:PROCESS_STATE_EXITED 03/23/2026 05:51:24.816 7f77ba172248 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1774245084815919, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774244607000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_EXITED', start_count=1, stop_count=0, exit_count=1, last_start_time='1774244607000000', last_stop_time='', last_exit_time='1774245084815870', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774244607000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-10-06 01:00:31.164147", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11344-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 03/23/2026 05:51:41.810 7f5aee3da2c0 [contrail-config-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 03/23/2026 05:51:41.810 7f5aee3da2c0 [contrail-config-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 03/23/2026 05:51:41.810 7f5aee3da2c0 [contrail-config-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 03/23/2026 05:51:41.829 7f5aeb339ea0 [contrail-config-database-nodemgr] [ERROR]: Cannot write http_port 8112 to /tmp/contrail-config-database-nodemgr.0.http_port 03/23/2026 05:51:41.829 7f5aeb339ea0 [contrail-config-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8112 03/23/2026 05:51:41.834 7f5aeb4ac2c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 03/23/2026 05:51:41.835 7f5aeb4ac2c0 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 03/23/2026 05:51:41.835 7f5aeb4ac2c0 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/23/2026 05:51:41.835 7f5aeb4ac2c0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 03/23/2026 05:51:41.835 7f5aeb4ac2c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:51:41.835 7f5aeb4ac2c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/23/2026 05:51:41.835 7f5aeb4ac2c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:51:41.835 7f5aeb4ac2c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:51:41.836 7f5aeb4ac2c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774245101835405 collector_name = collector_ip = collector_list = [ 10.0.0.38:8086, 10.0.0.50:8086, 10.0.0.33:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 03/23/2026 05:51:41.836 7f5aeb4ac2c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:51:42.133 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 11919437999148458603425851049136047508783505283213456741494531921597492753953, 'start': '1774245098000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'zookeeper', 'group': 'zookeeper', 'pid': 2041598588641536593071540382177349222172203510198549344786175061735615275451, 'start': '1774245098000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-config-database-nodemgr', 'group': 'contrail-config-database-nodemgr', 'pid': 111474724924553835336804673813322587034144335217738448292587597565953271293779, 'start': '1774245099000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 03/23/2026 05:51:42.187 7f5aeb4acae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:51:42.187 7f5aeb4acae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. build_info = {"build-info": [{"build-version": "v0+nightly", "build-time": "2026-03-23 00:50:56.516868", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-14440-1.novalocal", "build-id": "v0+nightly", "build-number": "202603230008"}]} system_cpu_info = << num_socket = 1 num_cpu = 4 num_core_per_socket = 4 num_thread_per_core = 1 >> >> 03/23/2026 05:51:42.187 7f5aeb4acae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:51:42.188 7f5aee3da2c0 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1774245102187943, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 03/23/2026 05:51:42.188 7f5aee3da2c0 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1774245102188239, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774245098000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774245098000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774245099000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info": [{"build-version": "v0+nightly", "build-time": "2026-03-23 00:50:56.516868", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-14440-1.novalocal", "build-id": "v0+nightly", "build-number": "202603230008"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 03/23/2026 05:51:42.198 7f5aeb4acae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:51:42.198 7f5aeb4acae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/23/2026 05:51:42.198 7f5aeb4acae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:51:42.198 7f5aeb4acae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:51:42.198 7f5aeb4acae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774245098000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = zookeeper process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774245098000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774245099000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info": [{"build-version": "v0+nightly", "build-time": "2026-03-23 00:50:56.516868", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-14440-1.novalocal", "build-id": "v0+nightly", "build-number": "202603230008"}]} >> 03/23/2026 05:51:42.198 7f5aeb4acae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:51:42.573 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 05:51:43.173 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 03/23/2026 05:51:43.174 7f5aeb2fc040 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:51:43.174 7f5aeb2fc040 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. database_usage = [ << disk_space_used_1k = 11998888 disk_space_available_1k = 69091596 config_db_size_1k = 4008 >>, ] >> 03/23/2026 05:51:43.174 7f5aeb2fc040 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:51:43.934 7f5aee3da2c0 [contrail-config-database-nodemgr] [ERROR]: Failed to connect to database by CQL and socket: Result: 1 03/23/2026 05:51:43.934 7f5aee3da2c0 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1774245103934402, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 03/23/2026 05:51:43.935 7f5aeb4acea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:51:43.935 7f5aeb4acea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/23/2026 05:51:43.935 7f5aeb4acea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/23/2026 05:51:45.836 7f5aeb4acea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/23/2026 05:51:45.837 7f5aeb4acea0 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected 03/23/2026 05:51:45.837 7f5aeb4acea0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/23/2026 05:51:45.837 7f5aeb4acea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:51:45.838 7f5aeb4acea0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5515-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/23/2026 05:51:45.838 7f5aeb4acea0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/23/2026 05:51:45.838 7f5aeb4acea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 03/23/2026 05:51:45.838 7f5aeb4acea0 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 03/23/2026 05:51:45.838 7f5aeb4acea0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 03/23/2026 05:51:45.838 7f5aeb4acea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:51:45.838 7f5aeb4acea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/23/2026 05:51:45.841 7f5aeb2fc180 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001017 03/23/2026 05:51:45.842 7f5aeb2fc180 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 03/23/2026 05:51:45.843 7f5aeb4acea0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 03/23/2026 05:51:45.843 7f5aeb4acea0 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 03/23/2026 05:51:45.843 7f5aeb4acea0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 03/23/2026 05:51:45.843 7f5aeb4acea0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 03/23/2026 05:51:45.843 7f5aeb4acea0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/23/2026 05:51:45.843 7f5aeb4acea0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 03/23/2026 05:51:45.844 7f5aeb2fcea0 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 03/23/2026 05:51:45.844 7f5aeb2fcea0 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 03/23/2026 05:51:46.353 7f5aee3da2c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 05:51:49.140 7f5aee3da2c0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr config_db_uuid'. Error is: Result: 1 03/23/2026 05:51:50.716 7f5aee3da2c0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr to_bgp_keyspace'. Error is: Result: 1 03/23/2026 05:51:51.722 7f5aee3da2c0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr svc_monitor_keyspace'. Error is: Result: 1 03/23/2026 05:51:52.981 7f5aee3da2c0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr DISCOVERY_SERVER'. Error is: Result: 1 03/23/2026 05:51:54.293 7f5aee3da2c0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr useragent'. Error is: Result: 1 03/23/2026 05:51:55.088 7f5aee3da2c0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr dm_keyspace'. Error is: Result: 1 03/23/2026 05:52:00.503 7f5aee3da2c0 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1774245120503726, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774245098000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774245098000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774245099000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info": [{"build-version": "v0+nightly", "build-time": "2026-03-23 00:50:56.516868", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-14440-1.novalocal", "build-id": "v0+nightly", "build-number": "202603230008"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 03/23/2026 05:52:00.504 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=11993052, partition_space_available_1k=69097432, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1326448, cpu_share=0.0, mem_res=1309652, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=212876, cpu_share=0.0, mem_res=185944, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56356, cpu_share=0.0, mem_res=45188, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=5027964, free=2564748, buffers=408260, cached=8496752, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.26, five_min_avg=4.71, fifteen_min_avg=3.64, cpu_share=0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 05:52:42.388 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 05:52:43.181 7f5aee3da2c0 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5515-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1774245163180837, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.38:8086'], status='Up', description='ClientInit to Established on EvSandeshCtrlMessageRecv')]), flag_infos=None, description='')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 03/23/2026 05:52:43.980 7f5aee3da2c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 05:52:49.528 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12010128, partition_space_available_1k=69080356, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1330696, cpu_share=0.19, mem_res=1311468, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=253312, cpu_share=0.04, mem_res=224860, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56372, cpu_share=0.01, mem_res=45188, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6528988, free=1027388, buffers=410044, cached=8530028, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.01, five_min_avg=4.47, fifteen_min_avg=3.61, cpu_share=12.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 05:53:42.437 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 05:53:44.014 7f5aee3da2c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 05:53:49.526 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12011920, partition_space_available_1k=69078564, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1370948, cpu_share=0.08, mem_res=1349552, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=242888, cpu_share=0.01, mem_res=214440, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56376, cpu_share=0.01, mem_res=45188, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6648616, free=906676, buffers=410068, cached=8530988, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.11, five_min_avg=3.84, fifteen_min_avg=3.45, cpu_share=8.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 05:54:42.424 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 05:54:43.969 7f5aee3da2c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 05:54:49.547 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12017660, partition_space_available_1k=69072824, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1349828, cpu_share=0.05, mem_res=1328460, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=242976, cpu_share=0.0, mem_res=214524, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56376, cpu_share=0.01, mem_res=45188, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6645472, free=903732, buffers=410080, cached=8537176, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.32, five_min_avg=3.3, fifteen_min_avg=3.29, cpu_share=7.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 05:55:42.411 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 05:55:43.887 7f5aee3da2c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 05:55:49.547 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12018108, partition_space_available_1k=69072376, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1340724, cpu_share=0.04, mem_res=1319372, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=243004, cpu_share=0.0, mem_res=214548, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56384, cpu_share=0.01, mem_res=45188, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6672068, free=875692, buffers=410088, cached=8537580, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.05, five_min_avg=2.86, fifteen_min_avg=3.13, cpu_share=7.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 05:56:42.476 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 05:56:43.990 7f5aee3da2c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 05:56:49.564 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12023912, partition_space_available_1k=69066572, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1318112, cpu_share=0.04, mem_res=1296756, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=243116, cpu_share=0.0, mem_res=214652, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56408, cpu_share=0.01, mem_res=45188, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6654032, free=887840, buffers=410092, cached=8543468, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.88, five_min_avg=2.49, fifteen_min_avg=2.99, cpu_share=7.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 05:57:42.447 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 05:57:44.018 7f5aee3da2c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 05:57:49.593 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12032164, partition_space_available_1k=69058320, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1343600, cpu_share=0.06, mem_res=1318944, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=249148, cpu_share=0.01, mem_res=220676, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56404, cpu_share=0.01, mem_res=45188, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6681820, free=851084, buffers=410100, cached=8552284, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.46, five_min_avg=2.33, fifteen_min_avg=2.9, cpu_share=7.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 05:58:42.461 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 05:58:43.939 7f5aee3da2c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 05:58:49.587 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12033124, partition_space_available_1k=69057360, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1339796, cpu_share=0.04, mem_res=1315180, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=249224, cpu_share=0.0, mem_res=220732, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56416, cpu_share=0.01, mem_res=45188, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6691440, free=840804, buffers=410104, cached=8552892, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.3, five_min_avg=2.12, fifteen_min_avg=2.79, cpu_share=6.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 05:59:42.455 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 05:59:44.002 7f5aee3da2c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 05:59:49.595 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12038492, partition_space_available_1k=69051992, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1327680, cpu_share=0.04, mem_res=1303052, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=249300, cpu_share=0.0, mem_res=220812, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56412, cpu_share=0.01, mem_res=45188, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6676700, free=850036, buffers=410104, cached=8558512, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.11, five_min_avg=1.92, fifteen_min_avg=2.68, cpu_share=6.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 06:00:42.474 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 06:00:44.024 7f5aee3da2c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 06:00:49.617 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12034724, partition_space_available_1k=69055760, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1339644, cpu_share=0.05, mem_res=1314972, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=249312, cpu_share=0.0, mem_res=220824, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56420, cpu_share=0.01, mem_res=45188, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6745044, free=785448, buffers=410108, cached=8554820, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.4, five_min_avg=1.84, fifteen_min_avg=2.6, cpu_share=7.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 06:01:42.482 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 06:01:43.951 7f5aee3da2c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 06:01:49.615 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12040564, partition_space_available_1k=69049920, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1342108, cpu_share=0.04, mem_res=1317436, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=249320, cpu_share=0.0, mem_res=220828, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56420, cpu_share=0.01, mem_res=45188, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6743120, free=781548, buffers=410108, cached=8560572, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.14, five_min_avg=1.68, fifteen_min_avg=2.5, cpu_share=6.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 06:02:42.472 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 06:02:43.931 7f5aee3da2c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 06:02:49.628 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12046568, partition_space_available_1k=69043916, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1348296, cpu_share=0.05, mem_res=1322772, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=249316, cpu_share=0.0, mem_res=220828, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56428, cpu_share=0.01, mem_res=45188, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6762384, free=755760, buffers=410108, cached=8567128, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.23, five_min_avg=1.61, fifteen_min_avg=2.42, cpu_share=7.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 06:03:42.482 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 06:03:43.962 7f5aee3da2c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 06:03:49.635 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12047304, partition_space_available_1k=69043180, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1338968, cpu_share=0.04, mem_res=1313384, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=249324, cpu_share=0.0, mem_res=220828, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56436, cpu_share=0.01, mem_res=45188, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6745160, free=770904, buffers=410108, cached=8569132, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.23, five_min_avg=1.53, fifteen_min_avg=2.34, cpu_share=7.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 06:04:42.502 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 06:04:44.074 7f5aee3da2c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 06:04:49.656 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12052412, partition_space_available_1k=69038072, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1330332, cpu_share=0.04, mem_res=1304768, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=249324, cpu_share=0.0, mem_res=220836, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56436, cpu_share=0.01, mem_res=45188, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6747524, free=760416, buffers=410156, cached=8577148, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.42, five_min_avg=1.56, fifteen_min_avg=2.3, cpu_share=7.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 06:05:42.501 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 06:05:44.048 7f5aee3da2c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 06:05:49.652 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12053008, partition_space_available_1k=69037476, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1333572, cpu_share=0.04, mem_res=1307960, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=249324, cpu_share=0.0, mem_res=220836, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56444, cpu_share=0.01, mem_res=45188, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6772672, free=734640, buffers=410156, cached=8577900, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.15, five_min_avg=1.46, fifteen_min_avg=2.22, cpu_share=6.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 06:06:42.518 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 06:06:44.027 7f5aee3da2c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 06:06:49.673 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12058392, partition_space_available_1k=69032092, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1347976, cpu_share=0.05, mem_res=1322300, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=249344, cpu_share=0.0, mem_res=220856, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56448, cpu_share=0.01, mem_res=45188, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6760344, free=746532, buffers=410164, cached=8578364, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.2, five_min_avg=1.41, fifteen_min_avg=2.15, cpu_share=6.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 06:07:42.550 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 06:07:44.089 7f5aee3da2c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 06:07:49.680 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12060264, partition_space_available_1k=69030220, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1338204, cpu_share=0.05, mem_res=1311648, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=249348, cpu_share=0.0, mem_res=220860, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56444, cpu_share=0.01, mem_res=45188, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6810532, free=546300, buffers=411396, cached=8726420, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.53, five_min_avg=1.45, fifteen_min_avg=2.12, cpu_share=7.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 06:07:52.124 7f5aeb4ac360 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 03/23/2026 06:07:57.144 7f5aeb4ac360 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 03/23/2026 06:08:42.536 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 06:08:44.011 7f5aee3da2c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 06:08:49.674 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12205364, partition_space_available_1k=68885120, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1337508, cpu_share=0.04, mem_res=1310996, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=249356, cpu_share=0.0, mem_res=220864, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56452, cpu_share=0.01, mem_res=45188, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6810324, free=533620, buffers=413124, cached=8737312, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.76, five_min_avg=1.5, fifteen_min_avg=2.09, cpu_share=7.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 06:09:42.548 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 06:09:44.054 7f5aee3da2c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 06:09:49.688 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12213240, partition_space_available_1k=68877244, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1356724, cpu_share=0.08, mem_res=1329992, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=249388, cpu_share=0.0, mem_res=220892, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56452, cpu_share=0.01, mem_res=45188, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6854556, free=479400, buffers=413144, cached=8747200, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.19, five_min_avg=1.38, fifteen_min_avg=2.01, cpu_share=7.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 06:10:42.560 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 06:10:44.273 7f5aee3da2c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 06:10:49.694 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12220664, partition_space_available_1k=68869820, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1364292, cpu_share=0.18, mem_res=1337264, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=249636, cpu_share=0.0, mem_res=221112, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56460, cpu_share=0.01, mem_res=45188, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6895540, free=429072, buffers=413152, cached=8756132, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.77, five_min_avg=1.53, fifteen_min_avg=2.02, cpu_share=8.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 06:11:42.557 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 06:11:44.520 7f5aee3da2c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 06:11:49.709 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12238008, partition_space_available_1k=68852476, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1381984, cpu_share=0.18, mem_res=1354644, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=268144, cpu_share=0.01, mem_res=239576, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56460, cpu_share=0.01, mem_res=45188, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6964080, free=343572, buffers=413152, cached=8773368, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.35, five_min_avg=1.8, fifteen_min_avg=2.08, cpu_share=9.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 06:12:42.562 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 06:12:44.202 7f5aee3da2c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 06:12:49.714 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12251972, partition_space_available_1k=68838512, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1355268, cpu_share=0.15, mem_res=1326780, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=258692, cpu_share=0.01, mem_res=230116, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56476, cpu_share=0.01, mem_res=45188, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6963776, free=343144, buffers=412392, cached=8773256, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.76, five_min_avg=2.09, fifteen_min_avg=2.17, cpu_share=9.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 06:13:42.575 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 06:13:44.059 7f5aee3da2c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 06:13:49.709 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12269412, partition_space_available_1k=68821072, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1353232, cpu_share=0.11, mem_res=1324684, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=258792, cpu_share=0.0, mem_res=230200, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56468, cpu_share=0.01, mem_res=45188, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=6974532, free=336368, buffers=412376, cached=8769552, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.38, five_min_avg=2.1, fifteen_min_avg=2.16, cpu_share=8.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 06:14:42.572 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 06:14:44.394 7f5aee3da2c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 06:14:49.712 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12280348, partition_space_available_1k=68810136, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1369856, cpu_share=0.1, mem_res=1341068, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=258756, cpu_share=0.01, mem_res=230232, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56464, cpu_share=0.01, mem_res=45188, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7000544, free=338452, buffers=411776, cached=8742408, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.87, five_min_avg=2.22, fifteen_min_avg=2.19, cpu_share=8.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 06:15:42.579 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 06:15:44.110 7f5aee3da2c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 06:15:49.724 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12306560, partition_space_available_1k=68783924, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1354496, cpu_share=0.08, mem_res=1325728, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=258652, cpu_share=0.0, mem_res=230376, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56440, cpu_share=0.01, mem_res=45188, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7095084, free=319264, buffers=411356, cached=8668204, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.44, five_min_avg=2.36, fifteen_min_avg=2.24, cpu_share=8.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 06:16:42.589 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 06:16:44.096 7f5aee3da2c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 06:16:49.723 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12316988, partition_space_available_1k=68773496, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1384344, cpu_share=0.1, mem_res=1355508, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=258596, cpu_share=0.0, mem_res=230380, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56436, cpu_share=0.01, mem_res=45188, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7077944, free=356968, buffers=410312, cached=8648988, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.03, five_min_avg=2.54, fifteen_min_avg=2.31, cpu_share=8.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 06:17:42.602 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 06:17:44.087 7f5aee3da2c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 06:17:49.734 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12331300, partition_space_available_1k=68759184, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1357928, cpu_share=0.08, mem_res=1328072, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=256604, cpu_share=0.01, mem_res=228408, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56436, cpu_share=0.01, mem_res=45188, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7096288, free=333812, buffers=410016, cached=8653784, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.2, five_min_avg=2.39, fifteen_min_avg=2.27, cpu_share=8.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 06:18:42.610 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 06:18:44.127 7f5aee3da2c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 06:18:49.737 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12338692, partition_space_available_1k=68751792, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1363336, cpu_share=0.07, mem_res=1333456, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=256496, cpu_share=0.0, mem_res=228424, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56428, cpu_share=0.01, mem_res=45188, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7113368, free=358516, buffers=408872, cached=8613780, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.39, five_min_avg=2.12, fifteen_min_avg=2.18, cpu_share=7.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 06:19:42.618 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 06:19:44.125 7f5aee3da2c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 06:19:49.736 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12346308, partition_space_available_1k=68744176, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1367352, cpu_share=0.08, mem_res=1337128, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=256444, cpu_share=0.0, mem_res=228428, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56424, cpu_share=0.01, mem_res=45188, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7202080, free=266092, buffers=408732, cached=8617816, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.78, five_min_avg=2.44, fifteen_min_avg=2.29, cpu_share=7.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 06:20:42.615 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 06:20:44.155 7f5aee3da2c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 06:20:49.755 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12367248, partition_space_available_1k=68723236, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1366512, cpu_share=0.07, mem_res=1335736, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=256220, cpu_share=0.0, mem_res=228464, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56404, cpu_share=0.01, mem_res=45188, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7177364, free=370376, buffers=408740, cached=8539036, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.61, five_min_avg=2.42, fifteen_min_avg=2.29, cpu_share=7.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 06:21:42.615 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 06:21:44.143 7f5aee3da2c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 06:21:49.754 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12372636, partition_space_available_1k=68717848, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1366604, cpu_share=0.08, mem_res=1335880, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=257628, cpu_share=0.0, mem_res=229860, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56396, cpu_share=0.01, mem_res=45188, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7192860, free=339272, buffers=409032, cached=8554484, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.01, five_min_avg=2.56, fifteen_min_avg=2.35, cpu_share=8.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 06:22:42.613 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 06:22:44.088 7f5aee3da2c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/23/2026 06:22:49.754 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5515-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12379708, partition_space_available_1k=68710776, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1370200, cpu_share=0.04, mem_res=1338692, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=257556, cpu_share=0.0, mem_res=229860, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=56400, cpu_share=0.01, mem_res=45188, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175636, used=7200296, free=352212, buffers=409040, cached=8534080, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.24, five_min_avg=2.41, fifteen_min_avg=2.31, cpu_share=6.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/23/2026 06:23:41.957 7f5aeb2fc7c0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 03/23/2026 06:23:42.632 7f5aee3da2c0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/23/2026 06:23:44.372 7f5aee3da2c0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1