03/26/2026 05:24:54.461 7f87eae3fdf0 [contrail-config-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 03/26/2026 05:24:54.461 7f87eae3fdf0 [contrail-config-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 03/26/2026 05:24:54.462 7f87eae3fdf0 [contrail-config-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 03/26/2026 05:24:54.468 7f87e56f1548 [contrail-config-database-nodemgr] [ERROR]: Cannot write http_port 8112 to /tmp/contrail-config-database-nodemgr.0.http_port 03/26/2026 05:24:54.469 7f87e56f1548 [contrail-config-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8112 03/26/2026 05:24:54.470 7f87e56f1d48 [contrail-config-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 03/26/2026 05:24:54.470 7f87e56f1d48 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 03/26/2026 05:24:54.471 7f87e56f1d48 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/26/2026 05:24:54.471 7f87e56f1d48 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 03/26/2026 05:24:54.471 7f87e56f1d48 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:24:54.471 7f87e56f1d48 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/26/2026 05:24:54.471 7f87e56f1d48 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:24:54.471 7f87e56f1d48 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:24:54.471 7f87e56f1d48 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 03/26/2026 05:24:54.471 7f87e56f1d48 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:24:54.525 7f87eae3fdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 84755038961334919458078586997610798608053589957181516736888456705188594616068, 'start': '1774502691000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'zookeeper', 'group': 'zookeeper', 'pid': 35123161966967307064494183079845637437870586061261374677531464525313043303203, 'start': '1774502691000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-config-database-nodemgr', 'group': 'contrail-config-database-nodemgr', 'pid': 56093402697086538778499493165740912784918614745172713217925070760473462031352, 'start': '1774502692000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 03/26/2026 05:24:54.682 7f87e5394048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:24:54.683 7f87e5394048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. build_info = {"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-10-06 01:00:31.164147", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11344-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 4 num_core_per_socket = 4 num_thread_per_core = 1 >> running_package_version = v0+nightly.r24.1.centos-nightly_r24.1_centos.el7 installed_package_version = v0+nightly.r24.1.centos-nightly_r24.1_centos.el7 >> 03/26/2026 05:24:54.683 7f87e5394048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:24:54.683 7f87eae3fdf0 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5546-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1774502694683508, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 03/26/2026 05:24:54.684 7f87eae3fdf0 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5546-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1774502694683932, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774502691000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774502691000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774502692000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-10-06 01:00:31.164147", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11344-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 03/26/2026 05:24:54.686 7f87e5394048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:24:54.686 7f87e5394048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/26/2026 05:24:54.686 7f87e5394048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:24:54.687 7f87e5394048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:24:54.687 7f87e5394048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774502691000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = zookeeper process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774502691000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774502692000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-10-06 01:00:31.164147", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11344-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} >> 03/26/2026 05:24:54.687 7f87e5394048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:24:54.943 7f87eae3fdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 05:24:55.130 7f87eae3fdf0 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 03/26/2026 05:24:55.132 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:24:55.133 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. database_usage = [ << disk_space_used_1k = 4558644 disk_space_available_1k = 76531840 config_db_size_1k = 4 >>, ] >> 03/26/2026 05:24:55.133 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:24:55.563 7f87eae3fdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to connect to database by CQL: Result: 1 03/26/2026 05:24:55.565 7f87eae3fdf0 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5546-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1774502695565104, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 03/26/2026 05:24:55.566 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:24:55.567 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/26/2026 05:24:55.567 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:24:58.428 7f87eae3fdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 05:24:58.475 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:24:58.476 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:24:58.477 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:24:58.477 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:24:58.477 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:24:58.477 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:24:58.477 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:24:58.478 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:24:58.478 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:24:58.478 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:24:58.479 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:24:58.479 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:24:58.479 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.242:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:24:58.479 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:01.149 7f87eae3fdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr config_db_uuid'. Error is: Result: 1 03/26/2026 05:25:02.100 7f87eae3fdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr to_bgp_keyspace'. Error is: Result: 1 03/26/2026 05:25:02.477 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:25:02.479 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:25:02.480 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:25:02.480 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:25:02.480 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:25:02.480 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:25:02.480 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:25:02.481 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:25:02.481 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:02.481 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:25:02.482 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:02.482 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:02.482 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:25:02.482 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:03.224 7f87eae3fdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr svc_monitor_keyspace'. Error is: Result: 1 03/26/2026 05:25:04.399 7f87eae3fdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr DISCOVERY_SERVER'. Error is: Result: 1 03/26/2026 05:25:05.451 7f87eae3fdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr useragent'. Error is: Result: 1 03/26/2026 05:25:06.479 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:25:06.481 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:25:06.482 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:25:06.482 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:25:06.482 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:25:06.482 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:25:06.482 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:25:06.483 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:25:06.483 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:06.483 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:25:06.484 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:06.484 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:06.485 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:25:06.485 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:06.618 7f87eae3fdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr dm_keyspace'. Error is: Result: 1 03/26/2026 05:25:10.482 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:25:10.485 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:25:10.486 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:25:10.486 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:25:10.486 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:25:10.487 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:25:10.487 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:25:10.488 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:25:10.488 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:10.488 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:25:10.489 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:10.489 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:10.489 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.242:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:25:10.489 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:11.650 7f87eae3fdf0 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5546-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1774502711650425, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774502691000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774502691000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774502692000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-10-06 01:00:31.164147", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11344-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 03/26/2026 05:25:11.650 7f87eae3fdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=4910776, partition_space_available_1k=76179708, percentage_partition_space_used=6, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1246872, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57472, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51128, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=2102504, free=9899900, buffers=103268, cached=4418140, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.68, five_min_avg=0.51, fifteen_min_avg=0.2, cpu_share=0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 05:25:11.651 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:11.651 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774502691000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = zookeeper process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774502691000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774502692000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-10-06 01:00:31.164147", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11344-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} >> 03/26/2026 05:25:11.651 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:11.651 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:11.651 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 4910776 partition_space_available_1k = 76179708 percentage_partition_space_used = 6 >> } all_core_file_list = [ ] process_mem_cpu_usage = { cassandra : << mem_virt = 1246872 cpu_share = 0 mem_res = 0 >>zookeeper : << mem_virt = 57472 cpu_share = 0 mem_res = 0 >>contrail-config-database-nodemgr : << mem_virt = 51128 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175640 used = 2102504 free = 9899900 buffers = 103268 cached = 4418140 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 1.68 five_min_avg = 0.51 fifteen_min_avg = 0.2 cpu_share = 0 node_type = config-database-node >> >> 03/26/2026 05:25:11.651 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:14.487 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:25:14.489 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:25:14.490 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:25:14.490 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:25:14.490 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:25:14.490 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:25:14.490 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:25:14.491 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:25:14.491 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:14.491 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:25:14.491 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:14.491 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:14.491 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:25:14.492 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:18.493 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:25:18.495 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:25:18.495 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:25:18.495 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:25:18.495 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:25:18.495 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:25:18.495 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:25:18.496 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:25:18.496 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:18.496 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:25:18.496 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:18.496 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:18.496 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:25:18.496 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:22.496 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:25:22.497 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:25:22.498 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:25:22.498 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:25:22.498 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:25:22.498 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:25:22.498 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:25:22.499 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:25:22.499 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:22.499 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:25:22.499 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:22.499 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:22.499 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.242:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:25:22.499 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:26.499 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:25:26.501 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:25:26.502 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:25:26.502 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:25:26.502 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:25:26.502 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:25:26.502 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:25:26.503 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:25:26.503 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:26.503 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:25:26.503 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:26.503 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:26.503 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:25:26.504 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:30.504 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:25:30.506 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:25:30.506 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:25:30.506 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:25:30.506 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:25:30.506 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:25:30.506 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:25:30.507 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:25:30.507 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:30.507 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:25:30.507 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:30.507 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:30.507 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:25:30.508 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:34.508 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:25:34.510 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:25:34.510 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:25:34.510 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:25:34.510 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:25:34.510 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:25:34.510 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:25:34.511 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:25:34.511 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:34.511 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:25:34.511 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:34.511 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:34.511 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.242:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:25:34.512 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:38.513 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:25:38.516 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:25:38.516 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:25:38.517 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:25:38.517 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:25:38.518 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:25:38.518 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:25:38.519 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:25:38.519 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:38.520 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:25:38.520 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:38.521 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:38.521 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:25:38.521 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:42.517 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:25:42.519 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:25:42.519 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:25:42.520 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:25:42.521 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:25:42.521 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:25:42.522 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:25:42.523 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:25:42.524 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:42.524 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:25:42.525 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:42.526 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:42.526 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:25:42.527 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:46.520 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:25:46.523 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:25:46.524 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:25:46.524 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:25:46.525 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:25:46.525 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:25:46.525 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:25:46.526 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:25:46.527 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:46.527 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:25:46.527 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:46.528 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:46.528 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.242:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:25:46.528 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:50.524 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:25:50.528 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:25:50.529 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:25:50.529 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:25:50.530 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:25:50.530 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:25:50.530 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:25:50.531 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:25:50.532 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:50.532 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:25:50.533 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:50.533 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:50.533 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:25:50.534 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:54.529 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:25:54.532 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:25:54.532 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:25:54.533 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:25:54.533 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:25:54.533 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:25:54.534 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:25:54.535 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:25:54.535 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:54.536 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:25:54.536 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:54.536 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:54.537 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:25:54.537 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:54.939 7f87eae3fdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 05:25:55.173 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:55.173 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. database_usage = [ << disk_space_used_1k = 5831980 disk_space_available_1k = 75258504 config_db_size_1k = 1552 >>, ] >> 03/26/2026 05:25:55.173 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:55.823 7f87eae3fdf0 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5546-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1774502755823700, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.38:8086'], status='Down', description='Connect to Idle on EvTcpConnectFail')]), flag_infos=None, description='Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 03/26/2026 05:25:55.825 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:55.826 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:25:55.826 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:57.013 7f87eae3fdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 05:25:58.532 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:25:58.534 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:25:58.535 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:25:58.535 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:25:58.535 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:25:58.535 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:25:58.535 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:25:58.535 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:25:58.536 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:58.536 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:25:58.536 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:58.536 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:25:58.536 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.242:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:25:58.536 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:02.044 7f87eae3fdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=5832088, partition_space_available_1k=75258396, percentage_partition_space_used=7, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1305948, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57312, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51396, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=2159868, free=9151936, buffers=175448, cached=5036552, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.85, five_min_avg=1.02, fifteen_min_avg=0.39, cpu_share=10.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 05:26:02.045 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:02.046 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 5832088 partition_space_available_1k = 75258396 percentage_partition_space_used = 7 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1305948 cpu_share = 0 mem_res = 0 >>zookeeper : << mem_virt = 57312 cpu_share = 0 mem_res = 0 >>contrail-config-database-nodemgr : << mem_virt = 51396 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175640 used = 2159868 free = 9151936 buffers = 175448 cached = 5036552 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 2.85 five_min_avg = 1.02 fifteen_min_avg = 0.39 cpu_share = 10.25 node_type = config-database-node >> >> 03/26/2026 05:26:02.046 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:02.535 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:26:02.539 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:26:02.539 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:26:02.539 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:26:02.539 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:26:02.539 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:26:02.539 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:26:02.540 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:26:02.540 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:02.540 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:26:02.540 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:02.541 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:02.541 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:26:02.541 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:06.543 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:26:06.544 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:26:06.545 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:26:06.545 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:26:06.546 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:26:06.546 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:26:06.546 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:26:06.546 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:26:06.547 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:06.547 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:26:06.547 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:06.547 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:06.547 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:26:06.548 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:10.545 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:26:10.546 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:26:10.547 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:26:10.547 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:26:10.547 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:26:10.547 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:26:10.547 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:26:10.548 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:26:10.548 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:10.549 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:26:10.549 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:10.549 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:10.549 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.242:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:26:10.549 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:14.548 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:26:14.550 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:26:14.550 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:26:14.550 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:26:14.550 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:26:14.551 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:26:14.551 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:26:14.551 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:26:14.552 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:14.552 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:26:14.552 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:14.552 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:14.552 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:26:14.552 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:18.552 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:26:18.557 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:26:18.557 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:26:18.557 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:26:18.557 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:26:18.557 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:26:18.557 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:26:18.558 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:26:18.558 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:18.558 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:26:18.558 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:18.558 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:18.559 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:26:18.559 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:22.560 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:26:22.562 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:26:22.562 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:26:22.562 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:26:22.562 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:26:22.563 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:26:22.563 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:26:22.563 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:26:22.563 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:22.564 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:26:22.564 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:22.564 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:22.564 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.242:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:26:22.564 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:26.562 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:26:26.564 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:26:26.565 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:26:26.565 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:26:26.565 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:26:26.567 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:26:26.567 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:26:26.568 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:26:26.568 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:26.569 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:26:26.569 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:26.569 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:26.569 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:26:26.571 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:30.568 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:26:30.570 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:26:30.570 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:26:30.570 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:26:30.571 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:26:30.571 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:26:30.571 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:26:30.571 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:26:30.571 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:30.572 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:26:30.572 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:30.572 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:30.572 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:26:30.572 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:34.572 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:26:34.576 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:26:34.577 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:26:34.578 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:26:34.578 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:26:34.579 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:26:34.579 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:26:34.579 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:26:34.580 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:34.580 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:26:34.580 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:34.580 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:34.580 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.242:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:26:34.580 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:38.580 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:26:38.582 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:26:38.582 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:26:38.582 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:26:38.582 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:26:38.583 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:26:38.583 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:26:38.583 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:26:38.583 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:38.583 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:26:38.584 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:38.584 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:38.584 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:26:38.584 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:42.584 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:26:42.586 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:26:42.587 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:26:42.587 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:26:42.587 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:26:42.588 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:26:42.588 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:26:42.588 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:26:42.589 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:42.589 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:26:42.589 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:42.589 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:42.589 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:26:42.590 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:46.588 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:26:46.590 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:26:46.591 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:26:46.591 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:26:46.593 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:26:46.594 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:26:46.594 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:26:46.594 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:26:46.595 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:46.596 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:26:46.596 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:46.596 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:46.598 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.242:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:26:46.598 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:50.591 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:26:50.594 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:26:50.595 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:26:50.595 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:26:50.595 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:26:50.596 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:26:50.596 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:26:50.597 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:26:50.598 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:50.599 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:26:50.599 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:50.599 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:50.600 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:26:50.600 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:54.597 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:26:54.599 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:26:54.599 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:26:54.600 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:26:54.600 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:26:54.600 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:26:54.601 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:26:54.601 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:26:54.602 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:54.602 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:26:54.603 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:54.603 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:54.604 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:26:54.604 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:55.083 7f87eae3fdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 05:26:55.354 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:55.355 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. database_usage = [ << disk_space_used_1k = 6247976 disk_space_available_1k = 74842508 config_db_size_1k = 1792 >>, ] >> 03/26/2026 05:26:55.355 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:57.163 7f87eae3fdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 05:26:58.599 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:26:58.601 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:26:58.601 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:26:58.601 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:26:58.602 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:26:58.602 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:26:58.602 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:26:58.602 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:26:58.602 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:58.603 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:26:58.603 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:58.603 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:26:58.603 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.242:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:26:58.603 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:02.194 7f87eae3fdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6247964, partition_space_available_1k=74842520, percentage_partition_space_used=8, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1652888, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=71572, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51812, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=3176072, free=7626040, buffers=212708, cached=5509588, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.95, five_min_avg=1.64, fifteen_min_avg=0.64, cpu_share=13.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 05:27:02.195 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:02.195 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 6247964 partition_space_available_1k = 74842520 percentage_partition_space_used = 8 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1652888 cpu_share = 0 mem_res = 0 >>zookeeper : << mem_virt = 71572 cpu_share = 0 mem_res = 0 >>contrail-config-database-nodemgr : << mem_virt = 51812 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175640 used = 3176072 free = 7626040 buffers = 212708 cached = 5509588 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 3.95 five_min_avg = 1.64 fifteen_min_avg = 0.64 cpu_share = 13.0 node_type = config-database-node >> >> 03/26/2026 05:27:02.196 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:02.601 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:27:02.603 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:27:02.603 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:27:02.603 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:27:02.604 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:27:02.604 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:27:02.604 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:27:02.604 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:27:02.604 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:02.604 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:27:02.604 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:02.604 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:02.605 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:27:02.605 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:06.604 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:27:06.606 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:27:06.607 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:27:06.607 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:27:06.607 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:27:06.607 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:27:06.607 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:27:06.608 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:27:06.608 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:06.609 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:27:06.609 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:06.609 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:06.609 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:27:06.609 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:10.611 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:27:10.617 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:27:10.618 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:27:10.618 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:27:10.618 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:27:10.618 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:27:10.618 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:27:10.619 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:27:10.619 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:10.619 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:27:10.619 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:10.620 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:10.620 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.242:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:27:10.620 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:14.620 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:27:14.622 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:27:14.622 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:27:14.622 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:27:14.622 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:27:14.623 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:27:14.623 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:27:14.623 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:27:14.623 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:14.623 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:27:14.623 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:14.623 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:14.624 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:27:14.624 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:18.625 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:27:18.628 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:27:18.629 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:27:18.629 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:27:18.629 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:27:18.629 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:27:18.629 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:27:18.630 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:27:18.630 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:18.630 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:27:18.630 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:18.630 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:18.630 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:27:18.631 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:22.631 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:27:22.633 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:27:22.633 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:27:22.633 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:27:22.633 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:27:22.633 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:27:22.634 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:27:22.634 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:27:22.634 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:22.634 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:27:22.634 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:22.635 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:22.635 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.242:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:27:22.635 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:26.633 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:27:26.635 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:27:26.635 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:27:26.635 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:27:26.635 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:27:26.635 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:27:26.636 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:27:26.636 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:27:26.636 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:26.636 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:27:26.636 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:26.636 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:26.637 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:27:26.637 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:30.636 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:27:30.638 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:27:30.638 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:27:30.639 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:27:30.639 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:27:30.639 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:27:30.639 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:27:30.639 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:27:30.640 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:30.640 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:27:30.640 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:30.640 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:30.640 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:27:30.640 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:34.641 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:27:34.643 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:27:34.643 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:27:34.643 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:27:34.643 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:27:34.644 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:27:34.644 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:27:34.644 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:27:34.644 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:34.645 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:27:34.645 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:34.645 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:34.645 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.242:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:27:34.646 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:38.644 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:27:38.646 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:27:38.646 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:27:38.646 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:27:38.647 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:27:38.647 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:27:38.647 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:27:38.647 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:27:38.648 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:38.648 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:27:38.649 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:38.649 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:38.649 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:27:38.649 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:42.646 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:27:42.648 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:27:42.648 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:27:42.649 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:27:42.649 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:27:42.649 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:27:42.649 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:27:42.650 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:27:42.650 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:42.650 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:27:42.650 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:42.650 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:42.650 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:27:42.651 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:46.649 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:27:46.651 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:27:46.652 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:27:46.652 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:27:46.652 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:27:46.652 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:27:46.653 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:27:46.653 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:27:46.654 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:46.654 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:27:46.654 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:46.654 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:46.654 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.242:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:27:46.654 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:50.653 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:27:50.655 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:27:50.655 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:27:50.655 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:27:50.656 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:27:50.656 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:27:50.656 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:27:50.656 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:27:50.657 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:50.657 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:27:50.657 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:50.657 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:50.658 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:27:50.658 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:54.656 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:27:54.658 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:27:54.658 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:27:54.658 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:27:54.658 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:27:54.658 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:27:54.659 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:27:54.659 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:27:54.659 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:54.659 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:27:54.659 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:54.659 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:54.659 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:27:54.660 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:54.979 7f87eae3fdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 05:27:55.268 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:55.268 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. database_usage = [ << disk_space_used_1k = 6649768 disk_space_available_1k = 74440716 config_db_size_1k = 1792 >>, ] >> 03/26/2026 05:27:55.269 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:57.171 7f87eae3fdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 05:27:58.659 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:27:58.660 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:27:58.660 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:27:58.660 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:27:58.661 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:27:58.661 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:27:58.661 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:27:58.661 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:27:58.661 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:58.661 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:27:58.662 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:58.662 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:27:58.662 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.242:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:27:58.662 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:02.203 7f87eae3fdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6650036, partition_space_available_1k=74440448, percentage_partition_space_used=8, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1704236, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=75244, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51948, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=3314696, free=7013764, buffers=238908, cached=5957444, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.12, five_min_avg=1.79, fifteen_min_avg=0.76, cpu_share=7.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 05:28:02.204 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:02.204 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 6650036 partition_space_available_1k = 74440448 percentage_partition_space_used = 8 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1704236 cpu_share = 0 mem_res = 0 >>zookeeper : << mem_virt = 75244 cpu_share = 0 mem_res = 0 >>contrail-config-database-nodemgr : << mem_virt = 51948 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175640 used = 3314696 free = 7013764 buffers = 238908 cached = 5957444 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 3.12 five_min_avg = 1.79 fifteen_min_avg = 0.76 cpu_share = 7.75 node_type = config-database-node >> >> 03/26/2026 05:28:02.204 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:02.661 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:28:02.663 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:28:02.663 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:28:02.663 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:28:02.663 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:28:02.663 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:28:02.664 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:28:02.665 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:28:02.665 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:02.665 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:28:02.665 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:02.665 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:02.665 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:28:02.665 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:06.664 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:28:06.666 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:28:06.666 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:28:06.667 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:28:06.667 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:28:06.667 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:28:06.667 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:28:06.668 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:28:06.668 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:06.668 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:28:06.669 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:06.669 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:06.669 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:28:06.669 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:10.668 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:28:10.670 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:28:10.670 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:28:10.670 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:28:10.671 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:28:10.671 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:28:10.671 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:28:10.671 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:28:10.672 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:10.672 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:28:10.672 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:10.672 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:10.672 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.242:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:28:10.672 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:14.672 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:28:14.674 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:28:14.677 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:28:14.677 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:28:14.678 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:28:14.678 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:28:14.678 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:28:14.679 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:28:14.679 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:14.679 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:28:14.679 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:14.680 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:14.680 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:28:14.680 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:18.676 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:28:18.678 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:28:18.678 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:28:18.678 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:28:18.678 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:28:18.678 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:28:18.678 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:28:18.679 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:28:18.679 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:18.679 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:28:18.679 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:18.679 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:18.680 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:28:18.680 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:22.680 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:28:22.682 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:28:22.682 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:28:22.682 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:28:22.682 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:28:22.682 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:28:22.683 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:28:22.683 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:28:22.684 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:22.684 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:28:22.684 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:22.685 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:22.685 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.242:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:28:22.685 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:26.683 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:28:26.685 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:28:26.685 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:28:26.685 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:28:26.686 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:28:26.686 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:28:26.686 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:28:26.686 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:28:26.687 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:26.687 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:28:26.687 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:26.687 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:26.687 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:28:26.687 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:30.688 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:28:30.690 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:28:30.690 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:28:30.690 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:28:30.691 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:28:30.691 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:28:30.691 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:28:30.691 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:28:30.692 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:30.692 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:28:30.692 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:30.692 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:30.692 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:28:30.692 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:34.692 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:28:34.694 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:28:34.695 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:28:34.695 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:28:34.695 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:28:34.696 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:28:34.696 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:28:34.697 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:28:34.698 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:34.698 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:28:34.699 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:34.699 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:34.699 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.242:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:28:34.700 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:38.696 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:28:38.698 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:28:38.698 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:28:38.699 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:28:38.699 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:28:38.699 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:28:38.699 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:28:38.700 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:28:38.700 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:38.700 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:28:38.700 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:38.700 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:38.700 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:28:38.700 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:42.699 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:28:42.701 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:28:42.701 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:28:42.702 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:28:42.702 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:28:42.703 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:28:42.703 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:28:42.704 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:28:42.704 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:42.705 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:28:42.705 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:42.705 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:42.706 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:28:42.706 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:46.704 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:28:46.705 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:28:46.706 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:28:46.706 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:28:46.706 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:28:46.706 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:28:46.707 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:28:46.707 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:28:46.708 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:46.708 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:28:46.708 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:46.708 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:46.708 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774502694471130 collector_name = collector_ip = 10.0.0.242:8086 collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 03/26/2026 05:28:46.709 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:28:50.708 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:28:50.710 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected 03/26/2026 05:28:50.710 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:28:50.710 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:28:50.711 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:28:50.711 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:28:50.711 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 03/26/2026 05:28:50.711 7f87e5486248 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 03/26/2026 05:28:50.711 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 03/26/2026 05:28:50.711 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/26/2026 05:28:50.712 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/26/2026 05:28:55.105 7f87eae3fdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 05:28:55.460 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/26/2026 05:28:57.593 7f87eae3fdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 05:29:02.623 7f87eae3fdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6906532, partition_space_available_1k=74183952, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1706180, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=76584, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52228, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=4854312, free=4996412, buffers=256792, cached=6417784, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.56, five_min_avg=1.88, fifteen_min_avg=0.85, cpu_share=9.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 05:29:02.625 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/26/2026 05:29:03.172 7f87e5394048 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001016 03/26/2026 05:29:03.173 7f87e5394048 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 03/26/2026 05:29:03.174 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 03/26/2026 05:29:03.175 7f87e5486248 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 03/26/2026 05:29:03.175 7f87e5486248 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 03/26/2026 05:29:03.178 7f87e5486248 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 03/26/2026 05:29:03.180 7f87e5486248 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/26/2026 05:29:03.181 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 03/26/2026 05:29:03.183 7f87e56f1e48 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 03/26/2026 05:29:03.186 7f87e56f1e48 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 03/26/2026 05:29:13.839 7f87e5394048 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001439 03/26/2026 05:29:13.840 7f87e5394048 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 03/26/2026 05:29:13.841 7f87e56f1e48 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 03/26/2026 05:29:13.841 7f87e56f1e48 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 3 03/26/2026 05:29:13.841 7f87e56f1e48 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 03/26/2026 05:29:13.841 7f87e56f1e48 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 03/26/2026 05:29:13.841 7f87e56f1e48 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/26/2026 05:29:13.842 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 03/26/2026 05:29:13.842 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 03/26/2026 05:29:55.027 7f87eae3fdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 05:29:57.759 7f87eae3fdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 05:30:02.805 7f87eae3fdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7097472, partition_space_available_1k=73993012, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1705356, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=159568, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52400, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=5819100, free=3975532, buffers=269128, cached=6462288, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.44, five_min_avg=2.26, fifteen_min_avg=1.05, cpu_share=13.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 05:30:55.034 7f87eae3fdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 05:30:57.536 7f87eae3fdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 05:31:02.572 7f87eae3fdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7150240, partition_space_available_1k=73940244, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1707612, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=142340, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52480, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=5788632, free=3949436, buffers=269984, cached=6517976, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.53, five_min_avg=2.23, fifteen_min_avg=1.12, cpu_share=12.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 05:31:55.022 7f87eae3fdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 05:31:57.593 7f87eae3fdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 05:32:02.624 7f87eae3fdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7176224, partition_space_available_1k=73914260, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1710868, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=125148, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52504, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=5823748, free=3886608, buffers=270844, cached=6544836, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.51, five_min_avg=2.28, fifteen_min_avg=1.2, cpu_share=12.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 05:32:55.180 7f87eae3fdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 05:32:58.571 7f87eae3fdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 05:33:03.614 7f87eae3fdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7422780, partition_space_available_1k=73667704, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1723072, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=126440, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52528, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=5843500, free=3371940, buffers=275004, cached=7035664, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.85, five_min_avg=2.41, fifteen_min_avg=1.32, cpu_share=12.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 05:33:09.265 7f87e5486248 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 03/26/2026 05:33:55.175 7f87eae3fdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 05:33:57.302 7f87e56f1e48 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 03/26/2026 05:33:58.339 7f87eae3fdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 05:34:03.390 7f87eae3fdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7650224, partition_space_available_1k=73440260, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1741216, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=133172, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52584, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=5891388, free=3327424, buffers=278040, cached=7029180, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.17, five_min_avg=2.6, fifteen_min_avg=1.46, cpu_share=13.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 05:34:55.054 7f87eae3fdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 05:34:57.647 7f87eae3fdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 05:35:02.691 7f87eae3fdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7642156, partition_space_available_1k=73448328, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1777348, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=139792, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52616, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=5939840, free=3276516, buffers=279152, cached=7030516, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.56, five_min_avg=2.78, fifteen_min_avg=1.59, cpu_share=12.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 05:35:55.052 7f87eae3fdf0 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 05:35:57.973 7f87eae3fdf0 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 05:36:03.004 7f87eae3fdf0 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7657068, partition_space_available_1k=73433416, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1814452, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=145976, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52620, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6006760, free=3192996, buffers=280056, cached=7046228, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.23, five_min_avg=3.07, fifteen_min_avg=1.76, cpu_share=12.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 05:36:22.239 7f87e5394048 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 03/26/2026 05:36:22.240 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Established] 03/26/2026 05:36:22.244 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected 03/26/2026 05:36:22.244 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[Established] -> State[Connect] 03/26/2026 05:36:22.244 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:36:22.245 7f87e5486248 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Established to Connect on EvTcpClose >>, ] description = Collector:Collector[Established to Connect on EvTcpClose] connection down >>, ] >> 03/26/2026 05:36:22.245 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:36:22.245 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 03/26/2026 05:36:22.245 7f87e5486248 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 03/26/2026 05:36:22.246 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 03/26/2026 05:36:22.246 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/26/2026 05:36:22.246 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/26/2026 05:36:22.503 7f87e5394048 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001017 03/26/2026 05:36:22.504 7f87e5394048 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 03/26/2026 05:36:22.505 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 03/26/2026 05:36:22.505 7f87e5486248 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 03/26/2026 05:36:22.505 7f87e5486248 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 03/26/2026 05:36:22.505 7f87e5486248 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 03/26/2026 05:36:22.506 7f87e5486248 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/26/2026 05:36:22.506 7f87e5486248 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 03/26/2026 05:36:22.507 7f87e56f1e48 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 03/26/2026 05:36:22.509 7f87e56f1e48 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 03/26/2026 05:36:46.020 7f0600384f20 [contrail-config-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 03/26/2026 05:36:46.020 7f0600384f20 [contrail-config-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 03/26/2026 05:36:46.020 7f0600384f20 [contrail-config-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 03/26/2026 05:36:46.048 7f05fa5a1748 [contrail-config-database-nodemgr] [ERROR]: Cannot write http_port 8112 to /tmp/contrail-config-database-nodemgr.0.http_port 03/26/2026 05:36:46.050 7f05fa5a1748 [contrail-config-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8112 03/26/2026 05:36:46.073 7f05f93bd048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 03/26/2026 05:36:46.076 7f05f93bd048 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 03/26/2026 05:36:46.077 7f05f93bd048 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/26/2026 05:36:46.077 7f05f93bd048 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 03/26/2026 05:36:46.077 7f05f93bd048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:36:46.077 7f05f93bd048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/26/2026 05:36:46.077 7f05f93bd048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:36:46.078 7f05f93bd048 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:36:46.094 7f05f93bd048 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774503406077067 collector_name = collector_ip = collector_list = [ 10.0.0.33:8086, 10.0.0.38:8086, 10.0.0.242:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 03/26/2026 05:36:46.094 7f05f93bd048 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:36:46.870 7f0600384f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 84755038961334919458078586997610798608053589957181516736888456705188594616068, 'start': '1774503395000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'zookeeper', 'group': 'zookeeper', 'pid': 35123161966967307064494183079845637437870586061261374677531464525313043303203, 'start': '1774503395000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-config-database-nodemgr', 'group': 'contrail-config-database-nodemgr', 'pid': 56093402697086538778499493165740912784918614745172713217925070760473462031352, 'start': '1774503396000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 03/26/2026 05:36:47.640 7f05f93bd448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:36:47.640 7f05f93bd448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. build_info = {"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-10-06 01:00:31.164147", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11344-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 4 num_core_per_socket = 4 num_thread_per_core = 1 >> running_package_version = v0+nightly.r24.1.centos-nightly_r24.1_centos.el7 installed_package_version = v0+nightly.r24.1.centos-nightly_r24.1_centos.el7 >> 03/26/2026 05:36:47.640 7f05f93bd448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:36:47.661 7f0600384f20 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5546-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1774503407661385, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 03/26/2026 05:36:47.662 7f0600384f20 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5546-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1774503407662011, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774503395000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774503395000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774503396000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-10-06 01:00:31.164147", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11344-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 03/26/2026 05:36:47.666 7f05f93bd448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:36:47.666 7f05f93bd448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/26/2026 05:36:47.666 7f05f93bd448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:36:47.666 7f05f93bd448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:36:47.666 7f05f93bd448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774503395000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = zookeeper process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774503395000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774503396000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-10-06 01:00:31.164147", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11344-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} >> 03/26/2026 05:36:47.666 7f05f93bd448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:36:50.310 7f0600384f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 05:36:50.323 7f05f93bd448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:36:50.328 7f05f93bd448 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected 03/26/2026 05:36:50.328 7f05f93bd448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:36:50.328 7f05f93bd448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:36:50.329 7f05f93bd448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:36:50.329 7f05f93bd448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:36:50.329 7f05f93bd448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 03/26/2026 05:36:50.329 7f05f93bd448 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 03/26/2026 05:36:50.329 7f05f93bd448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 03/26/2026 05:36:50.330 7f05f93bd448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/26/2026 05:36:50.330 7f05f93bd448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/26/2026 05:36:52.550 7f0600384f20 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 03/26/2026 05:36:52.553 7f05f93bd448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/26/2026 05:36:54.118 7f05f9378448 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001016 03/26/2026 05:36:54.119 7f05f9378448 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 03/26/2026 05:36:54.119 7f05f93bd448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 03/26/2026 05:36:54.120 7f05f93bd448 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 03/26/2026 05:36:54.120 7f05f93bd448 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 03/26/2026 05:36:54.120 7f05f93bd448 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 03/26/2026 05:36:54.120 7f05f93bd448 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/26/2026 05:36:54.120 7f05f93bd448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 03/26/2026 05:36:54.121 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 03/26/2026 05:36:54.122 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 03/26/2026 05:36:55.304 7f0600384f20 [contrail-config-database-nodemgr] [ERROR]: Failed to connect to database by CQL: Result: 1 03/26/2026 05:36:55.305 7f0600384f20 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5546-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1774503415305016, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.33:8086'], status='Up', description='ClientInit to Established on EvSandeshCtrlMessageRecv')]), flag_infos=None, description='Cassandra state detected DOWN. ')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 03/26/2026 05:37:03.485 7f0600384f20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 05:37:16.442 7f0600384f20 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr config_db_uuid'. Error is: timed out 03/26/2026 05:37:22.994 7f0600384f20 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr to_bgp_keyspace'. Error is: Result: 1 03/26/2026 05:37:31.660 7f0600384f20 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr svc_monitor_keyspace'. Error is: Result: 1 03/26/2026 05:37:37.402 7f0600384f20 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr DISCOVERY_SERVER'. Error is: Result: 1 03/26/2026 05:37:40.931 7f0600384f20 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr useragent'. Error is: Result: 1 03/26/2026 05:37:44.029 7f0600384f20 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr dm_keyspace'. Error is: Result: 1 03/26/2026 05:37:49.492 7f0600384f20 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5546-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1774503469491893, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774503395000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774503395000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774503396000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-10-06 01:00:31.164147", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11344-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 03/26/2026 05:37:49.493 7f0600384f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8583576, partition_space_available_1k=72506908, percentage_partition_space_used=11, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1325988, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=251720, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44360, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=5605372, free=2479320, buffers=308764, cached=8132572, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=17.32, five_min_avg=7.97, fifteen_min_avg=3.65, cpu_share=0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 05:37:49.493 7f0600384f20 [contrail-config-database-nodemgr] [ERROR]: function do_periodic_events duration exceeded 60.000000 interval (took 61.831140) 03/26/2026 05:37:50.260 7f0600384f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 05:37:54.231 7f0600384f20 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5546-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1774503474231133, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.33:8086'], status='Up', description='ClientInit to Established on EvSandeshCtrlMessageRecv')]), flag_infos=None, description='')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 03/26/2026 05:37:58.272 7f0600384f20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 05:38:03.335 7f0600384f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8930060, partition_space_available_1k=72160424, percentage_partition_space_used=11, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1338716, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=261140, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44652, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=5615764, free=2076768, buffers=325304, cached=8508368, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=16.32, five_min_avg=8.23, fifteen_min_avg=3.81, cpu_share=22.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 05:38:49.831 7f0600384f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 05:38:53.124 7f0600384f20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 05:38:58.538 7f0600384f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=9921024, partition_space_available_1k=71169460, percentage_partition_space_used=12, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1729952, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=347500, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44664, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6083916, free=590852, buffers=368492, cached=9482780, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=8.42, five_min_avg=7.35, fifteen_min_avg=3.76, cpu_share=16.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 05:39:43.503 7f05f9378448 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 03/26/2026 05:39:43.504 7f05f93bd448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Established] 03/26/2026 05:39:43.506 7f05f93bd448 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected 03/26/2026 05:39:43.506 7f05f93bd448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[Established] -> State[Connect] 03/26/2026 05:39:43.507 7f05f93bd448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:39:43.507 7f05f93bd448 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Established to Connect on EvTcpClose >>, ] description = Collector:Collector[Established to Connect on EvTcpClose] connection down >>, ] >> 03/26/2026 05:39:43.507 7f05f93bd448 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:39:43.507 7f05f93bd448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 03/26/2026 05:39:43.508 7f05f93bd448 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 03/26/2026 05:39:43.509 7f05f93bd448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 03/26/2026 05:39:43.509 7f05f93bd448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/26/2026 05:39:43.510 7f05f93bd448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/26/2026 05:39:43.547 7f05f9378448 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001017 03/26/2026 05:39:43.548 7f05f9378448 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 03/26/2026 05:39:43.548 7f05f93bd448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 03/26/2026 05:39:43.549 7f05f93bd448 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 03/26/2026 05:39:43.549 7f05f93bd448 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 03/26/2026 05:39:43.549 7f05f93bd448 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 03/26/2026 05:39:43.549 7f05f93bd448 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/26/2026 05:39:43.550 7f05f93bd448 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 03/26/2026 05:39:43.550 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 03/26/2026 05:39:43.552 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 03/26/2026 05:39:43.641 7f05f9378448 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 03/26/2026 05:39:43.641 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Established] 03/26/2026 05:39:43.643 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected 03/26/2026 05:39:43.644 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[Established] -> State[Connect] 03/26/2026 05:39:43.644 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:39:43.645 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Initializing description = Established to Connect on EvTcpClose >>, ] description = Collector:Collector[Established to Connect on EvTcpClose] connection down >>, ] >> 03/26/2026 05:39:43.645 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:39:43.645 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 03/26/2026 05:39:43.646 7f05f93bd148 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 03/26/2026 05:39:43.646 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 03/26/2026 05:39:43.646 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/26/2026 05:39:43.646 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/26/2026 05:39:43.686 7f05f9378448 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 03/26/2026 05:39:43.687 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit] 03/26/2026 05:39:43.687 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle] 03/26/2026 05:39:43.687 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:39:43.687 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Down description = ClientInit to Idle on EvTcpClose >>, ] description = Collector:Collector[ClientInit to Idle on EvTcpClose] connection down >>, ] >> 03/26/2026 05:39:43.687 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:39:43.688 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:39:43.688 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 3 pid = 1 http_port = 8112 start_time = 1774503406077067 collector_name = collector_ip = 10.0.0.242:8086 collector_list = [ 10.0.0.33:8086, 10.0.0.38:8086, 10.0.0.242:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/26/2026 05:39:43.688 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:39:47.688 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:39:47.690 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected 03/26/2026 05:39:47.690 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:39:47.690 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:39:47.690 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:39:47.691 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:39:47.691 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 03/26/2026 05:39:47.691 7f05f93bd148 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 03/26/2026 05:39:47.691 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 03/26/2026 05:39:47.691 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/26/2026 05:39:47.692 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/26/2026 05:39:49.785 7f0600384f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 05:39:50.103 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/26/2026 05:39:51.898 7f0600384f20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 05:39:55.400 7f05f9378448 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 03/26/2026 05:39:55.401 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit] 03/26/2026 05:39:55.402 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle] 03/26/2026 05:39:55.402 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:39:55.402 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = ClientInit to Idle on EvTcpClose >>, ] description = Collector:Collector[ClientInit to Idle on EvTcpClose] connection down >>, ] >> 03/26/2026 05:39:55.402 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:39:55.403 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:39:55.403 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774503406077067 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, 10.0.0.38:8086, 10.0.0.242:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/26/2026 05:39:55.403 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:39:57.454 7f0600384f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=10587128, partition_space_available_1k=70503356, percentage_partition_space_used=13, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1764184, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=369444, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44684, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=5389868, free=964808, buffers=375816, cached=9794472, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.44, five_min_avg=6.66, fifteen_min_avg=3.74, cpu_share=11.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 05:39:57.455 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:39:57.455 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 10587128 partition_space_available_1k = 70503356 percentage_partition_space_used = 13 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1764184 cpu_share = 0 mem_res = 0 >>zookeeper : << mem_virt = 369444 cpu_share = 0 mem_res = 0 >>contrail-config-database-nodemgr : << mem_virt = 44684 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175640 used = 5389868 free = 964808 buffers = 375816 cached = 9794472 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 5.44 five_min_avg = 6.66 fifteen_min_avg = 3.74 cpu_share = 11.75 node_type = config-database-node >> >> 03/26/2026 05:39:57.455 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:39:59.403 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:39:59.406 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:39:59.406 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:39:59.406 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:39:59.406 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:39:59.406 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:39:59.407 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:39:59.407 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:39:59.407 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:39:59.407 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:39:59.407 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:39:59.408 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:39:59.408 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774503406077067 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.33:8086, 10.0.0.38:8086, 10.0.0.242:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/26/2026 05:39:59.408 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:03.408 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:40:03.410 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:40:03.411 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:40:03.411 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:40:03.411 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:40:03.411 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:40:03.411 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:40:03.412 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:40:03.412 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:03.412 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:40:03.412 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:03.413 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:03.413 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774503406077067 collector_name = collector_ip = 10.0.0.242:8086 collector_list = [ 10.0.0.33:8086, 10.0.0.38:8086, 10.0.0.242:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/26/2026 05:40:03.413 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:07.412 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:40:07.416 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:40:07.416 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:40:07.416 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:40:07.417 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:40:07.417 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:40:07.417 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:40:07.417 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:40:07.417 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:07.418 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:40:07.418 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:07.418 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:07.418 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774503406077067 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, 10.0.0.38:8086, 10.0.0.242:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/26/2026 05:40:07.418 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:11.418 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:40:11.420 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:40:11.420 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:40:11.420 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:40:11.421 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:40:11.421 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:40:11.421 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:40:11.422 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:40:11.422 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:11.422 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:40:11.422 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:11.423 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:11.423 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774503406077067 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.33:8086, 10.0.0.38:8086, 10.0.0.242:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/26/2026 05:40:11.423 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:15.422 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:40:15.423 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:40:15.423 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:40:15.423 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:40:15.423 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:40:15.423 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:40:15.424 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:40:15.424 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:40:15.424 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:15.424 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:40:15.424 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:15.424 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:15.425 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774503406077067 collector_name = collector_ip = 10.0.0.242:8086 collector_list = [ 10.0.0.33:8086, 10.0.0.38:8086, 10.0.0.242:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/26/2026 05:40:15.425 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:19.424 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:40:19.427 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:40:19.428 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:40:19.428 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:40:19.429 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:40:19.429 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:40:19.429 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:40:19.431 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:40:19.431 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:19.431 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:40:19.431 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:19.431 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:19.431 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774503406077067 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, 10.0.0.38:8086, 10.0.0.242:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/26/2026 05:40:19.431 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:23.428 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:40:23.431 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:40:23.433 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:40:23.433 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:40:23.434 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:40:23.434 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:40:23.434 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:40:23.435 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:40:23.435 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:23.436 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:40:23.436 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:23.437 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:23.437 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774503406077067 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.33:8086, 10.0.0.38:8086, 10.0.0.242:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/26/2026 05:40:23.437 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:27.434 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:40:27.436 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:40:27.436 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:40:27.436 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:40:27.436 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:40:27.436 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:40:27.437 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:40:27.437 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:40:27.437 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:27.437 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:40:27.437 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:27.437 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:27.438 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774503406077067 collector_name = collector_ip = 10.0.0.242:8086 collector_list = [ 10.0.0.33:8086, 10.0.0.38:8086, 10.0.0.242:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/26/2026 05:40:27.438 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:31.438 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:40:31.440 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:40:31.441 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:40:31.441 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:40:31.441 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:40:31.441 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:40:31.441 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:40:31.442 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:40:31.442 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:31.442 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:40:31.442 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:31.443 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:31.443 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774503406077067 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, 10.0.0.38:8086, 10.0.0.242:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/26/2026 05:40:31.443 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:35.441 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:40:35.443 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:40:35.443 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:40:35.443 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:40:35.443 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:40:35.445 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:40:35.445 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:40:35.445 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:40:35.446 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:35.446 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:40:35.446 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:35.446 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:35.446 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774503406077067 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.33:8086, 10.0.0.38:8086, 10.0.0.242:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/26/2026 05:40:35.446 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:39.444 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:40:39.447 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:40:39.447 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:40:39.447 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:40:39.447 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:40:39.447 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:40:39.448 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:40:39.448 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:40:39.449 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:39.449 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:40:39.449 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:39.449 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:39.450 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774503406077067 collector_name = collector_ip = 10.0.0.242:8086 collector_list = [ 10.0.0.33:8086, 10.0.0.38:8086, 10.0.0.242:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/26/2026 05:40:39.450 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:43.448 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:40:43.450 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:40:43.450 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:40:43.451 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:40:43.451 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:40:43.451 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:40:43.451 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:40:43.452 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:40:43.452 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:43.453 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:40:43.453 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:43.453 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:43.453 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774503406077067 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, 10.0.0.38:8086, 10.0.0.242:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/26/2026 05:40:43.453 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:47.452 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:40:47.454 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:40:47.454 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:40:47.454 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:40:47.454 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:40:47.454 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:40:47.454 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:40:47.455 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:40:47.455 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:47.455 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:40:47.455 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:47.455 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:47.455 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774503406077067 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.33:8086, 10.0.0.38:8086, 10.0.0.242:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/26/2026 05:40:47.455 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:49.850 7f0600384f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 05:40:50.156 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:50.156 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. database_usage = [ << disk_space_used_1k = 10496480 disk_space_available_1k = 70594004 config_db_size_1k = 2992 >>, ] >> 03/26/2026 05:40:50.156 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:51.454 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:40:51.455 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:40:51.455 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:40:51.455 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:40:51.455 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:40:51.455 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:40:51.456 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:40:51.456 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:40:51.456 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:51.456 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:40:51.456 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:51.456 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:51.456 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774503406077067 collector_name = collector_ip = 10.0.0.242:8086 collector_list = [ 10.0.0.33:8086, 10.0.0.38:8086, 10.0.0.242:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/26/2026 05:40:51.457 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:54.459 7f0600384f20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 05:40:55.456 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:40:55.458 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:40:55.458 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:40:55.459 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:40:55.459 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:40:55.460 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:40:55.460 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:40:55.461 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:40:55.462 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:55.462 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:40:55.463 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:55.463 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:55.464 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774503406077067 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, 10.0.0.38:8086, 10.0.0.242:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/26/2026 05:40:55.465 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:59.458 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:40:59.460 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:40:59.460 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:40:59.460 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:40:59.460 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:40:59.461 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:40:59.461 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:40:59.461 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:40:59.461 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:59.461 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:40:59.462 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:59.462 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:40:59.462 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774503406077067 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.33:8086, 10.0.0.38:8086, 10.0.0.242:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/26/2026 05:40:59.462 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:41:00.186 7f0600384f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=10499144, partition_space_available_1k=70591340, percentage_partition_space_used=13, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1808536, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=370212, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44780, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=5377792, free=1027592, buffers=379852, cached=9739660, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.91, five_min_avg=5.89, fifteen_min_avg=3.66, cpu_share=7.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 05:41:00.186 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:41:00.186 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 10499144 partition_space_available_1k = 70591340 percentage_partition_space_used = 13 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1808536 cpu_share = 0 mem_res = 0 >>zookeeper : << mem_virt = 370212 cpu_share = 0 mem_res = 0 >>contrail-config-database-nodemgr : << mem_virt = 44780 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175640 used = 5377792 free = 1027592 buffers = 379852 cached = 9739660 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 3.91 five_min_avg = 5.89 fifteen_min_avg = 3.66 cpu_share = 7.25 node_type = config-database-node >> >> 03/26/2026 05:41:00.186 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:41:03.460 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:41:03.462 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:41:03.462 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:41:03.462 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:41:03.462 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:41:03.462 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:41:03.462 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:41:03.462 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:41:03.462 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:41:03.463 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:41:03.463 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:41:03.463 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:41:03.463 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774503406077067 collector_name = collector_ip = 10.0.0.242:8086 collector_list = [ 10.0.0.33:8086, 10.0.0.38:8086, 10.0.0.242:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/26/2026 05:41:03.463 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:41:07.464 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:41:07.466 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:41:07.467 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:41:07.467 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:41:07.467 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:41:07.467 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:41:07.468 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:41:07.468 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:41:07.468 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:41:07.469 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:41:07.469 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:41:07.469 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:41:07.469 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774503406077067 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, 10.0.0.38:8086, 10.0.0.242:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/26/2026 05:41:07.469 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:41:11.469 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:41:11.471 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:41:11.471 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:41:11.472 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:41:11.472 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:41:11.472 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:41:11.472 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:41:11.473 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:41:11.473 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:41:11.473 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:41:11.473 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:41:11.473 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:41:11.474 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774503406077067 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.33:8086, 10.0.0.38:8086, 10.0.0.242:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/26/2026 05:41:11.475 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:41:15.472 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:41:15.474 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:41:15.474 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:41:15.475 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:41:15.475 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:41:15.475 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:41:15.475 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:41:15.476 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:41:15.476 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:41:15.476 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:41:15.476 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:41:15.476 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:41:15.477 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774503406077067 collector_name = collector_ip = 10.0.0.242:8086 collector_list = [ 10.0.0.33:8086, 10.0.0.38:8086, 10.0.0.242:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/26/2026 05:41:15.477 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:41:19.476 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:41:19.478 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:41:19.479 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:41:19.479 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:41:19.479 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:41:19.480 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:41:19.480 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:41:19.481 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:41:19.481 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:41:19.481 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:41:19.481 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:41:19.482 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:41:19.482 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774503406077067 collector_name = collector_ip = 10.0.0.33:8086 collector_list = [ 10.0.0.33:8086, 10.0.0.38:8086, 10.0.0.242:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/26/2026 05:41:19.482 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:41:23.480 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:41:23.482 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:41:23.483 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:41:23.484 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:41:23.485 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:41:23.485 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:41:23.485 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:41:23.486 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:41:23.486 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:41:23.486 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:41:23.486 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:41:23.486 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:41:23.486 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774503406077067 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.33:8086, 10.0.0.38:8086, 10.0.0.242:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/26/2026 05:41:23.487 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:41:27.485 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:41:27.487 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/26/2026 05:41:27.488 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:41:27.488 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:41:27.488 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:41:27.488 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:41:27.488 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/26/2026 05:41:27.489 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/26/2026 05:41:27.489 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:41:27.489 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/26/2026 05:41:27.489 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:41:27.489 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:41:27.489 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8112 start_time = 1774503406077067 collector_name = collector_ip = 10.0.0.242:8086 collector_list = [ 10.0.0.33:8086, 10.0.0.38:8086, 10.0.0.242:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 5 >> 03/26/2026 05:41:27.489 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:41:31.488 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:41:31.490 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected 03/26/2026 05:41:31.490 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:41:31.491 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:41:31.491 7f05f93bd148 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.33:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:41:31.492 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:41:31.492 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 03/26/2026 05:41:31.492 7f05f93bd148 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 03/26/2026 05:41:31.499 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 03/26/2026 05:41:31.499 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/26/2026 05:41:31.500 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/26/2026 05:41:34.116 7f05f9378448 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001016 03/26/2026 05:41:34.116 7f05f9378448 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 03/26/2026 05:41:34.117 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 03/26/2026 05:41:34.117 7f05f93bd148 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 03/26/2026 05:41:34.118 7f05f93bd148 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 03/26/2026 05:41:34.118 7f05f93bd148 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 03/26/2026 05:41:34.118 7f05f93bd148 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/26/2026 05:41:34.118 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 03/26/2026 05:41:34.119 7f05f93bd448 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 03/26/2026 05:41:34.121 7f05f93bd448 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 03/26/2026 05:41:44.110 7f05f9378448 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001438 03/26/2026 05:41:44.117 7f05f9378448 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 03/26/2026 05:41:44.117 7f05f93bd448 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 03/26/2026 05:41:44.117 7f05f93bd448 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 3 03/26/2026 05:41:44.118 7f05f93bd448 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 03/26/2026 05:41:44.118 7f05f93bd448 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 03/26/2026 05:41:44.118 7f05f93bd448 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/26/2026 05:41:44.118 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 03/26/2026 05:41:44.119 7f05f93bd148 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 03/26/2026 05:41:49.808 7f0600384f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 05:41:52.666 7f0600384f20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 05:41:58.636 7f0600384f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=10717928, partition_space_available_1k=70372556, percentage_partition_space_used=13, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1835160, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=370800, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44960, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6912224, free=290084, buffers=373572, cached=8910588, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.8, five_min_avg=5.47, fifteen_min_avg=3.65, cpu_share=13.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 05:42:49.904 7f0600384f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 05:42:52.566 7f0600384f20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 05:42:58.628 7f0600384f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=10831004, partition_space_available_1k=70259480, percentage_partition_space_used=13, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1877496, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=371052, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45212, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6882352, free=239292, buffers=377112, cached=8958428, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.56, five_min_avg=4.79, fifteen_min_avg=3.53, cpu_share=10.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 05:43:49.793 7f0600384f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 05:43:52.123 7f0600384f20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 05:43:57.638 7f0600384f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=10881108, partition_space_available_1k=70209376, percentage_partition_space_used=13, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1882228, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=370852, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45224, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6943024, free=295172, buffers=378204, cached=8841900, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.79, five_min_avg=4.4, fifteen_min_avg=3.47, cpu_share=9.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 05:44:49.886 7f0600384f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 05:44:53.253 7f0600384f20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 05:44:58.638 7f0600384f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=10899624, partition_space_available_1k=70190860, percentage_partition_space_used=13, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2145856, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=370588, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45228, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=7206224, free=346188, buffers=378168, cached=8530148, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.53, five_min_avg=4.0, fifteen_min_avg=3.38, cpu_share=10.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 05:45:49.996 7f0600384f20 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 05:45:52.752 7f0600384f20 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 05:45:58.019 7f05f9378248 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_process_state:zookeeper,group:zookeeper,state:PROCESS_STATE_EXITED 03/26/2026 05:45:58.020 7f05f9378248 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5546-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1774503958020390, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774503395000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_EXITED', start_count=1, stop_count=0, exit_count=1, last_start_time='1774503395000000', last_stop_time='', last_exit_time='1774503958020178', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774503396000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly.r24.1.centos", "build-time" : "2025-10-06 01:00:31.164147", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-11344-1.novalocal", "build-id" : "v0+nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 03/26/2026 05:45:58.685 7f0600384f20 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=11988952, partition_space_available_1k=69101532, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=2156800, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=0, cpu_share=0, mem_res=0, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=45228, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6508424, free=1168848, buffers=400000, cached=8393556, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.63, five_min_avg=4.01, fifteen_min_avg=3.42, cpu_share=13.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 05:46:26.644 7f77fca83040 [contrail-config-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 03/26/2026 05:46:26.644 7f77fca83040 [contrail-config-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 03/26/2026 05:46:26.645 7f77fca83040 [contrail-config-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 03/26/2026 05:46:26.658 7f77f99e2ea0 [contrail-config-database-nodemgr] [ERROR]: Cannot write http_port 8112 to /tmp/contrail-config-database-nodemgr.0.http_port 03/26/2026 05:46:26.659 7f77f99e2ea0 [contrail-config-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8112 03/26/2026 05:46:26.660 7f77f9b4c2c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 03/26/2026 05:46:26.662 7f77f9b4c2c0 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 03/26/2026 05:46:26.663 7f77f9b4c2c0 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/26/2026 05:46:26.663 7f77f9b4c2c0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 03/26/2026 05:46:26.663 7f77f9b4c2c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:46:26.663 7f77f9b4c2c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/26/2026 05:46:26.663 7f77f9b4c2c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:46:26.663 7f77f9b4c2c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:46:26.664 7f77f9b4c2c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2.:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1774503986662820 collector_name = collector_ip = collector_list = [ 10.0.0.242:8086, 10.0.0.33:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 03/26/2026 05:46:26.664 7f77f9b4c2c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:46:26.930 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 25045432502886160287819887992320313846838153392339524319529268279330823083114, 'start': '1774503982000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'zookeeper', 'group': 'zookeeper', 'pid': 115445352008924830399722598037967581269054711805977928618221080014906945792472, 'start': '1774503982000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-config-database-nodemgr', 'group': 'contrail-config-database-nodemgr', 'pid': 94967735800218513237477372210251702443566904938235686296246333200994138394816, 'start': '1774503982000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 03/26/2026 05:46:27.001 7f77f9b4c9a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:46:27.002 7f77f9b4c9a0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. build_info = {"build-info": [{"build-version": "v0+nightly", "build-time": "2026-03-26 00:52:50.045192", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-14499-1.novalocal", "build-id": "v0+nightly", "build-number": "202603260008"}]} system_cpu_info = << num_socket = 1 num_cpu = 4 num_core_per_socket = 4 num_thread_per_core = 1 >> >> 03/26/2026 05:46:27.002 7f77f9b4c9a0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:46:27.004 7f77fca83040 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5546-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1774503987004830, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 03/26/2026 05:46:27.005 7f77fca83040 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5546-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1774503987005333, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774503982000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774503982000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774503982000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info": [{"build-version": "v0+nightly", "build-time": "2026-03-26 00:52:50.045192", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-14499-1.novalocal", "build-id": "v0+nightly", "build-number": "202603260008"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 03/26/2026 05:46:27.008 7f77f9b4c9a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:46:27.016 7f77f9b4c9a0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/26/2026 05:46:27.016 7f77f9b4c9a0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:46:27.016 7f77f9b4c9a0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:46:27.017 7f77f9b4c9a0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774503982000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = zookeeper process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774503982000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1774503982000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info": [{"build-version": "v0+nightly", "build-time": "2026-03-26 00:52:50.045192", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-14499-1.novalocal", "build-id": "v0+nightly", "build-number": "202603260008"}]} >> 03/26/2026 05:46:27.017 7f77f9b4c9a0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:46:27.564 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 05:46:28.115 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 03/26/2026 05:46:28.117 7f77f9b4cf40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:46:28.118 7f77f9b4cf40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. database_usage = [ << disk_space_used_1k = 12030124 disk_space_available_1k = 69060360 config_db_size_1k = 4172 >>, ] >> 03/26/2026 05:46:28.118 7f77f9b4cf40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:46:28.856 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to connect to database by CQL and socket: Result: 1 03/26/2026 05:46:28.857 7f77fca83040 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5546-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1774503988857133, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 03/26/2026 05:46:28.863 7f77f9b4cf40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:46:28.863 7f77f9b4cf40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/26/2026 05:46:28.863 7f77f9b4cf40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/26/2026 05:46:30.664 7f77f9b4cf40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/26/2026 05:46:30.665 7f77f9b4cf40 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected 03/26/2026 05:46:30.665 7f77f9b4cf40 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/26/2026 05:46:30.665 7f77f9b4cf40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:46:30.666 7f77f9b4cf40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-5546-2. process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.242:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/26/2026 05:46:30.666 7f77f9b4cf40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/26/2026 05:46:30.666 7f77f9b4cf40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 03/26/2026 05:46:30.666 7f77f9b4cf40 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 03/26/2026 05:46:30.667 7f77f9b4cf40 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 03/26/2026 05:46:30.670 7f77f9b4cf40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/26/2026 05:46:30.671 7f77f9b4cf40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/26/2026 05:46:30.679 7f77f99a0a40 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001017 03/26/2026 05:46:30.680 7f77f99a0a40 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 03/26/2026 05:46:30.680 7f77f99a0e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 03/26/2026 05:46:30.680 7f77f99a0e00 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 03/26/2026 05:46:30.681 7f77f99a0e00 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 03/26/2026 05:46:30.681 7f77f99a0e00 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 03/26/2026 05:46:30.681 7f77f99a0e00 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/26/2026 05:46:30.682 7f77f99a0e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 03/26/2026 05:46:30.682 7f77f99a07c0 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 03/26/2026 05:46:30.683 7f77f99a07c0 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 03/26/2026 05:46:32.657 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 05:46:35.212 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr config_db_uuid'. Error is: Result: 1 03/26/2026 05:46:37.096 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr to_bgp_keyspace'. Error is: Result: 1 03/26/2026 05:46:39.282 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr svc_monitor_keyspace'. Error is: Result: 1 03/26/2026 05:46:41.281 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr DISCOVERY_SERVER'. Error is: Result: 1 03/26/2026 05:46:42.941 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr useragent'. Error is: Result: 1 03/26/2026 05:46:44.221 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr dm_keyspace'. Error is: Result: 1 03/26/2026 05:46:49.678 7f77fca83040 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5546-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1774504009678473, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774503982000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774503982000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1774503982000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info": [{"build-version": "v0+nightly", "build-time": "2026-03-26 00:52:50.045192", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-14499-1.novalocal", "build-id": "v0+nightly", "build-number": "202603260008"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 03/26/2026 05:46:49.679 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=11994764, partition_space_available_1k=69095720, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1292116, cpu_share=0.0, mem_res=1273460, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=241996, cpu_share=0.0, mem_res=216140, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57656, cpu_share=0.0, mem_res=47396, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=4910280, free=2638808, buffers=405108, cached=8542748, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.84, five_min_avg=4.25, fifteen_min_avg=3.53, cpu_share=0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 05:47:27.348 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 05:47:28.271 7f77fca83040 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-5546-2.', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1774504048271601, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.242:8086'], status='Up', description='ClientInit to Established on EvSandeshCtrlMessageRecv')]), flag_infos=None, description='')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 03/26/2026 05:47:29.678 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 05:47:34.723 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12012152, partition_space_available_1k=69078332, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1344412, cpu_share=0.37, mem_res=1321792, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=251244, cpu_share=0.02, mem_res=224116, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57680, cpu_share=0.02, mem_res=47396, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6179796, free=1340400, buffers=406836, cached=8570320, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.13, five_min_avg=4.19, fifteen_min_avg=3.55, cpu_share=15.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 05:48:27.276 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 05:48:29.267 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 05:48:34.309 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12017828, partition_space_available_1k=69072656, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1339032, cpu_share=0.12, mem_res=1314212, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=263100, cpu_share=0.02, mem_res=235828, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57676, cpu_share=0.01, mem_res=47396, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6792032, free=723952, buffers=406840, cached=8574524, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.26, five_min_avg=3.68, fifteen_min_avg=3.42, cpu_share=10.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 05:49:27.266 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 05:49:29.234 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 05:49:34.282 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12024684, partition_space_available_1k=69065800, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1341608, cpu_share=0.07, mem_res=1316776, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=263448, cpu_share=0.01, mem_res=236172, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57688, cpu_share=0.01, mem_res=47396, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6830584, free=676764, buffers=406856, cached=8582128, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.35, five_min_avg=3.16, fifteen_min_avg=3.25, cpu_share=8.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 05:50:27.279 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 05:50:29.189 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 05:50:34.242 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12024836, partition_space_available_1k=69065648, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1344100, cpu_share=0.07, mem_res=1319368, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=263588, cpu_share=0.0, mem_res=236312, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57688, cpu_share=0.01, mem_res=47396, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6851728, free=655088, buffers=406856, cached=8582620, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.3, five_min_avg=2.82, fifteen_min_avg=3.13, cpu_share=8.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 05:51:27.374 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 05:51:29.192 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 05:51:34.274 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12030388, partition_space_available_1k=69060096, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1331760, cpu_share=0.06, mem_res=1307004, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=263700, cpu_share=0.01, mem_res=236416, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57712, cpu_share=0.02, mem_res=47396, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6865956, free=635408, buffers=406856, cached=8588016, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.4, five_min_avg=2.82, fifteen_min_avg=3.1, cpu_share=7.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 05:52:27.370 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 05:52:29.354 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 05:52:34.400 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12038696, partition_space_available_1k=69051788, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1353848, cpu_share=0.08, mem_res=1325792, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=263776, cpu_share=0.01, mem_res=236480, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57708, cpu_share=0.01, mem_res=47396, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6932392, free=560440, buffers=406868, cached=8596512, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.84, five_min_avg=2.56, fifteen_min_avg=3.0, cpu_share=7.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 05:53:27.313 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 05:53:29.403 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 05:53:34.452 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12039692, partition_space_available_1k=69050792, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1353288, cpu_share=0.06, mem_res=1325240, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=264016, cpu_share=0.01, mem_res=236716, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57712, cpu_share=0.01, mem_res=47396, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6897744, free=593916, buffers=406868, cached=8597724, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.42, five_min_avg=2.33, fifteen_min_avg=2.89, cpu_share=8.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 05:54:27.349 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 05:54:29.414 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 05:54:34.461 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12045012, partition_space_available_1k=69045472, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1356140, cpu_share=0.07, mem_res=1328088, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=264024, cpu_share=0.0, mem_res=236728, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57724, cpu_share=0.01, mem_res=47396, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6919508, free=566736, buffers=406868, cached=8603196, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.44, five_min_avg=2.15, fifteen_min_avg=2.79, cpu_share=7.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 05:55:27.343 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 05:55:29.253 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 05:55:34.315 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12045620, partition_space_available_1k=69044864, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1347612, cpu_share=0.06, mem_res=1319484, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=264104, cpu_share=0.01, mem_res=236804, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57724, cpu_share=0.01, mem_res=47396, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6946896, free=541796, buffers=406868, cached=8600716, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.13, five_min_avg=2.18, fifteen_min_avg=2.76, cpu_share=7.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 05:56:27.378 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 05:56:29.519 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 05:56:34.582 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12047424, partition_space_available_1k=69043060, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1358184, cpu_share=0.07, mem_res=1330124, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=264116, cpu_share=0.0, mem_res=236820, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57728, cpu_share=0.01, mem_res=47396, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6971320, free=512424, buffers=406868, cached=8605688, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.47, five_min_avg=1.98, fifteen_min_avg=2.65, cpu_share=7.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 05:57:27.404 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 05:57:29.504 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 05:57:34.550 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12054112, partition_space_available_1k=69036372, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1369952, cpu_share=0.07, mem_res=1340936, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=264116, cpu_share=0.0, mem_res=236820, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57736, cpu_share=0.01, mem_res=47396, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6951956, free=524876, buffers=406868, cached=8612592, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.29, five_min_avg=1.82, fifteen_min_avg=2.55, cpu_share=7.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 05:58:27.370 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 05:58:29.452 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 05:58:34.499 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12055232, partition_space_available_1k=69035252, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1354400, cpu_share=0.06, mem_res=1325380, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=264124, cpu_share=0.0, mem_res=236828, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57740, cpu_share=0.01, mem_res=47396, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6952016, free=521860, buffers=406868, cached=8615584, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.3, five_min_avg=1.71, fifteen_min_avg=2.47, cpu_share=7.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 05:59:27.377 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 05:59:29.377 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 05:59:34.415 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12060576, partition_space_available_1k=69029908, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1352148, cpu_share=0.06, mem_res=1323176, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=264136, cpu_share=0.0, mem_res=236832, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57748, cpu_share=0.01, mem_res=47396, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6968636, free=499744, buffers=406868, cached=8621024, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.4, five_min_avg=1.66, fifteen_min_avg=2.4, cpu_share=7.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 06:00:27.401 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 06:00:29.433 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 06:00:34.489 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12066000, partition_space_available_1k=69024484, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1355324, cpu_share=0.07, mem_res=1326348, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=276872, cpu_share=0.01, mem_res=249572, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57740, cpu_share=0.01, mem_res=47396, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6991708, free=470892, buffers=406872, cached=8626820, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.93, five_min_avg=1.78, fifteen_min_avg=2.39, cpu_share=7.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 06:01:27.410 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 06:01:29.502 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 06:01:34.561 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12066596, partition_space_available_1k=69023888, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1344696, cpu_share=0.07, mem_res=1315696, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=262888, cpu_share=0.0, mem_res=235588, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57748, cpu_share=0.01, mem_res=47396, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6941940, free=517020, buffers=406956, cached=8630312, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.23, five_min_avg=1.89, fifteen_min_avg=2.39, cpu_share=8.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 06:02:27.421 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 06:02:29.514 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 06:02:34.561 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12073044, partition_space_available_1k=69017440, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1358984, cpu_share=0.07, mem_res=1329008, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=262892, cpu_share=0.0, mem_res=235588, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57748, cpu_share=0.01, mem_res=47396, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=6986832, free=464592, buffers=406956, cached=8636880, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.09, five_min_avg=1.89, fifteen_min_avg=2.36, cpu_share=7.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 06:03:27.399 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 06:03:29.210 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 06:03:34.261 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12073624, partition_space_available_1k=69016860, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1355408, cpu_share=0.06, mem_res=1325460, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=262900, cpu_share=0.0, mem_res=235596, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57752, cpu_share=0.01, mem_res=47396, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=7013224, free=437628, buffers=406956, cached=8637476, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.73, five_min_avg=1.82, fifteen_min_avg=2.3, cpu_share=7.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 06:04:27.371 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 06:04:29.303 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 06:04:34.350 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12078676, partition_space_available_1k=69011808, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1350092, cpu_share=0.07, mem_res=1320048, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=262904, cpu_share=0.0, mem_res=235596, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57760, cpu_share=0.01, mem_res=47396, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=7039144, free=382792, buffers=406956, cached=8666520, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.8, five_min_avg=1.87, fifteen_min_avg=2.29, cpu_share=7.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 06:04:39.772 7f77f9b4cf40 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 03/26/2026 06:05:05.360 7f77f99a0040 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 03/26/2026 06:05:27.374 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 06:05:29.305 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 06:05:34.348 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12222536, partition_space_available_1k=68867948, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1360652, cpu_share=0.07, mem_res=1330920, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=262636, cpu_share=0.0, mem_res=235596, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57832, cpu_share=0.01, mem_res=47396, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=7016472, free=407556, buffers=407080, cached=8665004, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.81, five_min_avg=1.84, fifteen_min_avg=2.25, cpu_share=8.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 06:06:27.398 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 06:06:29.411 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 06:06:34.464 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12223104, partition_space_available_1k=68867380, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1360848, cpu_share=0.06, mem_res=1331060, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=262628, cpu_share=0.0, mem_res=235596, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57828, cpu_share=0.01, mem_res=47396, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=7023492, free=395616, buffers=407240, cached=8669620, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.67, five_min_avg=1.78, fifteen_min_avg=2.2, cpu_share=7.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 06:07:27.397 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 06:07:29.465 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 06:07:34.510 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12232392, partition_space_available_1k=68858092, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1369968, cpu_share=0.19, mem_res=1339028, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=263120, cpu_share=0.01, mem_res=236088, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57836, cpu_share=0.01, mem_res=47396, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=7040612, free=369384, buffers=407120, cached=8678164, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.03, five_min_avg=1.85, fifteen_min_avg=2.2, cpu_share=9.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 06:08:27.397 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 06:08:30.288 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 06:08:35.334 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12242840, partition_space_available_1k=68847644, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1377300, cpu_share=0.19, mem_res=1346000, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=263904, cpu_share=0.01, mem_res=236948, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57820, cpu_share=0.01, mem_res=47396, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=7092676, free=351348, buffers=405712, cached=8645892, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.08, five_min_avg=1.88, fifteen_min_avg=2.18, cpu_share=10.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 06:09:27.409 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 06:09:29.477 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 06:09:34.522 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12255248, partition_space_available_1k=68835236, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1368184, cpu_share=0.21, mem_res=1336796, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=263996, cpu_share=0.01, mem_res=237064, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57828, cpu_share=0.01, mem_res=47396, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=7064880, free=385364, buffers=405240, cached=8640116, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.68, five_min_avg=1.81, fifteen_min_avg=2.14, cpu_share=9.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 06:10:27.435 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 06:10:29.516 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 06:10:34.560 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12267572, partition_space_available_1k=68822912, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1389940, cpu_share=0.19, mem_res=1358120, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=263944, cpu_share=0.01, mem_res=237096, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57816, cpu_share=0.01, mem_res=47396, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=7126372, free=351396, buffers=404856, cached=8613340, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.25, five_min_avg=1.66, fifteen_min_avg=2.06, cpu_share=9.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 06:11:27.449 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 06:11:29.703 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 06:11:34.762 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12304756, partition_space_available_1k=68785728, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1361356, cpu_share=0.17, mem_res=1328924, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=264004, cpu_share=0.01, mem_res=237184, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57816, cpu_share=0.01, mem_res=47396, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=7114996, free=346740, buffers=405012, cached=8629408, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.73, five_min_avg=1.72, fifteen_min_avg=2.06, cpu_share=10.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 06:12:27.469 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 06:12:29.494 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 06:12:34.539 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12318804, partition_space_available_1k=68771680, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1410364, cpu_share=0.13, mem_res=1377076, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=263904, cpu_share=0.01, mem_res=237192, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57808, cpu_share=0.01, mem_res=47396, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=7142840, free=349980, buffers=404864, cached=8598792, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.3, five_min_avg=1.87, fifteen_min_avg=2.09, cpu_share=9.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 06:13:27.408 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 06:13:29.333 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 06:13:34.377 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12326440, partition_space_available_1k=68764044, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1388028, cpu_share=0.14, mem_res=1354664, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=263912, cpu_share=0.01, mem_res=237200, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57816, cpu_share=0.01, mem_res=47396, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=7116052, free=379056, buffers=404880, cached=8596408, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.87, five_min_avg=2.06, fifteen_min_avg=2.14, cpu_share=9.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 06:14:27.395 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 06:14:29.376 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 06:14:34.428 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12336764, partition_space_available_1k=68753720, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1382580, cpu_share=0.14, mem_res=1349152, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=263832, cpu_share=0.01, mem_res=237212, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57800, cpu_share=0.01, mem_res=47396, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=7153656, free=370548, buffers=404820, cached=8567596, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.9, five_min_avg=1.95, fifteen_min_avg=2.09, cpu_share=9.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 06:15:27.444 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 06:15:29.451 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 06:15:34.496 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12347200, partition_space_available_1k=68743284, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1383840, cpu_share=0.1, mem_res=1350372, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=274916, cpu_share=0.01, mem_res=248372, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57800, cpu_share=0.01, mem_res=47396, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=7166532, free=375308, buffers=404740, cached=8550360, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.45, five_min_avg=2.14, fifteen_min_avg=2.15, cpu_share=8.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 06:16:27.427 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 06:16:29.378 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 06:16:34.429 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12360052, partition_space_available_1k=68730432, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1381880, cpu_share=0.1, mem_res=1348352, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=274900, cpu_share=0.0, mem_res=248376, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57800, cpu_share=0.01, mem_res=47396, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=7170688, free=368940, buffers=404752, cached=8552548, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.75, five_min_avg=2.03, fifteen_min_avg=2.12, cpu_share=8.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 06:17:27.442 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 06:17:29.512 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 06:17:34.555 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12371264, partition_space_available_1k=68719220, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1361656, cpu_share=0.11, mem_res=1327192, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=272084, cpu_share=0.01, mem_res=245588, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57848, cpu_share=0.01, mem_res=47396, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=7088728, free=451116, buffers=404780, cached=8552284, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.54, five_min_avg=1.93, fifteen_min_avg=2.07, cpu_share=8.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 06:18:27.556 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 06:18:29.497 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 06:18:34.551 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12381824, partition_space_available_1k=68708660, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1368052, cpu_share=0.11, mem_res=1333472, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=265876, cpu_share=0.01, mem_res=239364, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57856, cpu_share=0.02, mem_res=47396, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=7087864, free=433900, buffers=404980, cached=8570272, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.94, five_min_avg=2.01, fifteen_min_avg=2.09, cpu_share=8.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 06:19:27.446 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 06:19:29.449 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 06:19:34.501 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12387456, partition_space_available_1k=68703028, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1394384, cpu_share=0.15, mem_res=1359392, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=265896, cpu_share=0.01, mem_res=239372, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57856, cpu_share=0.01, mem_res=47396, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=7155328, free=359672, buffers=404980, cached=8576964, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.32, five_min_avg=2.59, fifteen_min_avg=2.28, cpu_share=9.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 06:20:27.583 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 03/26/2026 06:20:29.733 7f77fca83040 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 1 03/26/2026 06:20:34.784 7f77fca83040 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-5546-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12396556, partition_space_available_1k=68693928, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1358360, cpu_share=0.08, mem_res=1323356, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=265888, cpu_share=0.0, mem_res=239372, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=57860, cpu_share=0.01, mem_res=47396, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175640, used=7129832, free=368344, buffers=404996, cached=8593764, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.25, five_min_avg=2.31, fifteen_min_avg=2.2, cpu_share=8.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 03/26/2026 06:21:18.742 7f77f99a05e0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map