11/06/2025 01:40:53.914 7fc7a20f1940 [contrail-config-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 11/06/2025 01:40:53.914 7fc7a20f1940 [contrail-config-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 11/06/2025 01:40:53.914 7fc7a20f1940 [contrail-config-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 11/06/2025 01:40:53.925 7fc79f161400 [contrail-config-database-nodemgr] [ERROR]: Cannot write http_port 8112 to /tmp/contrail-config-database-nodemgr.0.http_port 11/06/2025 01:40:53.927 7fc79f161400 [contrail-config-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8112 11/06/2025 01:40:53.929 7fc79f1617c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 11/06/2025 01:40:53.931 7fc79f1617c0 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/06/2025 01:40:53.931 7fc79f1617c0 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 11/06/2025 01:40:53.932 7fc79f1617c0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 11/06/2025 01:40:53.932 7fc79f1617c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:40:53.932 7fc79f1617c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/06/2025 01:40:53.932 7fc79f1617c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:40:53.932 7fc79f1617c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:40:53.932 7fc79f1617c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762393253931888 collector_name = collector_ip = collector_list = [ 10.0.0.73:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 11/06/2025 01:40:53.932 7fc79f1617c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:40:54.148 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 35609187984594228239835986085753771469453544480899414486767797580069724735122, 'start': '1762393250000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'zookeeper', 'group': 'zookeeper', 'pid': 40195826946396677761494157204293543264777874173384937413942212647673390371248, 'start': '1762393250000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-config-database-nodemgr', 'group': 'contrail-config-database-nodemgr', 'pid': 32169413591333726638883218820206203554156734864532876009005223381641342500196, 'start': '1762393250000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 11/06/2025 01:40:54.513 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:40:54.514 7fc79f161860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 build_info = {"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-11-06 00:51:06.197767", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-12531-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed "}]} system_cpu_info = << num_socket = 1 num_cpu = 4 num_core_per_socket = 4 num_thread_per_core = 1 >> running_package_version = package contrail-lib is not installed installed_package_version = package contrail-lib is not installed >> 11/06/2025 01:40:54.514 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:40:54.514 7fc7a20f1940 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-juju-k8s-1689-1', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1762393254514756, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1689-1', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 11/06/2025 01:40:54.515 7fc7a20f1940 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-juju-k8s-1689-1', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1762393254515042, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1689-1', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1762393250000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1762393250000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1762393250000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-11-06 00:51:06.197767", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-12531-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed\n"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 11/06/2025 01:40:54.516 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:40:54.516 7fc79f161860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/06/2025 01:40:54.516 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:40:54.516 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:40:54.516 7fc79f161860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1762393250000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = zookeeper process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1762393250000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1762393250000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-11-06 00:51:06.197767", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-12531-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed "}]} >> 11/06/2025 01:40:54.516 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:40:55.035 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/06/2025 01:40:55.663 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 11/06/2025 01:40:55.664 7fc79f4657c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:40:55.665 7fc79f4657c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 database_usage = [ << disk_space_used_1k = 12052620 disk_space_available_1k = 69037864 config_db_size_1k = 4 >>, ] >> 11/06/2025 01:40:55.665 7fc79f4657c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:40:56.601 7fc7a20f1940 [contrail-config-database-nodemgr] [ERROR]: Failed to connect to database by CQL and socket: Result: 1 11/06/2025 01:40:56.601 7fc7a20f1940 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-juju-k8s-1689-1', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1762393256601635, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1689-1', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 11/06/2025 01:40:56.602 7fc79f4657c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:40:56.603 7fc79f4657c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 11/06/2025 01:40:56.603 7fc79f4657c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:40:56.982 7fc79f161f40 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/06/2025 01:40:57.931 7fc79f161f40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/06/2025 01:40:57.932 7fc79f161f40 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/06/2025 01:40:57.933 7fc79f161f40 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/06/2025 01:40:57.933 7fc79f161f40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:40:57.933 7fc79f161f40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/06/2025 01:40:57.933 7fc79f161f40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:40:57.933 7fc79f161f40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/06/2025 01:40:57.933 7fc79f161f40 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/06/2025 01:40:57.933 7fc79f161f40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:40:57.933 7fc79f161f40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/06/2025 01:40:57.933 7fc79f161f40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:40:57.933 7fc79f161f40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:40:57.934 7fc79f161f40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762393253931888 collector_name = collector_ip = 10.0.0.73:8086 collector_list = [ 10.0.0.73:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/06/2025 01:40:57.934 7fc79f161f40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:40:59.831 7fc7a20f1940 [contrail-config-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 2 11/06/2025 01:41:01.935 7fc79f465f40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/06/2025 01:41:01.936 7fc79f465f40 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/06/2025 01:41:01.936 7fc79f465f40 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/06/2025 01:41:01.936 7fc79f465f40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:41:01.936 7fc79f465f40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/06/2025 01:41:01.936 7fc79f465f40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:41:01.936 7fc79f465f40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/06/2025 01:41:01.936 7fc79f465f40 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/06/2025 01:41:01.937 7fc79f465f40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:01.937 7fc79f465f40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/06/2025 01:41:01.937 7fc79f465f40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:01.937 7fc79f465f40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:01.937 7fc79f465f40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762393253931888 collector_name = collector_ip = 10.0.0.73:8086 collector_list = [ 10.0.0.73:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/06/2025 01:41:01.937 7fc79f465f40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:01.979 7fc7a20f1940 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr config_db_uuid'. Error is: Result: 1 11/06/2025 01:41:02.996 7fc7a20f1940 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr to_bgp_keyspace'. Error is: Result: 1 11/06/2025 01:41:03.914 7fc7a20f1940 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr svc_monitor_keyspace'. Error is: Result: 1 11/06/2025 01:41:04.834 7fc7a20f1940 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr DISCOVERY_SERVER'. Error is: Result: 1 11/06/2025 01:41:05.692 7fc7a20f1940 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr useragent'. Error is: Result: 1 11/06/2025 01:41:05.936 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/06/2025 01:41:05.937 7fc79f465e00 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/06/2025 01:41:05.937 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/06/2025 01:41:05.938 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:41:05.938 7fc79f465e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/06/2025 01:41:05.938 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:41:05.938 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/06/2025 01:41:05.938 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/06/2025 01:41:05.938 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:05.938 7fc79f465e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/06/2025 01:41:05.939 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:05.939 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:05.939 7fc79f465e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762393253931888 collector_name = collector_ip = 10.0.0.73:8086 collector_list = [ 10.0.0.73:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/06/2025 01:41:05.939 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:06.512 7fc7a20f1940 [contrail-config-database-nodemgr] [ERROR]: Failed to run cmd: 'nodetool -p 7201 repair -pr dm_keyspace'. Error is: Result: 1 11/06/2025 01:41:09.938 7fc79f2422c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/06/2025 01:41:09.940 7fc79f2422c0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/06/2025 01:41:09.940 7fc79f2422c0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/06/2025 01:41:09.940 7fc79f2422c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:41:09.940 7fc79f2422c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/06/2025 01:41:09.940 7fc79f2422c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:41:09.940 7fc79f2422c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/06/2025 01:41:09.941 7fc79f2422c0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/06/2025 01:41:09.941 7fc79f2422c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:09.941 7fc79f2422c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/06/2025 01:41:09.941 7fc79f2422c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:09.941 7fc79f2422c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:09.941 7fc79f2422c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762393253931888 collector_name = collector_ip = 10.0.0.73:8086 collector_list = [ 10.0.0.73:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/06/2025 01:41:09.941 7fc79f2422c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:11.549 7fc7a20f1940 [contrail-config-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-juju-k8s-1689-1', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1762393271549370, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1689-1', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1762393250000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='zookeeper', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1762393250000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1762393250000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-11-06 00:51:06.197767", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-12531-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed\n"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 11/06/2025 01:41:11.549 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1689-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12059252, partition_space_available_1k=69031232, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1306920, cpu_share=0.0, mem_res=1294356, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52740, cpu_share=0.0, mem_res=48324, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54504, cpu_share=0.0, mem_res=42620, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=3381548, free=268628, buffers=433604, cached=12471888, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.91, five_min_avg=1.69, fifteen_min_avg=0.78, cpu_share=0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/06/2025 01:41:11.550 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:11.550 7fc79f465e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1762393250000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = zookeeper process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1762393250000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1762393250000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "v0+nightly", "build-time" : "2025-11-06 00:51:06.197767", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-12531-1.novalocal", "build-id" : "package contrail", "build-number" : "lib is not installed "}]} >> 11/06/2025 01:41:11.550 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:11.550 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:11.550 7fc79f465e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 12059252 partition_space_available_1k = 69031232 percentage_partition_space_used = 15 >> } all_core_file_list = [ ] process_mem_cpu_usage = { cassandra : << mem_virt = 1306920 cpu_share = 0.0 mem_res = 1294356 >>zookeeper : << mem_virt = 52740 cpu_share = 0.0 mem_res = 48324 >>contrail-config-database-nodemgr : << mem_virt = 54504 cpu_share = 0.0 mem_res = 42620 >> } system_mem_usage = << total = 16184208 used = 3381548 free = 268628 buffers = 433604 cached = 12471888 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 3.91 five_min_avg = 1.69 fifteen_min_avg = 0.78 cpu_share = 0 node_type = config-database-node >> >> 11/06/2025 01:41:11.550 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:13.943 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/06/2025 01:41:13.945 7fc79f465e00 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/06/2025 01:41:13.945 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/06/2025 01:41:13.945 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:41:13.945 7fc79f465e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/06/2025 01:41:13.945 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:41:13.945 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/06/2025 01:41:13.946 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/06/2025 01:41:13.946 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:13.946 7fc79f465e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/06/2025 01:41:13.946 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:13.946 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:13.946 7fc79f465e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762393253931888 collector_name = collector_ip = 10.0.0.73:8086 collector_list = [ 10.0.0.73:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/06/2025 01:41:13.946 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:17.950 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/06/2025 01:41:17.951 7fc79f465e00 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/06/2025 01:41:17.951 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/06/2025 01:41:17.951 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:41:17.951 7fc79f465e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/06/2025 01:41:17.951 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:41:17.951 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/06/2025 01:41:17.952 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/06/2025 01:41:17.952 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:17.952 7fc79f465e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/06/2025 01:41:17.952 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:17.952 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:17.952 7fc79f465e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762393253931888 collector_name = collector_ip = 10.0.0.73:8086 collector_list = [ 10.0.0.73:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/06/2025 01:41:17.952 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:21.955 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/06/2025 01:41:21.956 7fc79f465e00 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/06/2025 01:41:21.956 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/06/2025 01:41:21.956 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:41:21.956 7fc79f465e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/06/2025 01:41:21.957 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:41:21.957 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/06/2025 01:41:21.957 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/06/2025 01:41:21.957 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:21.957 7fc79f465e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/06/2025 01:41:21.957 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:21.957 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:21.957 7fc79f465e00 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762393253931888 collector_name = collector_ip = 10.0.0.73:8086 collector_list = [ 10.0.0.73:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/06/2025 01:41:21.957 7fc79f465e00 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:25.958 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/06/2025 01:41:25.959 7fc79f465b80 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/06/2025 01:41:25.959 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/06/2025 01:41:25.959 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:41:25.959 7fc79f465b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/06/2025 01:41:25.959 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:41:25.959 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/06/2025 01:41:25.960 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/06/2025 01:41:25.960 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:25.960 7fc79f465b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/06/2025 01:41:25.960 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:25.960 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:25.960 7fc79f465b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762393253931888 collector_name = collector_ip = 10.0.0.73:8086 collector_list = [ 10.0.0.73:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/06/2025 01:41:25.960 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:29.963 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/06/2025 01:41:29.964 7fc79f465b80 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/06/2025 01:41:29.964 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/06/2025 01:41:29.965 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:41:29.965 7fc79f465b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/06/2025 01:41:29.965 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:41:29.966 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/06/2025 01:41:29.966 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/06/2025 01:41:29.967 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:29.967 7fc79f465b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/06/2025 01:41:29.967 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:29.967 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:29.967 7fc79f465b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762393253931888 collector_name = collector_ip = 10.0.0.73:8086 collector_list = [ 10.0.0.73:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/06/2025 01:41:29.968 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:33.967 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/06/2025 01:41:33.968 7fc79f242680 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/06/2025 01:41:33.968 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/06/2025 01:41:33.968 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:41:33.969 7fc79f242680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/06/2025 01:41:33.969 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:41:33.969 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/06/2025 01:41:33.969 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/06/2025 01:41:33.969 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:33.969 7fc79f242680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/06/2025 01:41:33.969 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:33.969 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:33.969 7fc79f242680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762393253931888 collector_name = collector_ip = 10.0.0.73:8086 collector_list = [ 10.0.0.73:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/06/2025 01:41:33.969 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:37.973 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/06/2025 01:41:37.974 7fc79f465b80 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/06/2025 01:41:37.974 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/06/2025 01:41:37.975 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:41:37.975 7fc79f465b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/06/2025 01:41:37.975 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:41:37.975 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/06/2025 01:41:37.976 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/06/2025 01:41:37.976 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:37.976 7fc79f465b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/06/2025 01:41:37.976 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:37.977 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:37.977 7fc79f465b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762393253931888 collector_name = collector_ip = 10.0.0.73:8086 collector_list = [ 10.0.0.73:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/06/2025 01:41:37.977 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:41.975 7fc79f242ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/06/2025 01:41:41.976 7fc79f242ae0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/06/2025 01:41:41.976 7fc79f242ae0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/06/2025 01:41:41.976 7fc79f242ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:41:41.977 7fc79f242ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/06/2025 01:41:41.977 7fc79f242ae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:41:41.977 7fc79f242ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/06/2025 01:41:41.977 7fc79f242ae0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/06/2025 01:41:41.977 7fc79f242ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:41.977 7fc79f242ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/06/2025 01:41:41.977 7fc79f242ae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:41.977 7fc79f242ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:41.977 7fc79f242ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762393253931888 collector_name = collector_ip = 10.0.0.73:8086 collector_list = [ 10.0.0.73:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/06/2025 01:41:41.977 7fc79f242ae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:45.980 7fc79f242ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/06/2025 01:41:45.981 7fc79f242ae0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/06/2025 01:41:45.981 7fc79f242ae0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/06/2025 01:41:45.981 7fc79f242ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:41:45.981 7fc79f242ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/06/2025 01:41:45.981 7fc79f242ae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:41:45.981 7fc79f242ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/06/2025 01:41:45.982 7fc79f242ae0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/06/2025 01:41:45.982 7fc79f242ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:45.982 7fc79f242ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/06/2025 01:41:45.982 7fc79f242ae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:45.982 7fc79f242ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:45.982 7fc79f242ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762393253931888 collector_name = collector_ip = 10.0.0.73:8086 collector_list = [ 10.0.0.73:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/06/2025 01:41:45.982 7fc79f242ae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:49.984 7fc79f4657c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/06/2025 01:41:49.985 7fc79f4657c0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/06/2025 01:41:49.985 7fc79f4657c0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/06/2025 01:41:49.985 7fc79f4657c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:41:49.985 7fc79f4657c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/06/2025 01:41:49.985 7fc79f4657c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:41:49.985 7fc79f4657c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/06/2025 01:41:49.986 7fc79f4657c0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/06/2025 01:41:49.986 7fc79f4657c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:49.986 7fc79f4657c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/06/2025 01:41:49.986 7fc79f4657c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:49.986 7fc79f4657c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:49.986 7fc79f4657c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762393253931888 collector_name = collector_ip = 10.0.0.73:8086 collector_list = [ 10.0.0.73:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/06/2025 01:41:49.986 7fc79f4657c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:53.989 7fc79f242ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/06/2025 01:41:53.990 7fc79f242ae0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/06/2025 01:41:53.991 7fc79f242ae0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/06/2025 01:41:53.991 7fc79f242ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:41:53.991 7fc79f242ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/06/2025 01:41:53.991 7fc79f242ae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:41:53.991 7fc79f242ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/06/2025 01:41:53.991 7fc79f242ae0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/06/2025 01:41:53.991 7fc79f242ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:53.992 7fc79f242ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/06/2025 01:41:53.992 7fc79f242ae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:53.992 7fc79f242ae0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:53.992 7fc79f242ae0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762393253931888 collector_name = collector_ip = 10.0.0.73:8086 collector_list = [ 10.0.0.73:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/06/2025 01:41:53.992 7fc79f242ae0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:54.666 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/06/2025 01:41:54.864 7fc79f161b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:54.864 7fc79f161b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 database_usage = [ << disk_space_used_1k = 12097456 disk_space_available_1k = 68993028 config_db_size_1k = 1028 >>, ] >> 11/06/2025 01:41:54.864 7fc79f161b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:55.314 7fc7a20f1940 [contrail-config-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-juju-k8s-1689-1', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1762393315314810, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1689-1', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.73:8086'], status='Down', description='Connect to Idle on EvTcpConnectFail')]), flag_infos=None, description='Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo'), _seqnum=0) 11/06/2025 01:41:55.315 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:55.316 7fc79f161860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/06/2025 01:41:55.316 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:57.414 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-juju-k8s-1689-1', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1762393317414086, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-juju-k8s-1689-1', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0)], _table='ObjectConfigDatabaseInfo'), _seqnum=0) 11/06/2025 01:41:57.414 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:CassandraStatusUVE] map 11/06/2025 01:41:57.417 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:57.418 7fc79f161860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] CassandraStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 cassandra_compaction_task = << pending_compaction_tasks = 0 >> thread_pool_stats = [ << pool_name = ReadStage active = 0 pending = 0 all_time_blocked = 0 >>, << pool_name = CompactionExecutor active = 0 pending = 0 all_time_blocked = 0 >>, << pool_name = MutationStage active = 0 pending = 0 all_time_blocked = 0 >>, << pool_name = Native-Transport-Requests active = 0 pending = 0 all_time_blocked = 0 >>, << pool_name = MemtableFlushWriter active = 0 pending = 0 all_time_blocked = 0 >>, ] >> 11/06/2025 01:41:57.418 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:57.991 7fc79f2422c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/06/2025 01:41:57.993 7fc79f2422c0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/06/2025 01:41:57.993 7fc79f2422c0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/06/2025 01:41:57.993 7fc79f2422c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:41:57.993 7fc79f2422c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/06/2025 01:41:57.993 7fc79f2422c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:41:57.993 7fc79f2422c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/06/2025 01:41:57.994 7fc79f2422c0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/06/2025 01:41:57.994 7fc79f2422c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:57.994 7fc79f2422c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/06/2025 01:41:57.994 7fc79f2422c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:57.994 7fc79f2422c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:41:57.994 7fc79f2422c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762393253931888 collector_name = collector_ip = 10.0.0.73:8086 collector_list = [ 10.0.0.73:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/06/2025 01:41:57.994 7fc79f2422c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:01.995 7fc79f2422c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/06/2025 01:42:01.996 7fc79f2422c0 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/06/2025 01:42:01.996 7fc79f2422c0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/06/2025 01:42:01.996 7fc79f2422c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:42:01.996 7fc79f2422c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/06/2025 01:42:01.996 7fc79f2422c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:42:01.997 7fc79f2422c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/06/2025 01:42:01.997 7fc79f2422c0 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/06/2025 01:42:01.997 7fc79f2422c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:01.997 7fc79f2422c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/06/2025 01:42:01.997 7fc79f2422c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:01.997 7fc79f2422c0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:01.997 7fc79f2422c0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762393253931888 collector_name = collector_ip = 10.0.0.73:8086 collector_list = [ 10.0.0.73:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/06/2025 01:42:01.997 7fc79f2422c0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:02.869 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1689-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12097608, partition_space_available_1k=68992876, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1339676, cpu_share=0.38, mem_res=1323376, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=60476, cpu_share=0.02, mem_res=55736, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54320, cpu_share=0.01, mem_res=42532, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=4472344, free=299120, buffers=420392, cached=11303724, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.02, five_min_avg=2.64, fifteen_min_avg=1.15, cpu_share=12.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/06/2025 01:42:02.869 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:02.869 7fc79f161860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 12097608 partition_space_available_1k = 68992876 percentage_partition_space_used = 15 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1339676 cpu_share = 0.38 mem_res = 1323376 >>zookeeper : << mem_virt = 60476 cpu_share = 0.02 mem_res = 55736 >>contrail-config-database-nodemgr : << mem_virt = 54320 cpu_share = 0.01 mem_res = 42532 >> } system_mem_usage = << total = 16184208 used = 4472344 free = 299120 buffers = 420392 cached = 11303724 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 6.02 five_min_avg = 2.64 fifteen_min_avg = 1.15 cpu_share = 12.75 node_type = config-database-node >> >> 11/06/2025 01:42:02.869 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:05.999 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/06/2025 01:42:06.000 7fc79f161860 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/06/2025 01:42:06.001 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/06/2025 01:42:06.001 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:42:06.001 7fc79f161860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/06/2025 01:42:06.001 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:42:06.001 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/06/2025 01:42:06.001 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/06/2025 01:42:06.001 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:06.001 7fc79f161860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/06/2025 01:42:06.001 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:06.002 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:06.002 7fc79f161860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762393253931888 collector_name = collector_ip = 10.0.0.73:8086 collector_list = [ 10.0.0.73:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/06/2025 01:42:06.002 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:10.001 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/06/2025 01:42:10.002 7fc79f161860 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/06/2025 01:42:10.002 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/06/2025 01:42:10.002 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:42:10.002 7fc79f161860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/06/2025 01:42:10.002 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:42:10.002 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/06/2025 01:42:10.003 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/06/2025 01:42:10.003 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:10.003 7fc79f161860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/06/2025 01:42:10.003 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:10.003 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:10.003 7fc79f161860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762393253931888 collector_name = collector_ip = 10.0.0.73:8086 collector_list = [ 10.0.0.73:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/06/2025 01:42:10.003 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:14.006 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/06/2025 01:42:14.007 7fc79f161860 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/06/2025 01:42:14.007 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/06/2025 01:42:14.007 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:42:14.008 7fc79f161860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/06/2025 01:42:14.008 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:42:14.008 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/06/2025 01:42:14.008 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/06/2025 01:42:14.008 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:14.008 7fc79f161860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/06/2025 01:42:14.008 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:14.008 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:14.008 7fc79f161860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762393253931888 collector_name = collector_ip = 10.0.0.73:8086 collector_list = [ 10.0.0.73:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/06/2025 01:42:14.009 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:18.011 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/06/2025 01:42:18.013 7fc79f161860 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/06/2025 01:42:18.013 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/06/2025 01:42:18.013 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:42:18.013 7fc79f161860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/06/2025 01:42:18.013 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:42:18.013 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/06/2025 01:42:18.013 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/06/2025 01:42:18.014 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:18.014 7fc79f161860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/06/2025 01:42:18.014 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:18.014 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:18.014 7fc79f161860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762393253931888 collector_name = collector_ip = 10.0.0.73:8086 collector_list = [ 10.0.0.73:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/06/2025 01:42:18.014 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:22.016 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/06/2025 01:42:22.017 7fc79f161860 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/06/2025 01:42:22.017 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/06/2025 01:42:22.017 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:42:22.017 7fc79f161860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/06/2025 01:42:22.017 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:42:22.017 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/06/2025 01:42:22.018 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/06/2025 01:42:22.018 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:22.018 7fc79f161860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/06/2025 01:42:22.018 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:22.018 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:22.018 7fc79f161860 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762393253931888 collector_name = collector_ip = 10.0.0.73:8086 collector_list = [ 10.0.0.73:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/06/2025 01:42:22.018 7fc79f161860 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:26.018 7fc79f242220 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/06/2025 01:42:26.019 7fc79f242220 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/06/2025 01:42:26.020 7fc79f242220 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/06/2025 01:42:26.020 7fc79f242220 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:42:26.020 7fc79f242220 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/06/2025 01:42:26.020 7fc79f242220 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:42:26.020 7fc79f242220 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/06/2025 01:42:26.020 7fc79f242220 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/06/2025 01:42:26.020 7fc79f242220 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:26.021 7fc79f242220 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/06/2025 01:42:26.021 7fc79f242220 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:26.021 7fc79f242220 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:26.021 7fc79f242220 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762393253931888 collector_name = collector_ip = 10.0.0.73:8086 collector_list = [ 10.0.0.73:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/06/2025 01:42:26.021 7fc79f242220 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:30.021 7fc79f242220 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/06/2025 01:42:30.023 7fc79f242220 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/06/2025 01:42:30.023 7fc79f242220 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/06/2025 01:42:30.023 7fc79f242220 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:42:30.023 7fc79f242220 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/06/2025 01:42:30.023 7fc79f242220 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:42:30.023 7fc79f242220 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/06/2025 01:42:30.024 7fc79f242220 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/06/2025 01:42:30.024 7fc79f242220 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:30.024 7fc79f242220 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/06/2025 01:42:30.024 7fc79f242220 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:30.024 7fc79f242220 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:30.024 7fc79f242220 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762393253931888 collector_name = collector_ip = 10.0.0.73:8086 collector_list = [ 10.0.0.73:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/06/2025 01:42:30.024 7fc79f242220 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:34.026 7fc79f242220 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/06/2025 01:42:34.027 7fc79f242220 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/06/2025 01:42:34.027 7fc79f242220 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/06/2025 01:42:34.027 7fc79f242220 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:42:34.027 7fc79f242220 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/06/2025 01:42:34.027 7fc79f242220 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:42:34.027 7fc79f242220 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/06/2025 01:42:34.027 7fc79f242220 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/06/2025 01:42:34.028 7fc79f242220 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:34.028 7fc79f242220 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/06/2025 01:42:34.028 7fc79f242220 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:34.028 7fc79f242220 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:34.028 7fc79f242220 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762393253931888 collector_name = collector_ip = 10.0.0.73:8086 collector_list = [ 10.0.0.73:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/06/2025 01:42:34.028 7fc79f242220 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:38.029 7fc79f242220 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/06/2025 01:42:38.030 7fc79f242220 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/06/2025 01:42:38.031 7fc79f242220 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/06/2025 01:42:38.031 7fc79f242220 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:42:38.031 7fc79f242220 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/06/2025 01:42:38.031 7fc79f242220 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:42:38.031 7fc79f242220 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/06/2025 01:42:38.031 7fc79f242220 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/06/2025 01:42:38.031 7fc79f242220 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:38.032 7fc79f242220 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/06/2025 01:42:38.032 7fc79f242220 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:38.032 7fc79f242220 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:38.032 7fc79f242220 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762393253931888 collector_name = collector_ip = 10.0.0.73:8086 collector_list = [ 10.0.0.73:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/06/2025 01:42:38.032 7fc79f242220 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:42.032 7fc79f242720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/06/2025 01:42:42.033 7fc79f242720 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/06/2025 01:42:42.034 7fc79f242720 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/06/2025 01:42:42.034 7fc79f242720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:42:42.034 7fc79f242720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/06/2025 01:42:42.034 7fc79f242720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:42:42.034 7fc79f242720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/06/2025 01:42:42.034 7fc79f242720 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/06/2025 01:42:42.034 7fc79f242720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:42.035 7fc79f242720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/06/2025 01:42:42.035 7fc79f242720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:42.035 7fc79f242720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:42.035 7fc79f242720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762393253931888 collector_name = collector_ip = 10.0.0.73:8086 collector_list = [ 10.0.0.73:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/06/2025 01:42:42.035 7fc79f242720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:46.035 7fc79f242720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/06/2025 01:42:46.036 7fc79f242720 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/06/2025 01:42:46.036 7fc79f242720 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/06/2025 01:42:46.036 7fc79f242720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:42:46.037 7fc79f242720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/06/2025 01:42:46.037 7fc79f242720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:42:46.037 7fc79f242720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/06/2025 01:42:46.037 7fc79f242720 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/06/2025 01:42:46.037 7fc79f242720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:46.037 7fc79f242720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/06/2025 01:42:46.037 7fc79f242720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:46.037 7fc79f242720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:46.038 7fc79f242720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762393253931888 collector_name = collector_ip = 10.0.0.73:8086 collector_list = [ 10.0.0.73:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/06/2025 01:42:46.038 7fc79f242720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:50.039 7fc79f242720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/06/2025 01:42:50.040 7fc79f242720 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/06/2025 01:42:50.040 7fc79f242720 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/06/2025 01:42:50.040 7fc79f242720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:42:50.041 7fc79f242720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/06/2025 01:42:50.041 7fc79f242720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:42:50.041 7fc79f242720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/06/2025 01:42:50.041 7fc79f242720 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/06/2025 01:42:50.041 7fc79f242720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:50.041 7fc79f242720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/06/2025 01:42:50.041 7fc79f242720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:50.041 7fc79f242720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:50.041 7fc79f242720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762393253931888 collector_name = collector_ip = 10.0.0.73:8086 collector_list = [ 10.0.0.73:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/06/2025 01:42:50.042 7fc79f242720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:54.045 7fc79f242720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/06/2025 01:42:54.046 7fc79f242720 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/06/2025 01:42:54.046 7fc79f242720 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/06/2025 01:42:54.046 7fc79f242720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:42:54.046 7fc79f242720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/06/2025 01:42:54.046 7fc79f242720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:42:54.046 7fc79f242720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/06/2025 01:42:54.047 7fc79f242720 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/06/2025 01:42:54.047 7fc79f242720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:54.047 7fc79f242720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/06/2025 01:42:54.047 7fc79f242720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:54.047 7fc79f242720 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:54.047 7fc79f242720 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762393253931888 collector_name = collector_ip = 10.0.0.73:8086 collector_list = [ 10.0.0.73:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/06/2025 01:42:54.048 7fc79f242720 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:54.658 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/06/2025 01:42:54.850 7fc79f2425e0 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:54.851 7fc79f2425e0 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 database_usage = [ << disk_space_used_1k = 12100436 disk_space_available_1k = 68990048 config_db_size_1k = 1028 >>, ] >> 11/06/2025 01:42:54.851 7fc79f2425e0 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:58.047 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/06/2025 01:42:58.049 7fc79f465b80 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/06/2025 01:42:58.049 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/06/2025 01:42:58.049 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:42:58.049 7fc79f465b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/06/2025 01:42:58.049 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:42:58.049 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/06/2025 01:42:58.050 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/06/2025 01:42:58.050 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:58.050 7fc79f465b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/06/2025 01:42:58.050 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:58.050 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:42:58.050 7fc79f465b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762393253931888 collector_name = collector_ip = 10.0.0.73:8086 collector_list = [ 10.0.0.73:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/06/2025 01:42:58.050 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:43:02.051 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/06/2025 01:43:02.052 7fc79f465b80 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/06/2025 01:43:02.052 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/06/2025 01:43:02.052 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:43:02.052 7fc79f465b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/06/2025 01:43:02.052 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:43:02.052 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/06/2025 01:43:02.053 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/06/2025 01:43:02.053 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:43:02.053 7fc79f465b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/06/2025 01:43:02.053 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:43:02.053 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:43:02.053 7fc79f465b80 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762393253931888 collector_name = collector_ip = 10.0.0.73:8086 collector_list = [ 10.0.0.73:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/06/2025 01:43:02.053 7fc79f465b80 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:43:02.862 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1689-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12100504, partition_space_available_1k=68989980, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1314992, cpu_share=0.07, mem_res=1299076, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=63004, cpu_share=0.0, mem_res=58340, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54440, cpu_share=0.01, mem_res=42564, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=4472136, free=284868, buffers=421052, cached=11317524, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.03, five_min_avg=2.38, fifteen_min_avg=1.15, cpu_share=2.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/06/2025 01:43:02.863 7fc79f161f40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:43:02.863 7fc79f161f40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 12100504 partition_space_available_1k = 68989980 percentage_partition_space_used = 15 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1314992 cpu_share = 0.07 mem_res = 1299076 >>zookeeper : << mem_virt = 63004 cpu_share = 0.0 mem_res = 58340 >>contrail-config-database-nodemgr : << mem_virt = 54440 cpu_share = 0.01 mem_res = 42564 >> } system_mem_usage = << total = 16184208 used = 4472136 free = 284868 buffers = 421052 cached = 11317524 node_type = config-database-node >> system_cpu_usage = << one_min_avg = 3.03 five_min_avg = 2.38 fifteen_min_avg = 1.15 cpu_share = 2.5 node_type = config-database-node >> >> 11/06/2025 01:43:02.863 7fc79f161f40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:43:06.056 7fc79f161f40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/06/2025 01:43:06.057 7fc79f161f40 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/06/2025 01:43:06.057 7fc79f161f40 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/06/2025 01:43:06.057 7fc79f161f40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:43:06.057 7fc79f161f40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/06/2025 01:43:06.058 7fc79f161f40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:43:06.058 7fc79f161f40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/06/2025 01:43:06.058 7fc79f161f40 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/06/2025 01:43:06.058 7fc79f161f40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:43:06.058 7fc79f161f40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/06/2025 01:43:06.058 7fc79f161f40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:43:06.058 7fc79f161f40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:43:06.059 7fc79f161f40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762393253931888 collector_name = collector_ip = 10.0.0.73:8086 collector_list = [ 10.0.0.73:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/06/2025 01:43:06.059 7fc79f161f40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:43:10.060 7fc79f161f40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/06/2025 01:43:10.061 7fc79f161f40 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/06/2025 01:43:10.062 7fc79f161f40 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/06/2025 01:43:10.062 7fc79f161f40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:43:10.062 7fc79f161f40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/06/2025 01:43:10.062 7fc79f161f40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:43:10.062 7fc79f161f40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/06/2025 01:43:10.062 7fc79f161f40 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/06/2025 01:43:10.062 7fc79f161f40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:43:10.063 7fc79f161f40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/06/2025 01:43:10.063 7fc79f161f40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:43:10.063 7fc79f161f40 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:43:10.063 7fc79f161f40 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762393253931888 collector_name = collector_ip = 10.0.0.73:8086 collector_list = [ 10.0.0.73:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/06/2025 01:43:10.063 7fc79f161f40 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:43:14.066 7fc79f242d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/06/2025 01:43:14.067 7fc79f242d60 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/06/2025 01:43:14.067 7fc79f242d60 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/06/2025 01:43:14.067 7fc79f242d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:43:14.067 7fc79f242d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/06/2025 01:43:14.067 7fc79f242d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:43:14.067 7fc79f242d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/06/2025 01:43:14.067 7fc79f242d60 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/06/2025 01:43:14.068 7fc79f242d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:43:14.068 7fc79f242d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/06/2025 01:43:14.068 7fc79f242d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:43:14.068 7fc79f242d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:43:14.068 7fc79f242d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762393253931888 collector_name = collector_ip = 10.0.0.73:8086 collector_list = [ 10.0.0.73:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/06/2025 01:43:14.068 7fc79f242d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:43:18.071 7fc79f242d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/06/2025 01:43:18.072 7fc79f242d60 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/06/2025 01:43:18.075 7fc79f242d60 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/06/2025 01:43:18.075 7fc79f242d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:43:18.075 7fc79f242d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/06/2025 01:43:18.075 7fc79f242d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:43:18.076 7fc79f242d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/06/2025 01:43:18.076 7fc79f242d60 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/06/2025 01:43:18.076 7fc79f242d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:43:18.076 7fc79f242d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/06/2025 01:43:18.076 7fc79f242d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:43:18.076 7fc79f242d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:43:18.076 7fc79f242d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762393253931888 collector_name = collector_ip = 10.0.0.73:8086 collector_list = [ 10.0.0.73:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/06/2025 01:43:18.076 7fc79f242d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:43:19.656 7fc79f242d60 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/06/2025 01:43:22.074 7fc79f242d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/06/2025 01:43:22.076 7fc79f242d60 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/06/2025 01:43:22.076 7fc79f242d60 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/06/2025 01:43:22.076 7fc79f242d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:43:22.076 7fc79f242d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/06/2025 01:43:22.076 7fc79f242d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:43:22.076 7fc79f242d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/06/2025 01:43:22.076 7fc79f242d60 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/06/2025 01:43:22.076 7fc79f242d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:43:22.077 7fc79f242d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/06/2025 01:43:22.077 7fc79f242d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:43:22.077 7fc79f242d60 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:43:22.077 7fc79f242d60 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762393253931888 collector_name = collector_ip = 10.0.0.73:8086 collector_list = [ 10.0.0.73:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/06/2025 01:43:22.077 7fc79f242d60 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:43:26.077 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/06/2025 01:43:26.078 7fc79f242680 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/06/2025 01:43:26.078 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/06/2025 01:43:26.078 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:43:26.078 7fc79f242680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/06/2025 01:43:26.079 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:43:26.079 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/06/2025 01:43:26.079 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/06/2025 01:43:26.081 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:43:26.082 7fc79f242680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/06/2025 01:43:26.082 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:43:26.082 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:43:26.082 7fc79f242680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762393253931888 collector_name = collector_ip = 10.0.0.73:8086 collector_list = [ 10.0.0.73:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/06/2025 01:43:26.082 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:43:29.361 7fc79f242680 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/06/2025 01:43:30.079 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/06/2025 01:43:30.080 7fc79f242680 [contrail-config-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 11/06/2025 01:43:30.081 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/06/2025 01:43:30.081 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:43:30.081 7fc79f242680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/06/2025 01:43:30.081 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:43:30.081 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 11/06/2025 01:43:30.082 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 11/06/2025 01:43:30.082 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:43:30.082 7fc79f242680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 11/06/2025 01:43:30.082 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:43:30.082 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:43:30.082 7fc79f242680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1:ConfigDatabase:contrail-config-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8112 start_time = 1762393253931888 collector_name = collector_ip = 10.0.0.73:8086 collector_list = [ 10.0.0.73:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 11/06/2025 01:43:30.082 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 11/06/2025 01:43:34.083 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 11/06/2025 01:43:34.084 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Session Event: TCP Connected 11/06/2025 01:43:34.084 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 11/06/2025 01:43:34.084 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:43:34.084 7fc79f242680 [contrail-config-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1689-1 process_status = [ << module_id = contrail-config-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.73:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 11/06/2025 01:43:34.085 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 11/06/2025 01:43:34.085 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 11/06/2025 01:43:34.085 7fc79f242680 [contrail-config-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 11/06/2025 01:43:34.085 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 11/06/2025 01:43:34.085 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 11/06/2025 01:43:34.085 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 11/06/2025 01:43:34.087 7fc79f242d60 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001013 11/06/2025 01:43:34.088 7fc79f242d60 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 11/06/2025 01:43:34.088 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 11/06/2025 01:43:34.088 7fc79f242680 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 11/06/2025 01:43:34.088 7fc79f242680 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 11/06/2025 01:43:34.088 7fc79f242680 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:CassandraStatusUVE] map 11/06/2025 01:43:34.088 7fc79f242680 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/06/2025 01:43:34.089 7fc79f242680 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 11/06/2025 01:43:34.089 7fc79f242680 [contrail-config-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 11/06/2025 01:43:34.089 7fc79f242860 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 11/06/2025 01:43:34.090 7fc79f242860 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 11/06/2025 01:43:45.032 7fc79f242d60 [contrail-config-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001579 11/06/2025 01:43:45.032 7fc79f242d60 [contrail-config-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 11/06/2025 01:43:45.033 7fc79f242360 [contrail-config-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 11/06/2025 01:43:45.033 7fc79f242360 [contrail-config-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 4 11/06/2025 01:43:45.033 7fc79f242360 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:DatabaseUsage] map 11/06/2025 01:43:45.033 7fc79f242360 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:CassandraStatusUVE] map 11/06/2025 01:43:45.034 7fc79f242360 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/06/2025 01:43:45.034 7fc79f242360 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 11/06/2025 01:43:45.034 7fc79f465680 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 11/06/2025 01:43:45.035 7fc79f465680 [contrail-config-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 11/06/2025 01:43:52.284 7fc79f242360 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/06/2025 01:43:54.785 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/06/2025 01:44:05.248 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1689-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12960408, partition_space_available_1k=68130076, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1324052, cpu_share=0.08, mem_res=1306684, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=89952, cpu_share=0.04, mem_res=84832, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53552, cpu_share=0.01, mem_res=42732, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7340916, free=271488, buffers=376764, cached=8533004, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.3, five_min_avg=3.08, fifteen_min_avg=1.46, cpu_share=13.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/06/2025 01:44:46.641 7fc79f4650e0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/06/2025 01:44:50.436 7fc79f4650e0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/06/2025 01:44:54.124 7fc79f4650e0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/06/2025 01:44:54.795 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/06/2025 01:45:00.441 7fc79f242a40 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/06/2025 01:45:03.236 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1689-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13083184, partition_space_available_1k=68007300, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1324040, cpu_share=0.07, mem_res=1306628, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=92480, cpu_share=0.01, mem_res=87316, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53716, cpu_share=0.01, mem_res=42596, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7395296, free=392320, buffers=378780, cached=8357056, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.04, five_min_avg=3.12, fifteen_min_avg=1.58, cpu_share=12.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/06/2025 01:45:11.124 7fc79f161b80 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/06/2025 01:45:54.736 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/06/2025 01:46:03.242 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1689-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12994164, partition_space_available_1k=68096320, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1351452, cpu_share=0.07, mem_res=1333940, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=99512, cpu_share=0.0, mem_res=94332, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53936, cpu_share=0.01, mem_res=42700, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7470180, free=391732, buffers=379600, cached=8282048, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.97, five_min_avg=3.01, fifteen_min_avg=1.64, cpu_share=9.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/06/2025 01:46:19.412 7fc79f161ea0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/06/2025 01:46:24.475 7fc79f161ea0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/06/2025 01:46:30.379 7fc79f242b80 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/06/2025 01:46:34.658 7fc79f242b80 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/06/2025 01:46:39.593 7fc79f242b80 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/06/2025 01:46:45.889 7fc79f161d60 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/06/2025 01:46:49.499 7fc79f161d60 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/06/2025 01:46:53.639 7fc79f161d60 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/06/2025 01:46:54.732 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/06/2025 01:47:03.244 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1689-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13026960, partition_space_available_1k=68063524, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1362168, cpu_share=0.23, mem_res=1343716, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=92772, cpu_share=0.0, mem_res=87504, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53936, cpu_share=0.01, mem_res=42868, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7689644, free=252904, buffers=375992, cached=8205684, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.65, five_min_avg=4.0, fifteen_min_avg=2.07, cpu_share=16.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/06/2025 01:47:54.776 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/06/2025 01:48:03.250 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1689-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13034612, partition_space_available_1k=68055872, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1361824, cpu_share=0.07, mem_res=1343336, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=92804, cpu_share=0.0, mem_res=87504, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54144, cpu_share=0.01, mem_res=42880, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7685848, free=261380, buffers=375832, cached=8200424, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.23, five_min_avg=4.05, fifteen_min_avg=2.2, cpu_share=10.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/06/2025 01:48:54.794 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/06/2025 01:49:03.267 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1689-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13041880, partition_space_available_1k=68048604, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1351324, cpu_share=0.07, mem_res=1332700, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=92824, cpu_share=0.0, mem_res=87504, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54016, cpu_share=0.01, mem_res=43040, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7679064, free=257844, buffers=376024, cached=8210504, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.06, five_min_avg=3.62, fifteen_min_avg=2.17, cpu_share=8.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/06/2025 01:49:54.776 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/06/2025 01:50:03.269 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1689-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13048816, partition_space_available_1k=68041668, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1324928, cpu_share=0.07, mem_res=1306432, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=92832, cpu_share=0.0, mem_res=87504, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54056, cpu_share=0.01, mem_res=43080, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7630536, free=297556, buffers=376124, cached=8219252, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.17, five_min_avg=3.54, fifteen_min_avg=2.23, cpu_share=8.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/06/2025 01:50:20.010 7fc79f465680 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/06/2025 01:50:54.782 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/06/2025 01:51:03.288 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1689-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13057372, partition_space_available_1k=68033112, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1327604, cpu_share=0.08, mem_res=1308788, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=97788, cpu_share=0.01, mem_res=92500, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54020, cpu_share=0.01, mem_res=42844, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7651020, free=277032, buffers=375580, cached=8219864, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.85, five_min_avg=3.42, fifteen_min_avg=2.27, cpu_share=8.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/06/2025 01:51:54.809 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/06/2025 01:52:03.294 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1689-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13065428, partition_space_available_1k=68025056, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1323248, cpu_share=0.07, mem_res=1303988, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=116772, cpu_share=0.01, mem_res=111408, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54132, cpu_share=0.01, mem_res=42824, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7667048, free=250796, buffers=375732, cached=8229736, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.32, five_min_avg=3.18, fifteen_min_avg=2.26, cpu_share=8.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/06/2025 01:52:16.774 7fc79f4655e0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/06/2025 01:52:19.745 7fc79f4655e0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/06/2025 01:52:24.232 7fc79f4655e0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/06/2025 01:52:54.926 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/06/2025 01:53:03.326 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1689-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13079000, partition_space_available_1k=68011484, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1339244, cpu_share=0.08, mem_res=1319956, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=105732, cpu_share=0.0, mem_res=100284, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54108, cpu_share=0.01, mem_res=42872, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7688616, free=325248, buffers=374288, cached=8135696, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.81, five_min_avg=3.45, fifteen_min_avg=2.42, cpu_share=12.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/06/2025 01:53:54.813 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/06/2025 01:54:03.329 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1689-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13086476, partition_space_available_1k=68004008, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1339300, cpu_share=0.07, mem_res=1320116, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=106228, cpu_share=0.01, mem_res=100844, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53976, cpu_share=0.01, mem_res=42740, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7699180, free=303628, buffers=374356, cached=8146860, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.07, five_min_avg=3.33, fifteen_min_avg=2.44, cpu_share=8.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/06/2025 01:54:54.821 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/06/2025 01:55:03.353 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1689-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13093604, partition_space_available_1k=67996880, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1343656, cpu_share=0.07, mem_res=1324432, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=106644, cpu_share=0.0, mem_res=101240, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54032, cpu_share=0.01, mem_res=42468, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7727704, free=264240, buffers=374464, cached=8156952, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.47, five_min_avg=3.08, fifteen_min_avg=2.41, cpu_share=8.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/06/2025 01:55:54.828 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/06/2025 01:56:03.343 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1689-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13095500, partition_space_available_1k=67994984, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1343336, cpu_share=0.07, mem_res=1324168, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=107248, cpu_share=0.0, mem_res=101768, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54128, cpu_share=0.01, mem_res=42592, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7712784, free=275480, buffers=374484, cached=8159552, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.34, five_min_avg=2.91, fifteen_min_avg=2.39, cpu_share=7.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/06/2025 01:56:05.498 7fc79fea2540 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/06/2025 01:56:54.857 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/06/2025 01:57:04.352 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1689-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13857804, partition_space_available_1k=67232680, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1326156, cpu_share=0.07, mem_res=1306964, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=107592, cpu_share=0.0, mem_res=102296, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53912, cpu_share=0.01, mem_res=42664, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7728316, free=181556, buffers=365376, cached=8253728, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.44, five_min_avg=2.81, fifteen_min_avg=2.39, cpu_share=10.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/06/2025 01:57:14.189 7fc79fdfa040 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/06/2025 01:57:17.115 7fc79f242680 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/06/2025 01:57:23.315 7fc79f242680 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/06/2025 01:57:54.831 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/06/2025 01:58:03.381 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1689-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14372200, partition_space_available_1k=66718284, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1347872, cpu_share=0.13, mem_res=1328688, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=108900, cpu_share=0.0, mem_res=103484, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53952, cpu_share=0.01, mem_res=42724, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7929520, free=278500, buffers=400920, cached=7920936, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.7, five_min_avg=3.42, fifteen_min_avg=2.62, cpu_share=16.5, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/06/2025 01:58:54.869 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/06/2025 01:58:58.629 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-juju-k8s-1689-1', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1762394338629442, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-juju-k8s-1689-1', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=1, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0)], _table='ObjectConfigDatabaseInfo'), _seqnum=0) 11/06/2025 01:59:04.375 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1689-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14389960, partition_space_available_1k=66700524, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1347396, cpu_share=0.13, mem_res=1327688, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=111768, cpu_share=0.0, mem_res=106256, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53928, cpu_share=0.01, mem_res=42724, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7952672, free=235448, buffers=400980, cached=7941068, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.61, five_min_avg=3.28, fifteen_min_avg=2.62, cpu_share=10.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/06/2025 01:59:54.867 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/06/2025 01:59:57.680 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-config-database-nodemgr', _source='cn-jenkins-deploy-platform-juju-k8s-1689-1', _node_type='ConfigDatabase', _instance_id='0', _timestamp=1762394397680427, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-juju-k8s-1689-1', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0)], _table='ObjectConfigDatabaseInfo'), _seqnum=0) 11/06/2025 02:00:03.378 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1689-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14398720, partition_space_available_1k=66691764, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1341556, cpu_share=0.1, mem_res=1322112, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=112600, cpu_share=0.0, mem_res=107048, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53956, cpu_share=0.01, mem_res=42792, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7960544, free=262352, buffers=398788, cached=7908584, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.87, five_min_avg=3.12, fifteen_min_avg=2.61, cpu_share=8.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/06/2025 02:00:29.750 7fc79f465b80 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/06/2025 02:00:54.869 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/06/2025 02:01:03.392 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1689-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14401740, partition_space_available_1k=66688744, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1336976, cpu_share=0.07, mem_res=1317540, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=113060, cpu_share=0.0, mem_res=107576, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53920, cpu_share=0.01, mem_res=42668, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7992132, free=250764, buffers=399156, cached=7888012, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.16, five_min_avg=2.86, fifteen_min_avg=2.55, cpu_share=9.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/06/2025 02:01:54.917 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/06/2025 02:02:04.434 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1689-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14406732, partition_space_available_1k=66683752, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1338276, cpu_share=0.07, mem_res=1318268, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=113580, cpu_share=0.0, mem_res=108104, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54036, cpu_share=0.01, mem_res=42904, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7989968, free=277784, buffers=398456, cached=7864408, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.22, five_min_avg=2.94, fifteen_min_avg=2.59, cpu_share=11.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/06/2025 02:02:35.965 7fc79f465680 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/06/2025 02:02:38.598 7fc79f465680 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/06/2025 02:02:42.790 7fc79f465680 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/06/2025 02:02:54.894 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/06/2025 02:03:04.397 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1689-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14417368, partition_space_available_1k=66673116, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1338608, cpu_share=0.09, mem_res=1318460, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=114936, cpu_share=0.0, mem_res=109292, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54124, cpu_share=0.01, mem_res=42832, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7980820, free=267780, buffers=398672, cached=7883380, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.88, five_min_avg=2.87, fifteen_min_avg=2.58, cpu_share=10.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/06/2025 02:03:54.906 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/06/2025 02:04:03.445 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1689-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14748376, partition_space_available_1k=66342108, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1337340, cpu_share=0.09, mem_res=1317056, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=116716, cpu_share=0.0, mem_res=111008, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54060, cpu_share=0.01, mem_res=42728, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7940480, free=263608, buffers=399620, cached=7927668, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.81, five_min_avg=3.63, fifteen_min_avg=2.86, cpu_share=11.75, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/06/2025 02:04:31.626 7fc79f2427c0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/06/2025 02:04:54.969 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/06/2025 02:04:58.203 7fc79fe15cc0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/06/2025 02:05:05.424 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1689-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14767752, partition_space_available_1k=66322732, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1329020, cpu_share=0.08, mem_res=1308856, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=117088, cpu_share=0.0, mem_res=111668, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53992, cpu_share=0.01, mem_res=42772, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7984996, free=416704, buffers=398740, cached=7732064, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.87, five_min_avg=3.43, fifteen_min_avg=2.84, cpu_share=11.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/06/2025 02:05:28.038 7fc79f242540 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/06/2025 02:05:31.449 7fc79f242540 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/06/2025 02:05:55.048 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/06/2025 02:05:57.420 7fc79fdfa2c0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/06/2025 02:06:04.428 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1689-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14781084, partition_space_available_1k=66309400, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1329832, cpu_share=0.08, mem_res=1309612, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=117724, cpu_share=0.0, mem_res=112064, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=53960, cpu_share=0.01, mem_res=42748, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7910256, free=486184, buffers=399032, cached=7737252, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.25, five_min_avg=3.56, fifteen_min_avg=2.92, cpu_share=11.25, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/06/2025 02:06:54.927 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: dir for config is /var/lib/cassandra/data 11/06/2025 02:07:03.436 7fc7a20f1940 [contrail-config-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-k8s-1689-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14789112, partition_space_available_1k=66301372, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1338452, cpu_share=0.08, mem_res=1317440, _EventManager__key='cassandra'), 'zookeeper': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=118188, cpu_share=0.0, mem_res=112724, _EventManager__key='zookeeper'), 'contrail-config-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=54012, cpu_share=0.01, mem_res=42756, _EventManager__key='contrail-config-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16184208, used=7955928, free=415648, buffers=403396, cached=7757540, node_type='config-database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.85, five_min_avg=3.59, fifteen_min_avg=2.97, cpu_share=10.0, node_type='config-database-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigDatabaseInfo') 11/06/2025 02:07:40.828 7fc79fe15cc0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/06/2025 02:07:43.724 7fc79fe15cc0 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map 11/06/2025 02:07:53.130 7fc79fea2540 [contrail-config-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigDatabaseInfo:NodeStatusUVE] map