02/06/2025 00:50:54.397 7f932c0e6df0 [contrail-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 02/06/2025 00:50:54.397 7f932c0e6df0 [contrail-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 02/06/2025 00:50:54.397 7f932c0e6df0 [contrail-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 02/06/2025 00:50:54.403 7f93267ce648 [contrail-database-nodemgr] [ERROR]: Cannot write http_port 8103 to /tmp/contrail-database-nodemgr.0.http_port 02/06/2025 00:50:54.403 7f93267ce648 [contrail-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8103 02/06/2025 00:50:54.404 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 02/06/2025 00:50:54.405 7f93267cee48 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/06/2025 00:50:54.405 7f93267cee48 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/06/2025 00:50:54.405 7f93267cee48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 02/06/2025 00:50:54.405 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:54.405 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/06/2025 00:50:54.405 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:54.405 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:54.405 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1738803054405329 collector_name = collector_ip = collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 02/06/2025 00:50:54.405 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:54.479 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 12296267126395254931260686851182538102318512745894721643706265025502703869062, 'start': '1738803052000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-query-engine', 'group': 'contrail-query-engine', 'pid': 95843105703179667388544482977075118713506345829649114294630505547164516061787, 'start': '1738803052000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-database-nodemgr', 'group': 'contrail-database-nodemgr', 'pid': 89593704583773177814932759797770128151917371156964880898108095333209342109528, 'start': '1738803052000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 02/06/2025 00:50:54.587 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:54.587 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. build_info = {"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-03 01:16:02.341982", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7601-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 4 num_core_per_socket = 4 num_thread_per_core = 1 >> running_package_version = nightly.r24.1.centos-nightly_r24.1_centos.el7 installed_package_version = nightly.r24.1.centos-nightly_r24.1_centos.el7 >> 02/06/2025 00:50:54.588 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:54.588 7f932c0e6df0 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-2.', _node_type='Database', _instance_id='0', _timestamp=1738803054588260, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/06/2025 00:50:54.588 7f932c0e6df0 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-2.', _node_type='Database', _instance_id='0', _timestamp=1738803054588522, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738803052000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738803052000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738803052000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-03 01:16:02.341982", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7601-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/06/2025 00:50:54.590 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:54.590 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/06/2025 00:50:54.590 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:54.590 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:54.590 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1738803052000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-query-engine process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1738803052000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1738803052000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-03 01:16:02.341982", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7601-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} >> 02/06/2025 00:50:54.590 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:54.815 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 00:50:55.059 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:DatabaseUsage] map 02/06/2025 00:50:55.061 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:55.061 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. database_usage = [ << disk_space_used_1k = 6768144 disk_space_available_1k = 74322340 analytics_db_size_1k = 12 >>, ] >> 02/06/2025 00:50:55.061 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:55.467 7f932c0e6df0 [contrail-database-nodemgr] [ERROR]: Failed to connect to database by CQL: Result: 1 02/06/2025 00:50:55.467 7f932c0e6df0 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-2.', _node_type='Database', _instance_id='0', _timestamp=1738803055467292, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/06/2025 00:50:55.468 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:55.468 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/06/2025 00:50:55.468 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:57.691 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-2.', _node_type='Database', _instance_id='0', _timestamp=1738803057690971, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=1), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0)], _table='ObjectDatabaseInfo'), _seqnum=0) 02/06/2025 00:50:57.691 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 02/06/2025 00:50:57.693 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:57.694 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] CassandraStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. cassandra_compaction_task = << pending_compaction_tasks = 1 >> thread_pool_stats = [ << pool_name = ReadStage active = 0 pending = 0 all_time_blocked = 0 >>, << pool_name = CompactionExecutor active = 0 pending = 0 all_time_blocked = 0 >>, << pool_name = MutationStage active = 0 pending = 0 all_time_blocked = 0 >>, << pool_name = MemtableFlushWriter active = 0 pending = 0 all_time_blocked = 0 >>, ] >> 02/06/2025 00:50:57.694 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:58.405 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:50:58.407 7f93267cee48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:50:58.407 7f93267cee48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:50:58.407 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:50:58.407 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:50:58.407 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:50:58.407 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:50:58.407 7f93267cee48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:50:58.408 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:58.408 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:50:58.408 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:58.408 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:50:58.408 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1738803054405329 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:50:58.408 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:02.408 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:51:02.409 7f93267cee48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:51:02.409 7f93267cee48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:51:02.409 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:02.409 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:51:02.410 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:02.410 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:51:02.410 7f93267cee48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:51:02.410 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:02.410 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:51:02.410 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:02.410 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:02.411 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1738803054405329 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:51:02.411 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:02.786 7f932c0e6df0 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-2.', _node_type='Database', _instance_id='0', _timestamp=1738803062786348, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738803052000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738803052000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738803052000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-03 01:16:02.341982", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7601-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/06/2025 00:51:02.786 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=6769104, partition_space_available_1k=74321380, percentage_partition_space_used=8, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1257364, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=7500, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51472, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=4344732, free=5291968, buffers=240416, cached=6298516, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.3, five_min_avg=0.97, fifteen_min_avg=0.44, cpu_share=0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 00:51:02.787 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:02.787 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1738803052000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-query-engine process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1738803052000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1738803052000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-03 01:16:02.341982", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7601-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} >> 02/06/2025 00:51:02.787 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:02.787 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:02.787 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 6769104 partition_space_available_1k = 74321380 percentage_partition_space_used = 8 >> } all_core_file_list = [ ] process_mem_cpu_usage = { cassandra : << mem_virt = 1257364 cpu_share = 0 mem_res = 0 >>contrail-query-engine : << mem_virt = 7500 cpu_share = 0 mem_res = 0 >>contrail-database-nodemgr : << mem_virt = 51472 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175632 used = 4344732 free = 5291968 buffers = 240416 cached = 6298516 node_type = database-node >> system_cpu_usage = << one_min_avg = 1.3 five_min_avg = 0.97 fifteen_min_avg = 0.44 cpu_share = 0 node_type = database-node >> >> 02/06/2025 00:51:02.787 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:06.412 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:51:06.413 7f93267cee48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:51:06.413 7f93267cee48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:51:06.413 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:06.413 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:51:06.414 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:06.414 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:51:06.414 7f93267cee48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:51:06.414 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:06.414 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:51:06.414 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:06.414 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:06.414 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1738803054405329 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:51:06.414 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:10.414 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:51:10.415 7f93267cee48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:51:10.415 7f93267cee48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:51:10.415 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:10.415 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:51:10.415 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:10.415 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:51:10.416 7f93267cee48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:51:10.416 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:10.416 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:51:10.416 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:10.416 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:10.416 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1738803054405329 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:51:10.416 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:14.418 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:51:14.419 7f93267cee48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:51:14.420 7f93267cee48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:51:14.420 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:14.420 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:51:14.420 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:14.420 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:51:14.420 7f93267cee48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:51:14.420 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:14.420 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:51:14.420 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:14.421 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:14.421 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1738803054405329 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:51:14.421 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:18.421 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:51:18.422 7f93267cee48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:51:18.423 7f93267cee48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:51:18.423 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:18.423 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:51:18.423 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:18.423 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:51:18.423 7f93267cee48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:51:18.423 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:18.423 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:51:18.424 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:18.424 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:18.424 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1738803054405329 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:51:18.424 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:22.425 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:51:22.426 7f93267cee48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:51:22.427 7f93267cee48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:51:22.427 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:22.427 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:51:22.427 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:22.427 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:51:22.428 7f93267cee48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:51:22.428 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:22.428 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:51:22.428 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:22.428 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:22.428 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1738803054405329 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:51:22.428 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:26.427 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:51:26.428 7f93267cee48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:51:26.429 7f93267cee48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:51:26.429 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:26.429 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:51:26.429 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:26.429 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:51:26.429 7f93267cee48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:51:26.429 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:26.429 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:51:26.430 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:26.430 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:26.430 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1738803054405329 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:51:26.430 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:30.429 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:51:30.430 7f93267cee48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:51:30.430 7f93267cee48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:51:30.430 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:30.430 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:51:30.430 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:30.431 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:51:30.431 7f93267cee48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:51:30.431 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:30.431 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:51:30.431 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:30.431 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:30.431 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1738803054405329 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:51:30.431 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:34.433 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:51:34.434 7f93267cee48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:51:34.434 7f93267cee48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:51:34.434 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:34.434 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:51:34.434 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:34.435 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:51:34.435 7f93267cee48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:51:34.435 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:34.435 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:51:34.435 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:34.435 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:34.435 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1738803054405329 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:51:34.435 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:38.437 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:51:38.438 7f93267cee48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:51:38.439 7f93267cee48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:51:38.439 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:38.439 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:51:38.439 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:38.439 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:51:38.439 7f93267cee48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:51:38.439 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:38.439 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:51:38.440 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:38.440 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:38.440 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1738803054405329 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:51:38.440 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:42.440 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:51:42.442 7f93267cee48 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 00:51:42.442 7f93267cee48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:51:42.442 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:42.442 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:51:42.442 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:42.442 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 00:51:42.442 7f93267cee48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 00:51:42.443 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:42.443 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Cassandra state detected DOWN. Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 00:51:42.443 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:42.443 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:42.443 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1738803054405329 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:51:42.443 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:46.444 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:51:46.446 7f93267cee48 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 02/06/2025 00:51:46.446 7f93267cee48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:51:46.446 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:46.446 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:51:46.446 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:51:46.446 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 02/06/2025 00:51:46.447 7f93267cee48 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 02/06/2025 00:51:46.447 7f93267cee48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 02/06/2025 00:51:46.447 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 00:51:46.447 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 00:51:54.802 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 00:51:55.122 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 00:51:55.732 7f932c0e6df0 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-2.', _node_type='Database', _instance_id='0', _timestamp=1738803115732122, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.23:8086'], status='Initializing', description='Connect to ClientInit on EvTcpConnected')]), flag_infos=None, description='Collector:Collector[Connect to ClientInit on EvTcpConnected] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/06/2025 00:51:55.733 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 00:51:57.504 7f93268df348 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 02/06/2025 00:51:57.504 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit] 02/06/2025 00:51:57.504 7f93267cee48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle] 02/06/2025 00:51:57.505 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:57.505 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = ClientInit to Idle on EvTcpClose >>, ] description = Collector:Collector[ClientInit to Idle on EvTcpClose] connection down >>, ] >> 02/06/2025 00:51:57.505 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:57.505 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:57.505 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8103 start_time = 1738803054405329 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.23:8086, 10.0.0.41:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 00:51:57.505 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:58.058 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-2.', _node_type='Database', _instance_id='0', _timestamp=1738803118058246, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=0, pending=0, all_time_blocked=2), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0)], _table='ObjectDatabaseInfo'), _seqnum=0) 02/06/2025 00:51:58.060 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:51:58.062 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] CassandraStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. cassandra_compaction_task = << pending_compaction_tasks = 0 >> thread_pool_stats = [ << pool_name = ReadStage active = 0 pending = 0 all_time_blocked = 0 >>, << pool_name = CompactionExecutor active = 0 pending = 0 all_time_blocked = 0 >>, << pool_name = MutationStage active = 0 pending = 0 all_time_blocked = 0 >>, << pool_name = Native-Transport-Requests active = 0 pending = 0 all_time_blocked = 2 >>, << pool_name = MemtableFlushWriter active = 0 pending = 0 all_time_blocked = 0 >>, ] >> 02/06/2025 00:51:58.062 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 00:52:01.505 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 00:52:01.506 7f93267cee48 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 02/06/2025 00:52:01.506 7f93267cee48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 00:52:01.507 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:52:01.507 7f93267cee48 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 00:52:01.507 7f93267cee48 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 00:52:01.507 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 02/06/2025 00:52:01.507 7f93267cee48 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 02/06/2025 00:52:01.507 7f93267cee48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 02/06/2025 00:52:01.507 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 00:52:01.507 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 00:52:01.511 7f93268df348 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001017 02/06/2025 00:52:01.512 7f93268df348 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 02/06/2025 00:52:01.512 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 02/06/2025 00:52:01.513 7f93267cee48 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 02/06/2025 00:52:01.513 7f93267cee48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 02/06/2025 00:52:01.513 7f93267cee48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 02/06/2025 00:52:01.513 7f93267cee48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/06/2025 00:52:01.513 7f93267cee48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/06/2025 00:52:01.513 7f93267cee48 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 02/06/2025 00:52:01.514 7f9326794048 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/06/2025 00:52:01.515 7f9326794048 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/06/2025 00:52:03.798 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7080760, partition_space_available_1k=74009724, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1322760, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=9992, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51880, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=4595376, free=4696016, buffers=252764, cached=6631476, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.22, five_min_avg=1.02, fifteen_min_avg=0.5, cpu_share=4.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 00:52:09.901 7f93268df348 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001582 02/06/2025 00:52:09.902 7f93268df348 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 02/06/2025 00:52:09.902 7f93267cee48 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 02/06/2025 00:52:09.902 7f93267cee48 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 4 02/06/2025 00:52:09.902 7f93267cee48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 02/06/2025 00:52:09.902 7f93267cee48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 02/06/2025 00:52:09.902 7f93267cee48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/06/2025 00:52:09.902 7f93267cee48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/06/2025 00:52:09.903 7f9326794048 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/06/2025 00:52:09.903 7f9326794048 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/06/2025 00:52:54.820 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 00:53:03.831 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7241400, partition_space_available_1k=73849084, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1323020, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10128, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51936, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=5558312, free=3525552, buffers=263608, cached=6828160, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.18, five_min_avg=1.33, fifteen_min_avg=0.64, cpu_share=12.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 00:53:54.849 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 00:54:03.835 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7296664, partition_space_available_1k=73793820, percentage_partition_space_used=9, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1327528, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10248, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51940, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=5578108, free=3447272, buffers=265152, cached=6885100, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.81, five_min_avg=1.7, fifteen_min_avg=0.81, cpu_share=13.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 00:54:54.909 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 00:55:04.318 7f93267cee48 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/06/2025 00:55:04.831 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7706296, partition_space_available_1k=73384188, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1365128, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10288, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51964, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=5678540, free=2873328, buffers=272068, cached=7351696, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.57, five_min_avg=2.38, fifteen_min_avg=1.1, cpu_share=14.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 00:55:20.525 7f9326794048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/06/2025 00:55:54.894 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 00:56:03.842 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7765540, partition_space_available_1k=73324944, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1340476, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10508, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51980, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=5755708, free=2781184, buffers=273784, cached=7364956, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.84, five_min_avg=2.58, fifteen_min_avg=1.25, cpu_share=13.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 00:56:54.902 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 00:57:03.850 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7773092, partition_space_available_1k=73317392, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1346116, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10588, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52000, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=5824468, free=2699252, buffers=274920, cached=7376992, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.69, five_min_avg=2.78, fifteen_min_avg=1.41, cpu_share=13.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 00:57:54.903 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 00:58:03.859 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7777772, partition_space_available_1k=73312712, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1352460, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10612, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52008, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=5910104, free=2612736, buffers=275732, cached=7377060, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.5, five_min_avg=2.9, fifteen_min_avg=1.53, cpu_share=13.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 00:58:54.916 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 00:59:03.854 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7790212, partition_space_available_1k=73300272, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1361764, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10496, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52008, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=5964352, free=2543960, buffers=276276, cached=7391044, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.45, five_min_avg=2.97, fifteen_min_avg=1.64, cpu_share=12.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 00:59:54.887 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:00:03.866 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7799448, partition_space_available_1k=73291036, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1363376, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10512, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52024, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6042124, free=2455600, buffers=276680, cached=7401228, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.66, five_min_avg=3.08, fifteen_min_avg=1.76, cpu_share=13.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:00:54.903 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:01:03.893 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7806884, partition_space_available_1k=73283600, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1371400, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10552, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52036, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=5955484, free=2528700, buffers=277152, cached=7414296, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.07, five_min_avg=3.55, fifteen_min_avg=2.01, cpu_share=12.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:01:54.907 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:02:03.884 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7817120, partition_space_available_1k=73273364, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1377168, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10580, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52044, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6009548, free=2469040, buffers=277440, cached=7419604, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.16, five_min_avg=3.85, fifteen_min_avg=2.2, cpu_share=12.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:02:54.906 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:03:03.898 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7824848, partition_space_available_1k=73265636, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1380388, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10632, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52044, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6039828, free=2425068, buffers=277612, cached=7433124, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.82, five_min_avg=3.68, fifteen_min_avg=2.25, cpu_share=12.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:03:54.908 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:04:03.900 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7837004, partition_space_available_1k=73253480, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1388464, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10660, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52056, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6062408, free=2395160, buffers=277736, cached=7440328, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.67, five_min_avg=3.68, fifteen_min_avg=2.34, cpu_share=12.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:04:54.944 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:05:03.897 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7843068, partition_space_available_1k=73247416, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1390168, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10608, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52056, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6100016, free=2345636, buffers=277808, cached=7452172, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.56, five_min_avg=3.63, fifteen_min_avg=2.41, cpu_share=12.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:05:54.948 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:06:03.911 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7855336, partition_space_available_1k=73235148, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1404412, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10680, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52060, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6149780, free=2283260, buffers=277860, cached=7464732, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.58, five_min_avg=3.63, fifteen_min_avg=2.49, cpu_share=12.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:06:54.956 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:07:03.911 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7864448, partition_space_available_1k=73226036, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1407724, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10704, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52064, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6164872, free=2262740, buffers=277896, cached=7470124, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.72, five_min_avg=3.4, fifteen_min_avg=2.48, cpu_share=12.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:07:54.967 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:08:03.926 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7879220, partition_space_available_1k=73211264, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1413920, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10704, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52068, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6198048, free=2220016, buffers=277928, cached=7479640, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.99, five_min_avg=3.35, fifteen_min_avg=2.52, cpu_share=12.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:08:54.942 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:09:03.924 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7886476, partition_space_available_1k=73204008, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1411416, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10704, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52076, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6237864, free=2161584, buffers=277944, cached=7498240, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.43, five_min_avg=3.14, fifteen_min_avg=2.5, cpu_share=12.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:09:54.972 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:10:03.927 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7900700, partition_space_available_1k=73189784, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1417536, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10704, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52084, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6220828, free=2174156, buffers=277968, cached=7502680, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.83, five_min_avg=3.09, fifteen_min_avg=2.52, cpu_share=12.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:10:54.953 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:11:03.939 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7909480, partition_space_available_1k=73181004, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1425496, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10648, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52072, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6247896, free=2132948, buffers=277980, cached=7516808, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.19, five_min_avg=3.18, fifteen_min_avg=2.6, cpu_share=12.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:11:54.964 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:12:03.934 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7923624, partition_space_available_1k=73166860, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1426976, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10656, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52084, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6241368, free=2128968, buffers=277992, cached=7527304, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.07, five_min_avg=3.14, fifteen_min_avg=2.62, cpu_share=12.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:12:55.009 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:13:03.955 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7929740, partition_space_available_1k=73160744, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1435260, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10684, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52084, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6264404, free=2089312, buffers=277992, cached=7543924, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.92, five_min_avg=3.08, fifteen_min_avg=2.63, cpu_share=12.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:13:54.979 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:14:03.940 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7919228, partition_space_available_1k=73171256, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1435504, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10760, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52084, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6269980, free=2094272, buffers=278004, cached=7533376, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.47, five_min_avg=3.18, fifteen_min_avg=2.69, cpu_share=12.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:14:54.972 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:15:03.938 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7932752, partition_space_available_1k=73157732, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1443628, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10804, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52092, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6278412, free=2076912, buffers=278008, cached=7542300, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.12, five_min_avg=3.14, fifteen_min_avg=2.71, cpu_share=12.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:15:55.004 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:16:03.966 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7936996, partition_space_available_1k=73153488, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1450376, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10812, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52092, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6275220, free=2071792, buffers=278008, cached=7550612, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.85, five_min_avg=3.07, fifteen_min_avg=2.72, cpu_share=12.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:16:54.990 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:17:03.935 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7950148, partition_space_available_1k=73140336, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1458684, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10736, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52092, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6285904, free=2056548, buffers=278008, cached=7555172, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.5, five_min_avg=3.21, fifteen_min_avg=2.79, cpu_share=12.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:17:55.010 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:18:03.953 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7955836, partition_space_available_1k=73134648, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1456628, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10760, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52100, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6287256, free=2039588, buffers=278008, cached=7570780, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.05, five_min_avg=3.11, fifteen_min_avg=2.78, cpu_share=12.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:18:54.983 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:19:03.933 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7960264, partition_space_available_1k=73130220, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1462328, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10788, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52096, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6292124, free=2030084, buffers=278016, cached=7575408, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.9, five_min_avg=3.05, fifteen_min_avg=2.78, cpu_share=12.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:19:54.980 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:20:03.940 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7973328, partition_space_available_1k=73117156, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1467472, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10796, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52100, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6307744, free=2010916, buffers=278020, cached=7578952, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.2, five_min_avg=3.1, fifteen_min_avg=2.82, cpu_share=12.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:20:54.984 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:21:03.964 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7980948, partition_space_available_1k=73109536, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1479372, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10804, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52108, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6328164, free=1978616, buffers=278020, cached=7590832, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.6, five_min_avg=3.22, fifteen_min_avg=2.88, cpu_share=12.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:21:54.998 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:22:03.967 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7990868, partition_space_available_1k=73099616, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1477180, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10816, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52124, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6306828, free=1994200, buffers=278020, cached=7596584, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.56, five_min_avg=3.28, fifteen_min_avg=2.92, cpu_share=12.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:22:55.006 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:23:03.995 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=7994268, partition_space_available_1k=73096216, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1484108, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10832, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52136, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6317296, free=1969788, buffers=278020, cached=7610528, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.57, five_min_avg=3.33, fifteen_min_avg=2.96, cpu_share=12.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:23:55.035 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:24:03.987 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8002772, partition_space_available_1k=73087712, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1488584, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10864, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52140, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6331732, free=1951468, buffers=278020, cached=7614412, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.86, five_min_avg=3.18, fifteen_min_avg=2.93, cpu_share=12.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:24:55.009 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:25:03.996 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8013020, partition_space_available_1k=73077464, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1494104, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10888, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52352, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6331428, free=1946276, buffers=278024, cached=7619904, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.23, five_min_avg=3.23, fifteen_min_avg=2.97, cpu_share=12.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:25:55.007 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:26:04.012 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8021776, partition_space_available_1k=73068708, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1498976, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10892, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52360, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6337376, free=1926852, buffers=278024, cached=7633380, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.32, five_min_avg=3.48, fifteen_min_avg=3.07, cpu_share=12.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:26:55.040 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:27:04.025 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8025104, partition_space_available_1k=73065380, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1504496, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10892, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52360, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6345692, free=1909468, buffers=278028, cached=7642444, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.29, five_min_avg=3.33, fifteen_min_avg=3.04, cpu_share=12.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:27:55.059 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:28:04.039 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8034184, partition_space_available_1k=73056300, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1509648, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10896, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52364, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6342588, free=1908368, buffers=278028, cached=7646648, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.22, five_min_avg=3.56, fifteen_min_avg=3.13, cpu_share=12.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:28:55.079 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:29:04.049 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8048856, partition_space_available_1k=73041628, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1515236, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10764, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52364, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6356188, free=1885380, buffers=278028, cached=7656036, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.81, five_min_avg=3.53, fifteen_min_avg=3.15, cpu_share=12.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:29:55.053 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:30:04.049 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8053556, partition_space_available_1k=73036928, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1519204, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10776, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52364, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6359012, free=1876844, buffers=278028, cached=7661748, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.56, five_min_avg=3.52, fifteen_min_avg=3.17, cpu_share=12.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:30:55.173 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:30:58.794 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-2.', _node_type='Database', _instance_id='0', _timestamp=1738805458793982, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=0, pending=0, all_time_blocked=2)], _table='ObjectDatabaseInfo'), _seqnum=0) 02/06/2025 01:31:04.054 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8070120, partition_space_available_1k=73020364, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1502256, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10824, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52372, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6317648, free=1897044, buffers=278036, cached=7682904, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.67, five_min_avg=3.54, fifteen_min_avg=3.2, cpu_share=12.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:31:55.073 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:32:04.070 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8081816, partition_space_available_1k=73008668, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1506264, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10840, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52372, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6334388, free=1872992, buffers=278036, cached=7690216, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.51, five_min_avg=3.52, fifteen_min_avg=3.21, cpu_share=12.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:32:55.088 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:33:04.079 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8086220, partition_space_available_1k=73004264, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1509560, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10844, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52376, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6329648, free=1863052, buffers=278036, cached=7704896, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.92, five_min_avg=3.36, fifteen_min_avg=3.17, cpu_share=12.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:33:55.108 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:34:04.092 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8099864, partition_space_available_1k=72990620, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1513992, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10868, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52384, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6332692, free=1856324, buffers=278036, cached=7708580, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.09, five_min_avg=3.31, fifteen_min_avg=3.17, cpu_share=12.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:34:55.093 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:35:04.092 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8103168, partition_space_available_1k=72987316, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1517248, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10816, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52384, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6336824, free=1847928, buffers=278036, cached=7712844, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.32, five_min_avg=3.34, fifteen_min_avg=3.19, cpu_share=12.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:35:55.114 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:36:04.105 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8111604, partition_space_available_1k=72978880, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1522856, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10892, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52388, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6334984, free=1836800, buffers=278036, cached=7725812, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.48, five_min_avg=3.4, fifteen_min_avg=3.22, cpu_share=12.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:36:55.127 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:37:04.109 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8120256, partition_space_available_1k=72970228, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1528748, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10912, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52392, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6330148, free=1833140, buffers=278036, cached=7734308, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.86, five_min_avg=3.51, fifteen_min_avg=3.27, cpu_share=12.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:37:55.122 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:38:04.121 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8129764, partition_space_available_1k=72960720, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1531924, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10928, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52396, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6334144, free=1818568, buffers=278040, cached=7744880, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.31, five_min_avg=3.41, fifteen_min_avg=3.25, cpu_share=12.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:38:55.143 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:39:04.118 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8137848, partition_space_available_1k=72952636, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1537692, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10956, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52396, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6348316, free=1796232, buffers=278044, cached=7753040, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.47, five_min_avg=3.41, fifteen_min_avg=3.26, cpu_share=12.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:39:55.141 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:40:04.131 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8141504, partition_space_available_1k=72948980, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1543172, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10964, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52400, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6357212, free=1779300, buffers=278048, cached=7761072, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.39, five_min_avg=3.42, fifteen_min_avg=3.27, cpu_share=12.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:40:55.167 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:41:04.145 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8145904, partition_space_available_1k=72944580, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1547316, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10808, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52404, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6351876, free=1780128, buffers=278048, cached=7765580, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.1, five_min_avg=3.33, fifteen_min_avg=3.25, cpu_share=12.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:41:55.154 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:42:04.144 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8156916, partition_space_available_1k=72933568, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1553692, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10844, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52408, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6362200, free=1757476, buffers=278048, cached=7777908, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.71, five_min_avg=3.47, fifteen_min_avg=3.3, cpu_share=12.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:42:55.179 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:43:04.154 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8167360, partition_space_available_1k=72923124, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1558772, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10900, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52408, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6366624, free=1747344, buffers=278052, cached=7783612, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.45, five_min_avg=3.43, fifteen_min_avg=3.3, cpu_share=12.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:43:55.180 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:44:04.138 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8176072, partition_space_available_1k=72914412, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1564624, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10928, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52420, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6367608, free=1737636, buffers=278056, cached=7792332, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.79, five_min_avg=3.24, fifteen_min_avg=3.24, cpu_share=12.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:44:55.171 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:45:04.152 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8178312, partition_space_available_1k=72912172, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1570352, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10928, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52420, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6374240, free=1718272, buffers=278056, cached=7805064, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.97, five_min_avg=3.2, fifteen_min_avg=3.22, cpu_share=12.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:45:55.179 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:46:04.143 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8192036, partition_space_available_1k=72898448, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1573224, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10944, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52420, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6375676, free=1718408, buffers=278056, cached=7803492, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.25, five_min_avg=3.25, fifteen_min_avg=3.24, cpu_share=12.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:46:55.182 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:47:04.149 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8195936, partition_space_available_1k=72894548, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1577276, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10852, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52424, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6368384, free=1711788, buffers=278056, cached=7817404, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.12, five_min_avg=3.21, fifteen_min_avg=3.23, cpu_share=12.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:47:55.199 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:48:04.156 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8206028, partition_space_available_1k=72884456, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1580880, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10904, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52428, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6344068, free=1730452, buffers=278056, cached=7823056, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.79, five_min_avg=3.32, fifteen_min_avg=3.26, cpu_share=12.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:48:55.213 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:49:04.153 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8216540, partition_space_available_1k=72873944, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1586332, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10904, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52436, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6289088, free=1768436, buffers=278060, cached=7840048, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.29, five_min_avg=3.56, fifteen_min_avg=3.35, cpu_share=12.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:49:55.201 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:50:04.160 7f932c0e6df0 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8229280, partition_space_available_1k=72861204, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1532716, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10912, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=52436, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6280024, free=1765032, buffers=278064, cached=7852512, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.15, five_min_avg=3.68, fifteen_min_avg=3.41, cpu_share=12.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:50:58.300 7fbc9ae6af20 [contrail-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 02/06/2025 01:50:58.300 7fbc9ae6af20 [contrail-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 02/06/2025 01:50:58.300 7fbc9ae6af20 [contrail-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 02/06/2025 01:50:58.304 7fbc9528a748 [contrail-database-nodemgr] [ERROR]: Cannot write http_port 8103 to /tmp/contrail-database-nodemgr.0.http_port 02/06/2025 01:50:58.317 7fbc9528a748 [contrail-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8103 02/06/2025 01:50:58.317 7fbc940b4048 [contrail-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 02/06/2025 01:50:58.318 7fbc940b4048 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/06/2025 01:50:58.318 7fbc940b4048 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/06/2025 01:50:58.318 7fbc940b4048 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 02/06/2025 01:50:58.319 7fbc940b4048 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:50:58.319 7fbc940b4048 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/06/2025 01:50:58.319 7fbc940b4048 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:50:58.319 7fbc940b4048 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:50:58.319 7fbc940b4048 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1738806658318718 collector_name = collector_ip = collector_list = [ 10.0.0.41:8086, 10.0.0.23:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 02/06/2025 01:50:58.319 7fbc940b4048 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:50:58.690 7fbc9ae6af20 [contrail-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 12296267126395254931260686851182538102318512745894721643706265025502703869062, 'start': '1738806653000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-query-engine', 'group': 'contrail-query-engine', 'pid': 95843105703179667388544482977075118713506345829649114294630505547164516061787, 'start': '1738806652000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-database-nodemgr', 'group': 'contrail-database-nodemgr', 'pid': 89593704583773177814932759797770128151917371156964880898108095333209342109528, 'start': '1738806653000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 02/06/2025 01:50:59.049 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:50:59.053 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. build_info = {"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-03 01:16:02.341982", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7601-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 4 num_core_per_socket = 4 num_thread_per_core = 1 >> running_package_version = nightly.r24.1.centos-nightly_r24.1_centos.el7 installed_package_version = nightly.r24.1.centos-nightly_r24.1_centos.el7 >> 02/06/2025 01:50:59.053 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:50:59.057 7fbc9ae6af20 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-2.', _node_type='Database', _instance_id='0', _timestamp=1738806659057453, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/06/2025 01:50:59.057 7fbc9ae6af20 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-2.', _node_type='Database', _instance_id='0', _timestamp=1738806659057840, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738806653000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738806652000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738806653000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-03 01:16:02.341982", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7601-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/06/2025 01:50:59.074 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:50:59.075 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/06/2025 01:50:59.075 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:50:59.075 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:50:59.076 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1738806653000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-query-engine process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1738806652000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1738806653000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-03 01:16:02.341982", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7601-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]} >> 02/06/2025 01:50:59.076 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:51:00.192 7fbc9ae6af20 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:51:00.912 7fbc9ae6af20 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:DatabaseUsage] map 02/06/2025 01:51:00.914 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:51:00.914 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. database_usage = [ << disk_space_used_1k = 8083904 disk_space_available_1k = 73006580 analytics_db_size_1k = 30404 >>, ] >> 02/06/2025 01:51:00.914 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:51:02.319 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:51:02.320 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 02/06/2025 01:51:02.320 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:51:02.320 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:51:02.320 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:51:02.320 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:51:02.320 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 02/06/2025 01:51:02.320 7fbc940b4448 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 02/06/2025 01:51:02.321 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 02/06/2025 01:51:02.321 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 01:51:02.321 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 01:51:02.915 7fbc9ae6af20 [contrail-database-nodemgr] [ERROR]: Failed to connect to database by CQL: Result: 1 02/06/2025 01:51:02.915 7fbc9ae6af20 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-2.', _node_type='Database', _instance_id='0', _timestamp=1738806662915765, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.41:8086'], status='Initializing', description='Connect to ClientInit on EvTcpConnected')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[Connect to ClientInit on EvTcpConnected] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/06/2025 01:51:02.917 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 01:51:06.153 7fbc9ae6af20 [contrail-database-nodemgr] [ERROR]: Failed to get nodetool compactionstats: Result: 2 02/06/2025 01:51:06.658 7fbc94073448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 02/06/2025 01:51:06.658 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit] 02/06/2025 01:51:06.659 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle] 02/06/2025 01:51:06.659 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:51:06.659 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = ClientInit to Idle on EvTcpClose >>, ] description = Cassandra state detected DOWN. Collector:Collector[ClientInit to Idle on EvTcpClose] connection down >>, ] >> 02/06/2025 01:51:06.660 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:51:06.660 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:51:06.660 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8103 start_time = 1738806658318718 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.23:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 02/06/2025 01:51:06.660 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:51:10.660 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:51:10.662 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 02/06/2025 01:51:10.662 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:51:10.662 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:51:10.662 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:51:10.663 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:51:10.663 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 02/06/2025 01:51:10.663 7fbc940b4448 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 02/06/2025 01:51:10.663 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 02/06/2025 01:51:10.663 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 01:51:10.663 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 01:51:10.669 7fbc94073448 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001017 02/06/2025 01:51:10.670 7fbc94073448 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 02/06/2025 01:51:10.670 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 02/06/2025 01:51:10.671 7fbc940b4448 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 02/06/2025 01:51:10.671 7fbc940b4448 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 02/06/2025 01:51:10.671 7fbc940b4448 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/06/2025 01:51:10.671 7fbc940b4448 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/06/2025 01:51:10.671 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 02/06/2025 01:51:10.671 7fbc940b4148 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/06/2025 01:51:10.672 7fbc940b4148 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/06/2025 01:51:11.359 7fbc9ae6af20 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-2.', _node_type='Database', _instance_id='0', _timestamp=1738806671359272, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738806653000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738806652000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738806653000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly.r24.1.centos", "build-time" : "2025-02-03 01:16:02.341982", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7601-1.novalocal", "build-id" : "nightly.r24.1.centos", "build-number" : "nightly_r24.1_centos.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/06/2025 01:51:11.359 7fbc9ae6af20 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=8085472, partition_space_available_1k=73005012, percentage_partition_space_used=10, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1205888, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=8740, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44296, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=4600664, free=3400476, buffers=281332, cached=7893160, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=10.1, five_min_avg=5.07, fifteen_min_avg=3.88, cpu_share=0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:51:59.360 7fbc9ae6af20 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:52:00.363 7fbc9ae6af20 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-2.', _node_type='Database', _instance_id='0', _timestamp=1738806720362974, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.23:8086'], status='Up', description='ClientInit to Established on EvSandeshCtrlMessageRecv')]), flag_infos=None, description='')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/06/2025 01:52:02.914 7fbc9ae6af20 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-2.', _node_type='Database', _instance_id='0', _timestamp=1738806722914232, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0)], _table='ObjectDatabaseInfo'), _seqnum=0) 02/06/2025 01:52:02.914 7fbc9ae6af20 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 02/06/2025 01:52:08.132 7fbc9ae6af20 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=9377468, partition_space_available_1k=71713016, percentage_partition_space_used=12, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1334152, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=9848, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44372, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=5296052, free=1450560, buffers=353224, cached=9075796, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=8.99, five_min_avg=5.81, fifteen_min_avg=4.21, cpu_share=18.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:52:59.370 7fbc9ae6af20 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:53:08.138 7fbc9ae6af20 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=10452264, partition_space_available_1k=70638220, percentage_partition_space_used=13, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1346812, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=9924, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44544, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=5680356, free=164980, buffers=380804, cached=9949492, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.07, five_min_avg=5.46, fifteen_min_avg=4.18, cpu_share=11.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:53:29.967 7fbc94073448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 02/06/2025 01:53:29.968 7fbc940b4148 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Established] 02/06/2025 01:53:29.969 7fbc940b4148 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 02/06/2025 01:53:29.969 7fbc940b4148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[Established] -> State[Connect] 02/06/2025 01:53:29.969 7fbc940b4148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:29.969 7fbc940b4148 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Established to Connect on EvTcpClose >>, ] description = Collector:Collector[Established to Connect on EvTcpClose] connection down >>, ] >> 02/06/2025 01:53:29.969 7fbc940b4148 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:29.969 7fbc940b4148 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 02/06/2025 01:53:29.969 7fbc940b4148 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 02/06/2025 01:53:29.970 7fbc940b4148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 02/06/2025 01:53:29.970 7fbc940b4148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 01:53:29.970 7fbc940b4148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 01:53:30.038 7fbc94073448 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001017 02/06/2025 01:53:30.039 7fbc94073448 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 02/06/2025 01:53:30.039 7fbc940b4148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 02/06/2025 01:53:30.039 7fbc940b4148 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 02/06/2025 01:53:30.039 7fbc940b4148 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 02/06/2025 01:53:30.040 7fbc940b4148 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 02/06/2025 01:53:30.040 7fbc940b4148 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/06/2025 01:53:30.040 7fbc940b4148 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/06/2025 01:53:30.040 7fbc940b4148 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 02/06/2025 01:53:30.040 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/06/2025 01:53:30.042 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/06/2025 01:53:30.141 7fbc94073448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 02/06/2025 01:53:30.141 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Established] 02/06/2025 01:53:30.142 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 02/06/2025 01:53:30.142 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[Established] -> State[Connect] 02/06/2025 01:53:30.142 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:30.142 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Established to Connect on EvTcpClose >>, ] description = Collector:Collector[Established to Connect on EvTcpClose] connection down >>, ] >> 02/06/2025 01:53:30.142 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:30.142 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 02/06/2025 01:53:30.143 7fbc940b4448 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 02/06/2025 01:53:30.143 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 02/06/2025 01:53:30.143 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 01:53:30.143 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 01:53:30.175 7fbc94073448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 02/06/2025 01:53:30.176 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit] 02/06/2025 01:53:30.176 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle] 02/06/2025 01:53:30.177 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:30.177 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = ClientInit to Idle on EvTcpClose >>, ] description = Collector:Collector[ClientInit to Idle on EvTcpClose] connection down >>, ] >> 02/06/2025 01:53:30.177 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:30.177 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:30.177 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 4 pid = 1 http_port = 8103 start_time = 1738806658318718 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.23:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:53:30.177 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:34.177 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:53:34.178 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 02/06/2025 01:53:34.178 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:53:34.178 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:34.178 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:53:34.179 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:34.179 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 02/06/2025 01:53:34.179 7fbc940b4448 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 02/06/2025 01:53:34.179 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 02/06/2025 01:53:34.179 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 01:53:34.179 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 01:53:42.796 7fbc94073448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connection Closed 02/06/2025 01:53:42.797 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpClose] in state[ClientInit] 02/06/2025 01:53:42.797 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[ClientInit] -> State[Idle] 02/06/2025 01:53:42.797 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:42.797 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = ClientInit to Idle on EvTcpClose >>, ] description = Collector:Collector[ClientInit to Idle on EvTcpClose] connection down >>, ] >> 02/06/2025 01:53:42.797 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:42.797 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:42.797 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1738806658318718 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.23:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:53:42.798 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:46.797 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:53:46.799 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:53:46.799 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:53:46.799 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:46.799 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:53:46.799 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:46.799 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:53:46.799 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:53:46.799 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:46.800 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:53:46.800 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:46.800 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:46.800 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1738806658318718 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.23:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:53:46.800 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:50.802 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:53:50.803 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:53:50.803 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:53:50.804 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:50.804 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:53:50.804 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:50.804 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:53:50.804 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:53:50.804 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:50.804 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:53:50.804 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:50.804 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:50.805 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1738806658318718 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.23:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:53:50.805 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:54.803 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:53:54.805 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:53:54.805 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:53:54.805 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:54.805 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:53:54.805 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:54.805 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:53:54.806 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:53:54.806 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:54.806 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:53:54.806 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:54.806 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:54.806 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1738806658318718 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.23:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:53:54.806 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:58.808 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:53:58.809 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:53:58.810 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:53:58.810 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:58.810 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:53:58.810 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:53:58.810 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:53:58.810 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:53:58.810 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:58.810 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:53:58.810 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:58.811 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:58.811 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1738806658318718 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.23:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:53:58.811 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:59.256 7fbc9ae6af20 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:53:59.476 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:53:59.477 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. database_usage = [ << disk_space_used_1k = 10402684 disk_space_available_1k = 70687800 analytics_db_size_1k = 30480 >>, ] >> 02/06/2025 01:53:59.477 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:02.811 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:02.812 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:02.812 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:02.812 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:02.812 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:02.812 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:02.813 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:02.813 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:02.813 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:02.813 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:02.813 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:02.813 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:02.813 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1738806658318718 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.23:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:54:02.813 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:06.760 7fbc9ae6af20 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=10402688, partition_space_available_1k=70687796, percentage_partition_space_used=13, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1336352, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10756, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44600, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=4108216, free=1871300, buffers=385684, cached=9810432, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.93, five_min_avg=4.67, fifteen_min_avg=3.99, cpu_share=6.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:54:06.760 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:06.761 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 10402688 partition_space_available_1k = 70687796 percentage_partition_space_used = 13 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1336352 cpu_share = 0 mem_res = 0 >>contrail-query-engine : << mem_virt = 10756 cpu_share = 0 mem_res = 0 >>contrail-database-nodemgr : << mem_virt = 44600 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175632 used = 4108216 free = 1871300 buffers = 385684 cached = 9810432 node_type = database-node >> system_cpu_usage = << one_min_avg = 2.93 five_min_avg = 4.67 fifteen_min_avg = 3.99 cpu_share = 6.25 node_type = database-node >> >> 02/06/2025 01:54:06.761 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:06.813 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:06.814 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:06.814 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:06.814 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:06.815 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:06.815 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:06.815 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:06.815 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:06.815 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:06.815 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:06.815 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:06.815 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:06.815 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1738806658318718 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.23:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:54:06.815 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:10.814 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:10.816 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:10.816 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:10.816 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:10.816 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:10.816 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:10.816 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:10.816 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:10.817 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:10.817 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:10.817 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:10.817 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:10.817 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1738806658318718 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.23:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:54:10.817 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:14.818 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:14.819 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:14.819 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:14.819 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:14.819 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:14.819 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:14.820 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:14.820 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:14.820 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:14.820 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:14.820 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:14.820 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:14.820 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1738806658318718 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.23:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:54:14.820 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:18.821 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:18.823 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:18.823 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:18.823 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:18.823 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:18.823 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:18.823 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:18.824 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:18.824 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:18.824 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:18.824 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:18.824 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:18.824 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1738806658318718 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.23:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:54:18.824 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:22.825 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:22.826 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:22.826 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:22.826 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:22.826 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:22.826 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:22.827 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:22.827 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:22.827 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:22.827 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:22.827 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:22.827 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:22.827 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1738806658318718 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.23:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:54:22.827 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:26.829 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:26.830 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:26.830 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:26.830 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:26.830 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:26.830 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:26.830 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:26.831 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:26.831 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:26.831 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:26.831 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:26.831 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:26.831 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1738806658318718 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.23:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:54:26.831 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:30.833 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:30.835 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:30.835 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:30.835 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:30.835 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:30.835 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:30.835 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:30.835 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:30.836 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:30.836 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:30.836 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:30.836 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:30.836 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1738806658318718 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.23:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:54:30.836 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:34.837 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:34.838 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:34.838 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:34.838 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:34.839 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:34.839 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:34.839 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:34.839 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:34.839 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:34.839 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:34.839 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:34.839 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:34.839 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1738806658318718 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.23:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:54:34.840 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:38.841 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:38.842 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:38.842 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:38.842 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:38.843 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:38.843 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:38.843 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:38.843 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:38.843 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:38.843 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:38.843 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:38.843 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:38.844 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1738806658318718 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.23:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:54:38.844 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:42.845 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:42.846 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:42.846 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:42.846 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:42.847 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:42.847 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:42.847 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:42.847 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:42.847 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:42.847 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:42.847 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:42.847 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:42.848 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1738806658318718 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.23:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:54:42.848 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:46.847 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:46.849 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:46.849 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:46.849 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:46.849 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:46.849 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:46.849 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:46.850 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:46.850 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:46.850 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:46.850 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:46.850 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:46.850 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1738806658318718 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.23:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:54:46.850 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:50.851 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:50.852 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:50.852 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:50.852 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:50.852 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:50.852 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:50.853 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:50.853 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:50.853 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:50.853 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:50.853 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:50.853 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:50.853 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1738806658318718 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.23:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:54:50.853 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:54.853 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:54.854 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:54.854 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:54.854 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:54.854 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:54.855 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:54.855 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:54.855 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:54.855 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:54.855 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:54.856 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:54.856 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:54.856 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1738806658318718 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.23:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:54:54.856 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:58.857 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:54:58.859 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:54:58.859 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:54:58.859 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:58.860 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:54:58.860 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:54:58.860 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:54:58.861 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:54:58.861 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:58.861 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:54:58.861 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:58.861 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:58.862 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1738806658318718 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.23:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:54:58.862 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:59.255 7fbc9ae6af20 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:54:59.467 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:54:59.467 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. database_usage = [ << disk_space_used_1k = 10428004 disk_space_available_1k = 70662480 analytics_db_size_1k = 30480 >>, ] >> 02/06/2025 01:54:59.467 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:02.860 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:55:02.861 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:55:02.862 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:55:02.862 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:02.862 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:55:02.863 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:02.863 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:55:02.864 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:55:02.864 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:02.864 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:55:02.864 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:02.864 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:02.864 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1738806658318718 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.23:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:55:02.865 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:06.862 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:55:06.864 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:55:06.864 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:55:06.864 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:06.864 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:55:06.864 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:06.864 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:55:06.864 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:55:06.865 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:06.865 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:55:06.865 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:06.865 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:06.866 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1738806658318718 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.23:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:55:06.866 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:07.523 7fbc9ae6af20 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=10429932, partition_space_available_1k=70660552, percentage_partition_space_used=13, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1351532, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10992, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44692, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=4709692, free=1204532, buffers=390940, cached=9870468, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.43, five_min_avg=4.19, fifteen_min_avg=3.87, cpu_share=6.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:55:07.523 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:07.523 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 10429932 partition_space_available_1k = 70660552 percentage_partition_space_used = 13 >> } process_mem_cpu_usage = { cassandra : << mem_virt = 1351532 cpu_share = 0 mem_res = 0 >>contrail-query-engine : << mem_virt = 10992 cpu_share = 0 mem_res = 0 >>contrail-database-nodemgr : << mem_virt = 44692 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 16175632 used = 4709692 free = 1204532 buffers = 390940 cached = 9870468 node_type = database-node >> system_cpu_usage = << one_min_avg = 2.43 five_min_avg = 4.19 fifteen_min_avg = 3.87 cpu_share = 6.75 node_type = database-node >> >> 02/06/2025 01:55:07.523 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:10.865 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:55:10.866 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:55:10.866 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:55:10.866 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:10.866 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:55:10.866 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:10.867 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:55:10.867 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:55:10.867 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:10.867 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:55:10.867 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:10.867 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:10.867 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1738806658318718 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.23:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:55:10.867 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:14.869 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:55:14.870 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:55:14.870 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:55:14.870 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:14.870 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:55:14.870 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:14.870 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:55:14.871 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:55:14.871 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:14.871 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:55:14.871 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:14.871 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:14.871 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1738806658318718 collector_name = collector_ip = 10.0.0.41:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.23:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:55:14.871 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:18.871 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:55:18.873 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:55:18.873 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:55:18.873 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:18.873 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:55:18.873 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:18.873 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:55:18.874 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:55:18.874 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:18.874 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:55:18.874 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:18.874 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:18.874 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1738806658318718 collector_name = collector_ip = 10.0.0.23:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.23:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:55:18.874 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:22.873 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:55:22.874 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: Session Event: TCP Connect Fail 02/06/2025 01:55:22.874 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:55:22.874 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:22.874 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:55:22.875 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:22.875 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 02/06/2025 01:55:22.875 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 02/06/2025 01:55:22.875 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:22.875 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 02/06/2025 01:55:22.875 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:22.875 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:22.875 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 5 pid = 1 http_port = 8103 start_time = 1738806658318718 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.41:8086, 10.0.0.23:8086, 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 6 >> 02/06/2025 01:55:22.875 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 01:55:26.877 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 01:55:26.878 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 02/06/2025 01:55:26.878 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 01:55:26.878 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:26.878 7fbc940b4448 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.41:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 01:55:26.879 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 01:55:26.879 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 02/06/2025 01:55:26.879 7fbc940b4448 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 02/06/2025 01:55:26.879 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 02/06/2025 01:55:26.879 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 01:55:26.879 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 01:55:28.367 7fbc94073448 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001016 02/06/2025 01:55:28.368 7fbc94073448 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 02/06/2025 01:55:28.368 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 02/06/2025 01:55:28.368 7fbc940b4448 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 02/06/2025 01:55:28.368 7fbc940b4448 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 02/06/2025 01:55:28.368 7fbc940b4448 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 02/06/2025 01:55:28.369 7fbc940b4448 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/06/2025 01:55:28.369 7fbc940b4448 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/06/2025 01:55:28.369 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 02/06/2025 01:55:28.369 7fbc940b4148 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/06/2025 01:55:28.371 7fbc940b4148 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/06/2025 01:55:38.360 7fbc94073448 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001582 02/06/2025 01:55:38.361 7fbc94073448 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 02/06/2025 01:55:38.362 7fbc940b4148 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 02/06/2025 01:55:38.362 7fbc940b4148 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 4 02/06/2025 01:55:38.362 7fbc940b4148 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 02/06/2025 01:55:38.362 7fbc940b4148 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 02/06/2025 01:55:38.362 7fbc940b4148 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/06/2025 01:55:38.362 7fbc940b4148 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/06/2025 01:55:38.363 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/06/2025 01:55:38.363 7fbc940b4448 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/06/2025 01:55:59.307 7fbc9ae6af20 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:56:07.545 7fbc9ae6af20 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=10568160, partition_space_available_1k=70522324, percentage_partition_space_used=13, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1378384, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11460, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44788, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=5880208, free=324104, buffers=398696, cached=9572624, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.14, five_min_avg=3.79, fifteen_min_avg=3.75, cpu_share=8.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:56:59.298 7fbc9ae6af20 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:57:07.971 7fbc9ae6af20 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=10986232, partition_space_available_1k=70104252, percentage_partition_space_used=14, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1355340, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11484, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44796, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6013488, free=185428, buffers=408660, cached=9568056, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.6, five_min_avg=3.54, fifteen_min_avg=3.66, cpu_share=7.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:57:59.310 7fbc9ae6af20 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:58:06.945 7fbc9ae6af20 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=10998020, partition_space_available_1k=70092464, percentage_partition_space_used=14, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1363004, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11012, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44796, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=5983700, free=249520, buffers=407208, cached=9535204, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.18, five_min_avg=2.95, fifteen_min_avg=3.45, cpu_share=4.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:58:59.315 7fbc9ae6af20 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 01:59:06.952 7fbc9ae6af20 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=10997752, partition_space_available_1k=70092732, percentage_partition_space_used=14, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1388992, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11868, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44800, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6085296, free=223712, buffers=406456, cached=9460168, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.2, five_min_avg=2.59, fifteen_min_avg=3.29, cpu_share=4.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 01:59:59.311 7fbc9ae6af20 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:00:06.946 7fbc9ae6af20 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=11782700, partition_space_available_1k=69307784, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1385860, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12120, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=44804, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=4044424, free=2393532, buffers=417936, cached=9319740, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.09, five_min_avg=2.59, fifteen_min_avg=3.24, cpu_share=7.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:00:50.565 7f29b8c81a88 [contrail-database-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 02/06/2025 02:00:50.565 7f29b8c81a88 [contrail-database-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 02/06/2025 02:00:50.566 7f29b8c81a88 [contrail-database-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 02/06/2025 02:00:50.574 7f29b5d1da20 [contrail-database-nodemgr] [ERROR]: Cannot write http_port 8103 to /tmp/contrail-database-nodemgr.0.http_port 02/06/2025 02:00:50.574 7f29b5d1da20 [contrail-database-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8103 02/06/2025 02:00:50.574 7f29b5d23390 [contrail-database-nodemgr] [INFO]: Processing event[EvStart] in state[none] 02/06/2025 02:00:50.575 7f29b5d23390 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/06/2025 02:00:50.575 7f29b5d23390 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/06/2025 02:00:50.576 7f29b5d23390 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 02/06/2025 02:00:50.576 7f29b5d23390 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 02:00:50.576 7f29b5d23390 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/06/2025 02:00:50.576 7f29b5d23390 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 02:00:50.576 7f29b5d23390 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 02:00:50.576 7f29b5d23390 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2.:Database:contrail-database-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8103 start_time = 1738807250575905 collector_name = collector_ip = collector_list = [ 10.0.0.23:8086, 10.0.0.38:8086, 10.0.0.41:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 02/06/2025 02:00:50.576 7f29b5d23390 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 02:00:50.672 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'cassandra', 'group': 'cassandra', 'pid': 64004691485426413130568784027283496113899416558413857736166547095327408547901, 'start': '1738807248000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-query-engine', 'group': 'contrail-query-engine', 'pid': 97620487073845693865566349687001126827881683906294563621685310009677663026861, 'start': '1738807248000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-database-nodemgr', 'group': 'contrail-database-nodemgr', 'pid': 45735852880835183459843327633266029071987759336086765697890289357917385951304, 'start': '1738807248000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 02/06/2025 02:00:50.791 7f29bef10a20 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 02:00:50.791 7f29bef10a20 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-02-06 00:48:10.528155", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7689-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 4 num_core_per_socket = 4 num_thread_per_core = 1 >> running_package_version = nightly-nightly.el7 installed_package_version = nightly-nightly.el7 >> 02/06/2025 02:00:50.791 7f29bef10a20 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 02:00:50.792 7f29b8c81a88 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-2.', _node_type='Database', _instance_id='0', _timestamp=1738807250792208, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/06/2025 02:00:50.792 7f29b8c81a88 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-2.', _node_type='Database', _instance_id='0', _timestamp=1738807250792466, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738807248000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738807248000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738807248000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2025-02-06 00:48:10.528155", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7689-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/06/2025 02:00:50.794 7f29b85ed6d8 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 02:00:50.794 7f29b85ed6d8 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/06/2025 02:00:50.794 7f29b85ed6d8 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 02:00:50.794 7f29b85ed6d8 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 02:00:50.794 7f29b85ed6d8 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. deleted = False process_info = [ << process_name = cassandra process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1738807248000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-query-engine process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1738807248000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-database-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1738807248000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-02-06 00:48:10.528155", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7689-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} >> 02/06/2025 02:00:50.794 7f29b85ed6d8 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 02:00:51.046 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:00:51.365 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:DatabaseUsage] map 02/06/2025 02:00:51.366 7f29b66cb588 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 02:00:51.366 7f29b66cb588 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] DatabaseUsage: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. database_usage = [ << disk_space_used_1k = 12197276 disk_space_available_1k = 68893208 analytics_db_size_1k = 43056 >>, ] >> 02/06/2025 02:00:51.366 7f29b66cb588 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 02:00:51.796 7f29b8c81a88 [contrail-database-nodemgr] [ERROR]: Failed to connect to database by CQL: Result: 1 02/06/2025 02:00:51.797 7f29b8c81a88 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-2.', _node_type='Database', _instance_id='0', _timestamp=1738807251796690, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/06/2025 02:00:51.798 7f29b85ed0f0 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 02:00:51.798 7f29b85ed0f0 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Cassandra state detected DOWN. Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/06/2025 02:00:51.798 7f29b85ed0f0 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 02:00:54.426 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-2.', _node_type='Database', _instance_id='0', _timestamp=1738807254426587, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=1), thread_pool_stats=[CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=1, pending=1, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0)], _table='ObjectDatabaseInfo'), _seqnum=0) 02/06/2025 02:00:54.426 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: Add uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 02/06/2025 02:00:54.430 7f29b66cb2e8 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 02/06/2025 02:00:54.431 7f29b66cb2e8 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] CassandraStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. cassandra_compaction_task = << pending_compaction_tasks = 1 >> thread_pool_stats = [ << pool_name = MemtableFlushWriter active = 1 pending = 1 all_time_blocked = 0 >>, << pool_name = CompactionExecutor active = 0 pending = 0 all_time_blocked = 0 >>, ] >> 02/06/2025 02:00:54.432 7f29b66cb2e8 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 02/06/2025 02:00:54.576 7f29b4867978 [contrail-database-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 02/06/2025 02:00:54.577 7f29b4867978 [contrail-database-nodemgr] [INFO]: Session Event: TCP Connected 02/06/2025 02:00:54.577 7f29b4867978 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 02/06/2025 02:00:54.578 7f29b4867978 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 02/06/2025 02:00:54.578 7f29b4867978 [contrail-database-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2696-2. process_status = [ << module_id = contrail-database-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.23:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Cassandra state detected DOWN. Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/06/2025 02:00:54.578 7f29b4867978 [contrail-database-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 02/06/2025 02:00:54.578 7f29b4867978 [contrail-database-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 02/06/2025 02:00:54.578 7f29b4867978 [contrail-database-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 02/06/2025 02:00:54.578 7f29b4867978 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 02/06/2025 02:00:54.578 7f29b4867978 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 02:00:54.579 7f29b4867978 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 02/06/2025 02:00:54.583 7f29b4847978 [contrail-database-nodemgr] [DEBUG]: Extracted sandesh length: 0000001017 02/06/2025 02:00:54.583 7f29b4847978 [contrail-database-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 02/06/2025 02:00:54.584 7f29b871f048 [contrail-database-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 02/06/2025 02:00:54.585 7f29b871f048 [contrail-database-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 02/06/2025 02:00:54.585 7f29b871f048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:DatabaseUsage] map 02/06/2025 02:00:54.585 7f29b871f048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:CassandraStatusUVE] map 02/06/2025 02:00:54.585 7f29b871f048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/06/2025 02:00:54.586 7f29b871f048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 02/06/2025 02:00:54.586 7f29b871f048 [contrail-database-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 02/06/2025 02:00:54.587 7f29b4847d68 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 02/06/2025 02:00:54.588 7f29b4847d68 [contrail-database-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 02/06/2025 02:00:59.450 7f29b8c81a88 [contrail-database-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-2.', _node_type='Database', _instance_id='0', _timestamp=1738807259450060, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='cassandra', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738807248000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-query-engine', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738807248000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-database-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1738807248000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2025-02-06 00:48:10.528155", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-7689-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/06/2025 02:00:59.450 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12199088, partition_space_available_1k=68891396, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1243760, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=9652, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=49852, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=5642640, free=340284, buffers=432628, cached=9760080, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.77, five_min_avg=2.75, fifteen_min_avg=3.27, cpu_share=0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:01:50.996 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:01:51.860 7f29b8c81a88 [contrail-database-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-2.', _node_type='Database', _instance_id='0', _timestamp=1738807311860623, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-database-nodemgr', instance_id='0', state='Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.23:8086'], status='Up', description='ClientInit to Established on EvSandeshCtrlMessageRecv')]), flag_infos=None, description='')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo'), _seqnum=0) 02/06/2025 02:01:53.618 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: Sending UVE: CassandraStatusUVE(_context='', _scope='', _module='contrail-database-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2696-2.', _node_type='Database', _instance_id='0', _timestamp=1738807313618150, _versionsig=46427434, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=CassandraStatusData(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, cassandra_compaction_task=CassandraCompactionTask(pending_compaction_tasks=0), thread_pool_stats=[CassandraThreadPoolStats(pool_name='ReadStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='CompactionExecutor', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MutationStage', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='Native-Transport-Requests', active=0, pending=0, all_time_blocked=0), CassandraThreadPoolStats(pool_name='MemtableFlushWriter', active=0, pending=0, all_time_blocked=0)], _table='ObjectDatabaseInfo'), _seqnum=0) 02/06/2025 02:01:58.641 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12206648, partition_space_available_1k=68883836, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1330304, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10616, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50032, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=5747560, free=276036, buffers=431956, cached=9720080, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.26, five_min_avg=2.32, fifteen_min_avg=3.08, cpu_share=5.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:02:51.005 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:02:58.624 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12211896, partition_space_available_1k=68878588, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1326032, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10932, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=49988, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6127812, free=253900, buffers=429168, cached=9364752, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.05, five_min_avg=2.06, fifteen_min_avg=2.95, cpu_share=5.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:03:51.028 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:03:58.536 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12215560, partition_space_available_1k=68874924, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1331180, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10952, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50172, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6157192, free=246976, buffers=429944, cached=9341520, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.31, five_min_avg=1.96, fifteen_min_avg=2.85, cpu_share=4.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:04:51.029 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:04:58.491 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12218980, partition_space_available_1k=68871504, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1338852, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11056, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50176, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6156404, free=257252, buffers=429840, cached=9332136, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.81, five_min_avg=1.69, fifteen_min_avg=2.7, cpu_share=4.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:05:51.040 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:05:58.549 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12224352, partition_space_available_1k=68866132, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1374740, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=11132, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50184, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6172464, free=243208, buffers=429744, cached=9330216, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.73, five_min_avg=1.5, fifteen_min_avg=2.57, cpu_share=4.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:06:51.047 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:06:58.486 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12224956, partition_space_available_1k=68865528, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1350604, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10500, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50180, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6157128, free=268032, buffers=429772, cached=9320700, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.03, five_min_avg=1.45, fifteen_min_avg=2.49, cpu_share=4.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:07:51.056 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:07:58.478 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12228308, partition_space_available_1k=68862176, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1358992, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10572, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50192, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6177948, free=252668, buffers=429756, cached=9315260, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.82, five_min_avg=1.31, fifteen_min_avg=2.37, cpu_share=3.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:08:51.078 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:08:58.533 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12231752, partition_space_available_1k=68858732, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1382336, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10596, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50184, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6190012, free=260776, buffers=429720, cached=9295124, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.09, five_min_avg=1.27, fifteen_min_avg=2.29, cpu_share=4.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:09:51.083 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:09:58.544 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12234896, partition_space_available_1k=68855588, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1368556, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10664, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50180, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6179096, free=290472, buffers=429708, cached=9276356, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.99, five_min_avg=1.19, fifteen_min_avg=2.2, cpu_share=4.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:10:51.094 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:10:58.568 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12238264, partition_space_available_1k=68852220, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1375816, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10728, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50184, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6206620, free=266296, buffers=429704, cached=9273012, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.16, five_min_avg=1.18, fifteen_min_avg=2.12, cpu_share=4.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:11:51.097 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:11:58.531 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12242932, partition_space_available_1k=68847552, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1389000, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10732, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50180, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6229096, free=251544, buffers=429760, cached=9265232, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.23, five_min_avg=1.2, fifteen_min_avg=2.07, cpu_share=4.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:12:51.112 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:12:58.581 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12246344, partition_space_available_1k=68844140, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1388852, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10664, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50176, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6226772, free=272492, buffers=429908, cached=9246460, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.27, five_min_avg=1.22, fifteen_min_avg=2.02, cpu_share=5.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:13:51.112 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:13:58.550 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12250368, partition_space_available_1k=68840116, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1396344, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10696, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50176, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6238516, free=268744, buffers=429968, cached=9238404, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.91, five_min_avg=1.12, fifteen_min_avg=1.94, cpu_share=4.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:14:51.123 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:14:58.645 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12248888, partition_space_available_1k=68841596, percentage_partition_space_used=15, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1398860, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10728, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50184, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6262828, free=243196, buffers=429968, cached=9239640, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.77, five_min_avg=1.04, fifteen_min_avg=1.85, cpu_share=4.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:15:18.180 7f29b6997b70 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/06/2025 02:15:32.377 7f29b69cc048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/06/2025 02:15:51.136 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:15:58.601 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12610688, partition_space_available_1k=68479796, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1407436, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10892, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50068, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6265868, free=315600, buffers=434532, cached=9159632, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.1, five_min_avg=1.06, fifteen_min_avg=1.81, cpu_share=5.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:16:51.147 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:16:54.493 7f29b5d774e0 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/06/2025 02:16:59.026 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12615768, partition_space_available_1k=68474716, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1412884, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=10980, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50124, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6274376, free=305028, buffers=434588, cached=9161640, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.95, five_min_avg=1.01, fifteen_min_avg=1.74, cpu_share=4.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:17:51.144 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:17:58.864 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12624912, partition_space_available_1k=68465572, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1433656, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12328, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50116, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6303504, free=314424, buffers=434288, cached=9123416, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=0.87, five_min_avg=0.98, fifteen_min_avg=1.68, cpu_share=6.25, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:18:51.235 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:18:58.789 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12645492, partition_space_available_1k=68444992, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1461816, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12232, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50096, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6387016, free=280992, buffers=434248, cached=9073376, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.75, five_min_avg=1.22, fifteen_min_avg=1.72, cpu_share=9.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:19:51.156 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:19:58.752 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12662360, partition_space_available_1k=68428124, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1487256, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12304, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50076, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6392476, free=320992, buffers=434232, cached=9027932, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.05, five_min_avg=1.42, fifteen_min_avg=1.76, cpu_share=6.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:20:51.192 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:20:58.813 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12674752, partition_space_available_1k=68415732, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1503812, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12372, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50068, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6440328, free=295300, buffers=434248, cached=9005756, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.11, five_min_avg=1.57, fifteen_min_avg=1.79, cpu_share=8.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:21:51.173 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:21:59.087 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12684772, partition_space_available_1k=68405712, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1521952, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12428, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50060, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6427684, free=323740, buffers=434156, cached=8990052, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.96, five_min_avg=1.95, fifteen_min_avg=1.91, cpu_share=6.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:22:51.164 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:22:58.814 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12707440, partition_space_available_1k=68383044, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1656400, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12488, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50024, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6550280, free=290220, buffers=433600, cached=8901532, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.28, five_min_avg=1.93, fifteen_min_avg=1.91, cpu_share=7.5, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:23:32.590 7f29b8a4d048 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map 02/06/2025 02:23:51.172 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:23:58.727 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12713404, partition_space_available_1k=68377080, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1657440, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12564, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50048, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6516940, free=295620, buffers=433772, cached=8929300, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.4, five_min_avg=1.76, fifteen_min_avg=1.85, cpu_share=5.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:24:51.176 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:24:58.719 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12719440, partition_space_available_1k=68371044, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1663760, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12480, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50040, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6505364, free=323972, buffers=433596, cached=8912700, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.19, five_min_avg=2.06, fifteen_min_avg=1.96, cpu_share=5.0, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:25:51.186 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: dir for analytics is /var/lib/cassandra/data 02/06/2025 02:25:58.749 7f29b8c81a88 [contrail-database-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2696-2.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=12723448, partition_space_available_1k=68367036, percentage_partition_space_used=16, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'cassandra': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1669464, cpu_share=0, mem_res=0, _EventManager__key='cassandra'), 'contrail-query-engine': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=12528, cpu_share=0, mem_res=0, _EventManager__key='contrail-query-engine'), 'contrail-database-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50048, cpu_share=0, mem_res=0, _EventManager__key='contrail-database-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=16175632, used=6530200, free=304904, buffers=433604, cached=8906924, node_type='database-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.52, five_min_avg=1.88, fifteen_min_avg=1.9, cpu_share=4.75, node_type='database-node'), running_package_version=None, installed_package_version=None, _table='ObjectDatabaseInfo') 02/06/2025 02:26:00.321 7f29b5d202e8 [contrail-database-nodemgr] [DEBUG]: send sync_uve in the [ObjectDatabaseInfo:NodeStatusUVE] map