03/31/2025 02:14:11.915 7f1c5312dd08 [contrail-config-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 03/31/2025 02:14:11.916 7f1c5312dd08 [contrail-config-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 03/31/2025 02:14:11.916 7f1c5312dd08 [contrail-config-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 03/31/2025 02:14:11.931 7f1c5007c048 [contrail-config-nodemgr] [ERROR]: Cannot write http_port 8100 to /tmp/contrail-config-nodemgr.0.http_port 03/31/2025 02:14:11.932 7f1c5007c048 [contrail-config-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8100 03/31/2025 02:14:11.933 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Processing event[EvStart] in state[none] 03/31/2025 02:14:11.934 7f1c5007f978 [contrail-config-nodemgr] [DEBUG]: Add uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:14:11.934 7f1c5007f978 [contrail-config-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/31/2025 02:14:11.935 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 03/31/2025 02:14:11.935 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:11.935 7f1c5007f978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/31/2025 02:14:11.935 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:11.935 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:11.935 7f1c5007f978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 03/31/2025 02:14:11.935 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:12.129 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'contrail-api', 'group': 'contrail-api', 'pid': 89687563381206996764143234153987947026644395878519822414441533017957539030458, 'start': '1743387248000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-schema', 'group': 'contrail-schema', 'pid': 105345277605447083409614394513693784274539194127280236742747813444222253946254, 'start': '1743387249000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-svc-monitor', 'group': 'contrail-svc-monitor', 'pid': 69724438755935756807882633582921967358976811068699491446518003444561609945546, 'start': '1743387248000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-device-manager', 'group': 'contrail-device-manager', 'pid': 69018862291209388671414242610142566538003763121025687915929590108859189194331, 'start': '1743387249000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-config-nodemgr', 'group': 'contrail-config-nodemgr', 'pid': 114062809146710912949892395408653989002572669030098569576265568565221479422944, 'start': '1743387248000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 03/31/2025 02:14:12.564 7f1c5312dd08 [contrail-config-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-nodemgr', _source='cn-jenkins-deploy-platform-juju-os-494-1', _node_type='Config', _instance_id='0', _timestamp=1743387252564508, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigNode'), _seqnum=0) 03/31/2025 02:14:12.565 7f1c5312dd08 [contrail-config-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-nodemgr', _source='cn-jenkins-deploy-platform-juju-os-494-1', _node_type='Config', _instance_id='0', _timestamp=1743387252565253, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='contrail-api', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1743387248000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-schema', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1743387249000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-svc-monitor', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1743387248000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-device-manager', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1743387249000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1743387248000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2025-03-31 00:53:57.142496", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-8423-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigNode'), _seqnum=0) 03/31/2025 02:14:12.570 7f1c5937aa20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:12.572 7f1c5937aa20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-03-31 00:53:57.142496", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-8423-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 8 num_core_per_socket = 8 num_thread_per_core = 1 >> running_package_version = nightly-nightly.el7 installed_package_version = nightly-nightly.el7 >> 03/31/2025 02:14:12.572 7f1c5937aa20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:12.573 7f1c5937aa20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:12.573 7f1c5937aa20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/31/2025 02:14:12.573 7f1c5937aa20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:12.573 7f1c5937aa20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:12.575 7f1c5937aa20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 deleted = False process_info = [ << process_name = contrail-api process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1743387248000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-schema process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1743387249000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-svc-monitor process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1743387248000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-device-manager process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1743387249000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1743387248000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-03-31 00:53:57.142496", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-8423-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} >> 03/31/2025 02:14:12.580 7f1c5937aa20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:15.935 7f1c4f4a0828 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:14:15.936 7f1c4f4a0828 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:14:15.936 7f1c4f4a0828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:14:15.937 7f1c4f4a0828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:14:15.937 7f1c4f4a0828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:14:15.937 7f1c4f4a0828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:14:15.937 7f1c4f4a0828 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:14:15.937 7f1c4f4a0828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:14:15.937 7f1c4f4a0828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:15.938 7f1c4f4a0828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:14:15.938 7f1c4f4a0828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:15.938 7f1c4f4a0828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:15.938 7f1c4f4a0828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/31/2025 02:14:15.938 7f1c4f4a0828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:19.217 7f1c4f4912e8 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:14:19.938 7f1c4f491048 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:14:19.941 7f1c4f491048 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:14:19.941 7f1c4f491048 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:14:19.941 7f1c4f491048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:14:19.941 7f1c4f491048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:14:19.941 7f1c4f491048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:14:19.941 7f1c4f491048 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:14:19.942 7f1c4f491048 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:14:19.942 7f1c4f491048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:19.942 7f1c4f491048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:14:19.942 7f1c4f491048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:19.942 7f1c4f491048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:19.942 7f1c4f491048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/31/2025 02:14:19.942 7f1c4f491048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:21.701 7f1c5312dd08 [contrail-config-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-nodemgr', _source='cn-jenkins-deploy-platform-juju-os-494-1', _node_type='Config', _instance_id='0', _timestamp=1743387261701048, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='contrail-api', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1743387248000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-schema', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1743387249000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-svc-monitor', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1743387248000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-device-manager', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1743387249000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1743387248000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2025-03-31 00:53:57.142496", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-8423-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigNode'), _seqnum=0) 03/31/2025 02:14:21.701 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50194892, partition_space_available_1k=30895592, percentage_partition_space_used=62, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1584, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1584, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=6436, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1612, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=82800, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=11312756, free=561272, buffers=775676, cached=20015408, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.99, five_min_avg=3.23, fifteen_min_avg=2.23, cpu_share=0, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:14:21.701 7f1c509308d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:21.701 7f1c509308d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 deleted = False process_info = [ << process_name = contrail-api process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1743387248000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-schema process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1743387249000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-svc-monitor process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1743387248000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-device-manager process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1743387249000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1743387248000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-03-31 00:53:57.142496", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-8423-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} >> 03/31/2025 02:14:21.702 7f1c509308d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:21.702 7f1c509308d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:21.702 7f1c509308d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 50194892 partition_space_available_1k = 30895592 percentage_partition_space_used = 62 >> } all_core_file_list = [ ] process_mem_cpu_usage = { contrail-api : << mem_virt = 1584 cpu_share = 0 mem_res = 0 >>contrail-schema : << mem_virt = 1584 cpu_share = 0 mem_res = 0 >>contrail-svc-monitor : << mem_virt = 6436 cpu_share = 0 mem_res = 0 >>contrail-device-manager : << mem_virt = 1612 cpu_share = 0 mem_res = 0 >>contrail-config-nodemgr : << mem_virt = 82800 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 11312756 free = 561272 buffers = 775676 cached = 20015408 node_type = config-node >> system_cpu_usage = << one_min_avg = 4.99 five_min_avg = 3.23 fifteen_min_avg = 2.23 cpu_share = 0 node_type = config-node >> >> 03/31/2025 02:14:21.702 7f1c509308d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:23.942 7f1c52a8cd68 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:14:23.943 7f1c52a8cd68 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:14:23.944 7f1c52a8cd68 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:14:23.944 7f1c52a8cd68 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:14:23.944 7f1c52a8cd68 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:14:23.944 7f1c52a8cd68 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:14:23.944 7f1c52a8cd68 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:14:23.944 7f1c52a8cd68 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:14:23.944 7f1c52a8cd68 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:23.945 7f1c52a8cd68 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:14:23.945 7f1c52a8cd68 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:23.945 7f1c52a8cd68 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:23.945 7f1c52a8cd68 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/31/2025 02:14:23.945 7f1c52a8cd68 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:25.680 7f1c52a8cd68 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:14:27.945 7f1c52a8cd68 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:14:27.946 7f1c52a8cd68 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:14:27.946 7f1c52a8cd68 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:14:27.946 7f1c52a8cd68 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:14:27.947 7f1c52a8cd68 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:14:27.947 7f1c52a8cd68 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:14:27.947 7f1c52a8cd68 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:14:27.947 7f1c52a8cd68 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:14:27.947 7f1c52a8cd68 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:27.947 7f1c52a8cd68 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:14:27.948 7f1c52a8cd68 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:27.948 7f1c52a8cd68 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:27.948 7f1c52a8cd68 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/31/2025 02:14:27.948 7f1c52a8cd68 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:31.481 7f1c4f4ceb70 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:14:31.947 7f1c4f4ceb70 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:14:31.948 7f1c4f4ceb70 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:14:31.949 7f1c4f4ceb70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:14:31.949 7f1c4f4ceb70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:14:31.949 7f1c4f4ceb70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:14:31.949 7f1c4f4ceb70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:14:31.949 7f1c4f4ceb70 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:14:31.949 7f1c4f4ceb70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:14:31.950 7f1c4f4ceb70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:31.950 7f1c4f4ceb70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:14:31.950 7f1c4f4ceb70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:31.950 7f1c4f4ceb70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:31.950 7f1c4f4ceb70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/31/2025 02:14:31.950 7f1c4f4ceb70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:35.953 7f1c4f4ceb70 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:14:35.954 7f1c4f4ceb70 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:14:35.954 7f1c4f4ceb70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:14:35.954 7f1c4f4ceb70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:14:35.954 7f1c4f4ceb70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:14:35.954 7f1c4f4ceb70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:14:35.954 7f1c4f4ceb70 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:14:35.955 7f1c4f4ceb70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:14:35.955 7f1c4f4ceb70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:35.955 7f1c4f4ceb70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:14:35.955 7f1c4f4ceb70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:35.955 7f1c4f4ceb70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:35.955 7f1c4f4ceb70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/31/2025 02:14:35.955 7f1c4f4ceb70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:39.957 7f1c4f4ceb70 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:14:39.958 7f1c4f4ceb70 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:14:39.958 7f1c4f4ceb70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:14:39.958 7f1c4f4ceb70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:14:39.958 7f1c4f4ceb70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:14:39.958 7f1c4f4ceb70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:14:39.958 7f1c4f4ceb70 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:14:39.958 7f1c4f4ceb70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:14:39.959 7f1c4f4ceb70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:39.959 7f1c4f4ceb70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:14:39.959 7f1c4f4ceb70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:39.959 7f1c4f4ceb70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:39.959 7f1c4f4ceb70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/31/2025 02:14:39.959 7f1c4f4ceb70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:42.717 7f1c4f4ceb70 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:14:43.958 7f1c50857240 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:14:43.960 7f1c50857240 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:14:43.960 7f1c50857240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:14:43.960 7f1c50857240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:14:43.960 7f1c50857240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:14:43.960 7f1c50857240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:14:43.960 7f1c50857240 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:14:43.960 7f1c50857240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:14:43.961 7f1c50857240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:43.961 7f1c50857240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:14:43.961 7f1c50857240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:43.961 7f1c50857240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:43.961 7f1c50857240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/31/2025 02:14:43.961 7f1c50857240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:14:47.964 7f1c50857240 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:14:48.441 7f1c50857240 [contrail-config-nodemgr] [INFO]: Session Event: TCP Connected 03/31/2025 02:14:48.441 7f1c50857240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:14:48.441 7f1c50857240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:14:48.441 7f1c50857240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:14:48.441 7f1c50857240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:14:48.441 7f1c50857240 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 03/31/2025 02:14:48.442 7f1c50857240 [contrail-config-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 03/31/2025 02:14:48.442 7f1c50857240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 03/31/2025 02:14:48.442 7f1c50857240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/31/2025 02:14:48.442 7f1c50857240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/31/2025 02:14:48.454 7f1c52f23780 [contrail-config-nodemgr] [DEBUG]: Extracted sandesh length: 0000001011 03/31/2025 02:14:48.455 7f1c52f23780 [contrail-config-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 03/31/2025 02:14:48.455 7f1c50930198 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 03/31/2025 02:14:48.456 7f1c50930198 [contrail-config-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 03/31/2025 02:14:48.456 7f1c50930198 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:14:48.456 7f1c50930198 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/31/2025 02:14:48.456 7f1c50930198 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 03/31/2025 02:14:48.457 7f1c50857390 [contrail-config-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 03/31/2025 02:14:48.458 7f1c50857390 [contrail-config-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 03/31/2025 02:15:07.759 7f1c50941a20 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:15:21.633 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50823928, partition_space_available_1k=30266556, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=93280, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=82496, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=90784, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=191180, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58108, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14620688, free=11078172, buffers=803632, cached=6162620, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.91, five_min_avg=4.09, fifteen_min_avg=2.59, cpu_share=8.25, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:15:26.533 7f1c5314f828 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:15:41.488 7f1c4f4a0630 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:16:02.792 7f1c50956cc0 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:16:13.383 7f1c50956cc0 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:16:21.640 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50852556, partition_space_available_1k=30237928, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=99212, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83952, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88964, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188272, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58100, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14653036, free=9723000, buffers=841028, cached=7448048, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=8.52, five_min_avg=5.33, fifteen_min_avg=3.13, cpu_share=5.62, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:16:54.925 7f1c4f4ceb70 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:17:21.648 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50855224, partition_space_available_1k=30235260, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=99464, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83964, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88892, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188284, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58104, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14624600, free=9710176, buffers=842904, cached=7487432, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.53, five_min_avg=4.77, fifteen_min_avg=3.07, cpu_share=2.88, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:18:21.668 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50858572, partition_space_available_1k=30231912, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=99508, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83964, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88892, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188296, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58104, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14616044, free=9713124, buffers=842976, cached=7492968, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.75, five_min_avg=4.23, fifteen_min_avg=2.99, cpu_share=2.25, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:19:04.673 7f1c4f491e10 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:19:12.642 7f1c5312dd08 [contrail-config-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-nodemgr', _source='cn-jenkins-deploy-platform-juju-os-494-1', _node_type='Config', _instance_id='0', _timestamp=1743387552642231, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.17:8086'], status='Up', description='ClientInit to Established on EvSandeshCtrlMessageRecv')]), flag_infos=None, description='NTP state unsynchronized. ')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigNode'), _seqnum=0) 03/31/2025 02:19:13.435 7f1c50930d68 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:19:21.694 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50863632, partition_space_available_1k=30226852, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=99576, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83968, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88896, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188304, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58452, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14671492, free=9647516, buffers=843116, cached=7502988, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.16, five_min_avg=3.76, fifteen_min_avg=2.91, cpu_share=2.62, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:19:25.038 7f1c5008ea20 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:19:47.922 7f1c50937978 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:20:21.674 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50877800, partition_space_available_1k=30212684, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=99720, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83968, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88896, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188316, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58452, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14798776, free=9418604, buffers=843172, cached=7604560, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.52, five_min_avg=4.59, fifteen_min_avg=3.26, cpu_share=5.88, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:21:21.681 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50877928, partition_space_available_1k=30212556, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=99776, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83968, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88896, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188324, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58460, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14643308, free=9499948, buffers=843180, cached=7678676, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.21, five_min_avg=4.41, fifteen_min_avg=3.28, cpu_share=2.5, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:22:21.692 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50882028, partition_space_available_1k=30208456, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=99964, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83968, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88896, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188336, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58456, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14648724, free=9489336, buffers=843180, cached=7683872, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.67, five_min_avg=3.96, fifteen_min_avg=3.2, cpu_share=2.5, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:23:21.700 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50885576, partition_space_available_1k=30204908, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=100092, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83968, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88896, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188344, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58460, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14640292, free=9493524, buffers=843192, cached=7688104, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.9, five_min_avg=3.52, fifteen_min_avg=3.09, cpu_share=2.38, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:23:40.678 7f1c52ff12e8 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:23:49.273 7f1c52ff12e8 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:24:04.786 7f1c5007fb70 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:24:12.667 7f1c5312dd08 [contrail-config-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-nodemgr', _source='cn-jenkins-deploy-platform-juju-os-494-1', _node_type='Config', _instance_id='0', _timestamp=1743387852667557, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-nodemgr', instance_id='0', state='Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=['10.0.0.17:8086'], status='Up', description='ClientInit to Established on EvSandeshCtrlMessageRecv')]), flag_infos=None, description='')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigNode'), _seqnum=0) 03/31/2025 02:24:14.360 7f1c50876e10 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:24:21.718 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50894116, partition_space_available_1k=30196368, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=100128, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83968, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88896, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188352, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58464, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14700580, free=9415948, buffers=843244, cached=7705340, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.73, five_min_avg=4.56, fifteen_min_avg=3.47, cpu_share=4.25, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:25:21.714 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50899536, partition_space_available_1k=30190948, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=100196, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83968, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88900, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188360, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58464, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14691596, free=9417888, buffers=843244, cached=7712384, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.42, five_min_avg=4.0, fifteen_min_avg=3.35, cpu_share=2.62, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:26:21.720 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51005848, partition_space_available_1k=30084636, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=100204, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83968, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88908, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188372, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58468, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14678456, free=9424348, buffers=843248, cached=7719060, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.16, five_min_avg=3.83, fifteen_min_avg=3.33, cpu_share=2.75, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:27:21.728 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50907248, partition_space_available_1k=30183236, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=100244, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83968, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88908, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188380, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58608, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14675560, free=9490180, buffers=843248, cached=7656124, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.4, five_min_avg=3.49, fifteen_min_avg=3.25, cpu_share=2.62, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:28:03.068 7f1c50860390 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:28:11.573 7f1c50877eb8 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:28:21.727 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50915236, partition_space_available_1k=30175248, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=100312, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83976, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88908, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188388, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58472, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14716644, free=9443180, buffers=843248, cached=7662040, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.77, five_min_avg=3.38, fifteen_min_avg=3.22, cpu_share=2.88, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:28:22.672 7f1c50857390 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:28:32.241 7f1c50857390 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:29:21.744 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50914636, partition_space_available_1k=30175848, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=100328, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83972, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88908, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188404, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58476, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14682924, free=9473864, buffers=843248, cached=7665076, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.01, five_min_avg=3.02, fifteen_min_avg=3.1, cpu_share=2.75, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:30:21.739 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50923604, partition_space_available_1k=30166880, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=100508, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83972, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88916, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188412, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58476, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14714000, free=9423076, buffers=843252, cached=7684784, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.42, five_min_avg=3.25, fifteen_min_avg=3.18, cpu_share=4.38, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:31:21.747 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50928836, partition_space_available_1k=30161648, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=100588, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83972, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88912, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188420, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58480, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14697556, free=9435016, buffers=843280, cached=7689260, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.39, five_min_avg=2.98, fifteen_min_avg=3.08, cpu_share=2.5, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:32:21.746 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50932412, partition_space_available_1k=30158072, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=100592, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83972, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88912, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188432, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58476, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14689760, free=9437332, buffers=843296, cached=7694724, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.33, five_min_avg=2.82, fifteen_min_avg=3.02, cpu_share=2.5, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:33:21.437 7f1c50930d68 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:33:21.747 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50935872, partition_space_available_1k=30154612, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=100636, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83972, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88912, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188440, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58480, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14758024, free=9359868, buffers=843312, cached=7703908, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.9, five_min_avg=2.62, fifteen_min_avg=2.94, cpu_share=2.5, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:33:35.461 7f1c52aefeb8 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:33:44.049 7f1c4f4b1c18 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:33:55.073 7f1c5007ff60 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:34:21.762 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50940008, partition_space_available_1k=30150476, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=100704, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83968, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88912, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188452, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58484, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14686936, free=9422368, buffers=843560, cached=7712248, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.6, five_min_avg=2.65, fifteen_min_avg=2.93, cpu_share=2.75, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:35:21.751 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50950080, partition_space_available_1k=30140404, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=100704, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83972, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88912, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188464, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58488, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14721204, free=9349848, buffers=844576, cached=7749484, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.1, five_min_avg=3.21, fifteen_min_avg=3.11, cpu_share=4.25, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:36:21.756 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50955756, partition_space_available_1k=30134728, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=100744, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83968, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88916, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188468, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58484, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14728032, free=9335568, buffers=844580, cached=7756932, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.75, five_min_avg=3.3, fifteen_min_avg=3.15, cpu_share=2.5, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:37:21.768 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50959216, partition_space_available_1k=30131268, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=100852, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83968, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88916, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188480, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58488, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14739856, free=9318688, buffers=844584, cached=7761984, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.1, five_min_avg=2.88, fifteen_min_avg=3.01, cpu_share=2.5, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:37:44.121 7f1c5094cac8 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:38:21.782 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50962836, partition_space_available_1k=30127648, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=100952, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83968, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88924, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188488, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58492, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14722308, free=9331680, buffers=844592, cached=7766532, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.95, five_min_avg=2.68, fifteen_min_avg=2.93, cpu_share=2.38, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:39:08.301 7f1c50933c18 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:39:16.949 7f1c50956cc0 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:39:21.794 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50975728, partition_space_available_1k=30114756, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=101048, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83972, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88924, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188496, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58488, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14838236, free=9194948, buffers=844592, cached=7787336, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.71, five_min_avg=3.43, fifteen_min_avg=3.18, cpu_share=4.0, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:39:28.240 7f1c530df438 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:40:21.796 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50971524, partition_space_available_1k=30118960, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=101116, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83972, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88924, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188508, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58492, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14842892, free=9190496, buffers=844596, cached=7787128, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.95, five_min_avg=3.14, fifteen_min_avg=3.09, cpu_share=2.38, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:41:21.807 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50976308, partition_space_available_1k=30114176, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=101132, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83972, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88924, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188516, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58492, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14806336, free=9220024, buffers=844596, cached=7794156, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.42, five_min_avg=2.95, fifteen_min_avg=3.03, cpu_share=2.0, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:42:05.409 7f1c4f4b1978 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:42:21.817 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50980232, partition_space_available_1k=30110252, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=101168, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83968, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88932, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188528, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58496, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14811888, free=9208688, buffers=844596, cached=7799940, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.73, five_min_avg=2.93, fifteen_min_avg=3.02, cpu_share=2.25, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:43:21.826 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50983888, partition_space_available_1k=30106596, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=101236, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83968, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88928, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188536, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58500, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14797124, free=9218908, buffers=844596, cached=7804484, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.32, five_min_avg=2.78, fifteen_min_avg=2.96, cpu_share=1.88, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:44:15.262 7f1c50877ac8 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:44:21.845 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50987564, partition_space_available_1k=30102920, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=101244, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83972, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88928, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188548, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58500, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14859320, free=9151960, buffers=844596, cached=7809236, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.05, five_min_avg=2.85, fifteen_min_avg=2.97, cpu_share=2.12, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:44:23.905 7f1c50956198 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:44:38.110 7f1c508af438 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:45:21.847 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50993112, partition_space_available_1k=30097372, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=101284, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83972, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88928, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188560, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58504, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14896192, free=9101196, buffers=844596, cached=7823128, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.55, five_min_avg=3.48, fifteen_min_avg=3.19, cpu_share=4.12, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:46:21.854 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50997888, partition_space_available_1k=30092596, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=101560, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83972, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88928, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188572, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58500, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14943468, free=9047780, buffers=844596, cached=7829268, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.22, five_min_avg=3.28, fifteen_min_avg=3.13, cpu_share=2.5, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:46:47.125 7f1c50093eb8 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:47:21.857 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50997080, partition_space_available_1k=30093404, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=101664, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83972, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88932, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188580, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58504, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14959252, free=9031936, buffers=844596, cached=7829328, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.32, five_min_avg=3.01, fifteen_min_avg=3.05, cpu_share=2.5, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:47:52.401 7f1c52f23780 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connection Closed 03/31/2025 02:47:52.401 7f1c509374e0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpClose] in state[Established] 03/31/2025 02:47:57.406 7f1c509374e0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:47:57.406 7f1c509374e0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpClose] => State[Established] -> State[Connect] 03/31/2025 02:47:57.406 7f1c509374e0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:47:57.406 7f1c509374e0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Established to Connect on EvTcpClose >>, ] description = Collector:Collector[Established to Connect on EvTcpClose] connection down >>, ] >> 03/31/2025 02:47:57.406 7f1c509374e0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:47:57.406 7f1c509374e0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:47:57.407 7f1c509374e0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:47:57.407 7f1c509374e0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:47:57.407 7f1c509374e0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:47:57.407 7f1c509374e0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:47:57.407 7f1c509374e0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:47:57.407 7f1c509374e0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:47:57.407 7f1c509374e0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:01.410 7f1c509374e0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:48:06.413 7f1c509374e0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:48:06.413 7f1c509374e0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:48:06.413 7f1c509374e0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:48:06.413 7f1c509374e0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:48:06.413 7f1c509374e0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:48:06.413 7f1c509374e0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:48:06.414 7f1c509374e0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:48:06.414 7f1c509374e0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:06.414 7f1c509374e0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:48:06.414 7f1c509374e0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:06.414 7f1c509374e0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:06.414 7f1c509374e0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:48:06.414 7f1c509374e0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:10.418 7f1c508b8c18 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:48:15.421 7f1c508b8c18 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:48:15.421 7f1c508b8c18 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:48:15.421 7f1c508b8c18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:48:15.422 7f1c508b8c18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:48:15.422 7f1c508b8c18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:48:15.422 7f1c508b8c18 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:48:15.422 7f1c508b8c18 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:48:15.422 7f1c508b8c18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:15.422 7f1c508b8c18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:48:15.423 7f1c508b8c18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:15.423 7f1c508b8c18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:15.423 7f1c508b8c18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:48:15.423 7f1c508b8c18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:19.423 7f1c50933780 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:48:21.864 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51000832, partition_space_available_1k=30089652, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=101700, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83972, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88932, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188588, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58552, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15002568, free=8983704, buffers=844596, cached=7834244, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.99, five_min_avg=2.75, fifteen_min_avg=2.95, cpu_share=2.38, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:48:24.426 7f1c50933780 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:48:24.427 7f1c50933780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:48:24.427 7f1c50933780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:48:24.427 7f1c50933780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:48:24.427 7f1c50933780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:48:24.427 7f1c50933780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:48:24.427 7f1c50933780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51000832 partition_space_available_1k = 30089652 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 101700 cpu_share = 0 mem_res = 0 >>contrail-schema : << mem_virt = 83972 cpu_share = 0 mem_res = 0 >>contrail-svc-monitor : << mem_virt = 88932 cpu_share = 0 mem_res = 0 >>contrail-device-manager : << mem_virt = 188588 cpu_share = 0 mem_res = 0 >>contrail-config-nodemgr : << mem_virt = 58552 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 15002568 free = 8983704 buffers = 844596 cached = 7834244 node_type = config-node >> system_cpu_usage = << one_min_avg = 1.99 five_min_avg = 2.75 fifteen_min_avg = 2.95 cpu_share = 2.38 node_type = config-node >> >> 03/31/2025 02:48:24.427 7f1c50933780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:48:24.427 7f1c50933780 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:48:24.428 7f1c50933780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:48:24.428 7f1c50933780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:24.428 7f1c50933780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:48:24.428 7f1c50933780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:24.428 7f1c50933780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:24.428 7f1c50933780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:48:24.428 7f1c50933780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:28.429 7f1c50933780 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:48:33.434 7f1c50933780 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:48:33.435 7f1c50933780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:48:33.435 7f1c50933780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:48:33.435 7f1c50933780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:48:33.435 7f1c50933780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:48:33.435 7f1c50933780 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:48:33.435 7f1c50933780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:48:33.436 7f1c50933780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:33.436 7f1c50933780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:48:33.436 7f1c50933780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:33.436 7f1c50933780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:33.436 7f1c50933780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:48:33.436 7f1c50933780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:37.437 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:48:42.441 7f1c52bbd390 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:48:42.441 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:48:42.441 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:48:42.441 7f1c52bbd390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:48:42.441 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:48:42.441 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:48:42.442 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:48:42.442 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:42.442 7f1c52bbd390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:48:42.442 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:42.442 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:42.442 7f1c52bbd390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:48:42.442 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:46.445 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:48:51.447 7f1c52bbd390 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:48:51.447 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:48:51.448 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:48:51.448 7f1c52bbd390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:48:51.448 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:48:51.448 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:48:51.449 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:48:51.449 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:51.449 7f1c52bbd390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:48:51.449 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:51.449 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:51.449 7f1c52bbd390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:48:51.449 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:55.450 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:48:59.362 7f1c4f4a9780 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:49:00.453 7f1c52bbd390 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:49:00.453 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:49:00.453 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:49:00.454 7f1c52bbd390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:49:00.454 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:49:00.454 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:49:00.454 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:49:00.454 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:00.454 7f1c52bbd390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:49:00.454 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:00.454 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:00.455 7f1c52bbd390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:49:00.455 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:04.457 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:49:07.993 7f1c50956630 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:49:09.459 7f1c52bbd390 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:49:09.460 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:49:09.460 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:49:09.460 7f1c52bbd390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:49:09.460 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:49:09.460 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:49:09.460 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:49:09.461 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:09.461 7f1c52bbd390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:49:09.461 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:09.461 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:09.461 7f1c52bbd390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:49:09.461 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:13.461 7f1c50877240 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:49:18.463 7f1c50877240 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:49:18.464 7f1c50877240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:49:18.464 7f1c50877240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:49:18.464 7f1c50877240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:49:18.464 7f1c50877240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:49:18.464 7f1c50877240 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:49:18.465 7f1c50877240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:49:18.465 7f1c50877240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:18.465 7f1c50877240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:49:18.465 7f1c50877240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:18.465 7f1c50877240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:18.465 7f1c50877240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:49:18.465 7f1c50877240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:19.217 7f1c50937c18 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:49:21.880 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51004196, partition_space_available_1k=30086288, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=101644, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83972, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88940, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188600, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58540, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15070120, free=8910540, buffers=844604, cached=7839848, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.13, five_min_avg=2.65, fifteen_min_avg=2.91, cpu_share=2.5, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:49:21.881 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:21.881 7f1c59f230f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51004196 partition_space_available_1k = 30086288 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 101644 cpu_share = 0 mem_res = 0 >>contrail-schema : << mem_virt = 83972 cpu_share = 0 mem_res = 0 >>contrail-svc-monitor : << mem_virt = 88940 cpu_share = 0 mem_res = 0 >>contrail-device-manager : << mem_virt = 188600 cpu_share = 0 mem_res = 0 >>contrail-config-nodemgr : << mem_virt = 58540 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 15070120 free = 8910540 buffers = 844604 cached = 7839848 node_type = config-node >> system_cpu_usage = << one_min_avg = 2.13 five_min_avg = 2.65 fifteen_min_avg = 2.91 cpu_share = 2.5 node_type = config-node >> >> 03/31/2025 02:49:21.881 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:22.465 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:49:27.469 7f1c59f230f0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:49:27.469 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:49:27.469 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:49:27.469 7f1c59f230f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:49:27.469 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:49:27.469 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:49:27.470 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:49:27.470 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:27.470 7f1c59f230f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:49:27.470 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:27.470 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:27.470 7f1c59f230f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:49:27.470 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:31.473 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:49:36.475 7f1c59f230f0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:49:36.475 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:49:36.476 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:49:36.476 7f1c59f230f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:49:36.476 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:49:36.476 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:49:36.476 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:49:36.476 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:36.476 7f1c59f230f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:49:36.477 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:36.477 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:36.477 7f1c59f230f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:49:36.477 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:40.478 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:49:45.482 7f1c59f230f0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:49:45.482 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:49:45.483 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:49:45.483 7f1c59f230f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:49:45.483 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:49:45.483 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:49:45.483 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:49:45.483 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:45.483 7f1c59f230f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:49:45.484 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:45.484 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:45.484 7f1c59f230f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:49:45.484 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:49.483 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:49:54.487 7f1c59f230f0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:49:54.487 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:49:54.488 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:49:54.488 7f1c59f230f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:49:54.488 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:49:54.488 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:49:54.488 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:49:54.489 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:54.489 7f1c59f230f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:49:54.489 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:54.489 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:54.489 7f1c59f230f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:49:54.489 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:58.491 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:50:03.498 7f1c59f230f0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:50:03.498 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:50:03.498 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:50:03.498 7f1c59f230f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:50:03.498 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:50:03.498 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:50:03.499 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:50:03.499 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:03.499 7f1c59f230f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:50:03.499 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:03.499 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:03.499 7f1c59f230f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:50:03.499 7f1c59f230f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:07.499 7f1c50877a20 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:50:12.505 7f1c50877a20 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:50:12.505 7f1c50877a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:50:12.505 7f1c50877a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:50:12.505 7f1c50877a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:50:12.505 7f1c50877a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:50:12.505 7f1c50877a20 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:50:12.506 7f1c50877a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:50:12.507 7f1c50877a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:12.507 7f1c50877a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:50:12.507 7f1c50877a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:12.507 7f1c50877a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:12.507 7f1c50877a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:50:12.507 7f1c50877a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:16.505 7f1c50881828 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:50:21.551 7f1c50881828 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:50:21.552 7f1c50881828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:50:21.552 7f1c50881828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:50:21.552 7f1c50881828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:50:21.552 7f1c50881828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:50:21.552 7f1c50881828 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:50:21.552 7f1c50881828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:50:21.553 7f1c50881828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:21.553 7f1c50881828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:50:21.553 7f1c50881828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:21.553 7f1c50881828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:21.553 7f1c50881828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:50:21.553 7f1c50881828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:21.940 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51004348, partition_space_available_1k=30086136, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=101772, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83972, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88940, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188608, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58560, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15412144, free=8558328, buffers=844608, cached=7850032, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.78, five_min_avg=2.93, fifteen_min_avg=2.98, cpu_share=2.88, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:50:21.945 7f1c5093ed68 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:21.946 7f1c5093ed68 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51004348 partition_space_available_1k = 30086136 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 101772 cpu_share = 0 mem_res = 0 >>contrail-schema : << mem_virt = 83972 cpu_share = 0 mem_res = 0 >>contrail-svc-monitor : << mem_virt = 88940 cpu_share = 0 mem_res = 0 >>contrail-device-manager : << mem_virt = 188608 cpu_share = 0 mem_res = 0 >>contrail-config-nodemgr : << mem_virt = 58560 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 15412144 free = 8558328 buffers = 844608 cached = 7850032 node_type = config-node >> system_cpu_usage = << one_min_avg = 3.78 five_min_avg = 2.93 fifteen_min_avg = 2.98 cpu_share = 2.88 node_type = config-node >> >> 03/31/2025 02:50:21.946 7f1c5093ed68 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:25.553 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:50:30.560 7f1c52ed8780 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:50:30.560 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:50:30.560 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:50:30.560 7f1c52ed8780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:50:30.560 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:50:30.560 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:50:30.561 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:50:30.561 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:30.561 7f1c52ed8780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:50:30.561 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:30.561 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:30.561 7f1c52ed8780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:50:30.561 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:34.564 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:50:39.566 7f1c52ed8780 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:50:39.567 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:50:39.567 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:50:39.567 7f1c52ed8780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:50:39.567 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:50:39.567 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:50:39.568 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:50:39.568 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:39.568 7f1c52ed8780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:50:39.568 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:39.568 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:39.568 7f1c52ed8780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:50:39.568 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:43.570 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:50:48.185 7f1c4f4a9828 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:50:48.572 7f1c52ed8780 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:50:48.572 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:50:48.572 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:50:48.573 7f1c52ed8780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:50:48.573 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:50:48.573 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:50:48.573 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:50:48.573 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:48.574 7f1c52ed8780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:50:48.574 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:48.574 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:48.574 7f1c52ed8780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:50:48.574 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:52.573 7f1c52f518d0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:50:57.579 7f1c52f518d0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:50:57.579 7f1c52f518d0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:50:57.580 7f1c52f518d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:50:57.580 7f1c52f518d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:50:57.580 7f1c52f518d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:50:57.580 7f1c52f518d0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:50:57.580 7f1c52f518d0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:50:57.581 7f1c52f518d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:57.581 7f1c52f518d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:50:57.581 7f1c52f518d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:57.581 7f1c52f518d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:57.581 7f1c52f518d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:50:57.581 7f1c52f518d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:01.581 7f1c52f518d0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:51:06.588 7f1c52f518d0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:51:06.588 7f1c52f518d0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:51:06.588 7f1c52f518d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:51:06.588 7f1c52f518d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:51:06.588 7f1c52f518d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:51:06.588 7f1c52f518d0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:51:06.589 7f1c52f518d0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:51:06.589 7f1c52f518d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:06.589 7f1c52f518d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:51:06.589 7f1c52f518d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:06.589 7f1c52f518d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:06.589 7f1c52f518d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:51:06.589 7f1c52f518d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:10.591 7f1c5134e780 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:51:15.593 7f1c5134e780 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:51:15.593 7f1c5134e780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:51:15.593 7f1c5134e780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:51:15.594 7f1c5134e780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:51:15.594 7f1c5134e780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:51:15.594 7f1c5134e780 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:51:15.594 7f1c5134e780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:51:15.594 7f1c5134e780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:15.594 7f1c5134e780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:51:15.595 7f1c5134e780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:15.595 7f1c5134e780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:15.595 7f1c5134e780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:51:15.595 7f1c5134e780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:19.595 7f1c545df198 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:51:21.868 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51014352, partition_space_available_1k=30076132, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=101692, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83972, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88940, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188616, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58620, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15015880, free=8945852, buffers=844612, cached=7858768, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.82, five_min_avg=2.89, fifteen_min_avg=2.97, cpu_share=3.75, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:51:24.599 7f1c545df198 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:51:24.599 7f1c545df198 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:51:24.599 7f1c545df198 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:51:24.599 7f1c545df198 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:51:24.599 7f1c545df198 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:51:24.599 7f1c545df198 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:51:24.600 7f1c545df198 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51014352 partition_space_available_1k = 30076132 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 101692 cpu_share = 0 mem_res = 0 >>contrail-schema : << mem_virt = 83972 cpu_share = 0 mem_res = 0 >>contrail-svc-monitor : << mem_virt = 88940 cpu_share = 0 mem_res = 0 >>contrail-device-manager : << mem_virt = 188616 cpu_share = 0 mem_res = 0 >>contrail-config-nodemgr : << mem_virt = 58620 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 15015880 free = 8945852 buffers = 844612 cached = 7858768 node_type = config-node >> system_cpu_usage = << one_min_avg = 2.82 five_min_avg = 2.89 fifteen_min_avg = 2.97 cpu_share = 3.75 node_type = config-node >> >> 03/31/2025 02:51:24.600 7f1c545df198 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:51:24.600 7f1c545df198 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:51:24.600 7f1c545df198 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:51:24.600 7f1c545df198 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:24.600 7f1c545df198 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:51:24.600 7f1c545df198 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:24.601 7f1c545df198 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:24.601 7f1c545df198 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:51:24.601 7f1c545df198 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:28.602 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:51:33.606 7f1c52aef780 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:51:33.606 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:51:33.606 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:51:33.606 7f1c52aef780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:51:33.606 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:51:33.606 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:51:33.607 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:51:33.607 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:33.607 7f1c52aef780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:51:33.607 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:33.607 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:33.607 7f1c52aef780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:51:33.607 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:37.606 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:51:42.615 7f1c52ed8780 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:51:42.615 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:51:42.616 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:51:42.616 7f1c52ed8780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:51:42.616 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:51:42.616 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:51:42.617 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:51:42.617 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:42.617 7f1c52ed8780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:51:42.617 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:42.617 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:42.617 7f1c52ed8780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:51:42.617 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:46.619 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:51:51.625 7f1c52ed8780 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:51:51.626 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:51:51.626 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:51:51.626 7f1c52ed8780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:51:51.626 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:51:51.626 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:51:51.626 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:51:51.627 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:51.627 7f1c52ed8780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:51:51.627 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:51.627 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:51.627 7f1c52ed8780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:51:51.627 7f1c52ed8780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:55.628 7f1c52b83588 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:52:00.634 7f1c52b83588 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:52:00.635 7f1c52b83588 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:52:00.635 7f1c52b83588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:52:00.635 7f1c52b83588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:52:00.635 7f1c52b83588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:52:00.635 7f1c52b83588 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:52:00.636 7f1c52b83588 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:52:00.636 7f1c52b83588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:00.636 7f1c52b83588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:52:00.636 7f1c52b83588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:00.636 7f1c52b83588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:00.636 7f1c52b83588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:52:00.636 7f1c52b83588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:04.639 7f1c52b83588 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:52:09.641 7f1c52b83588 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:52:09.641 7f1c52b83588 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:52:09.641 7f1c52b83588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:52:09.642 7f1c52b83588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:52:09.642 7f1c52b83588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:52:09.642 7f1c52b83588 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:52:09.642 7f1c52b83588 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:52:09.642 7f1c52b83588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:09.642 7f1c52b83588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:52:09.643 7f1c52b83588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:09.643 7f1c52b83588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:09.643 7f1c52b83588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:52:09.643 7f1c52b83588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:13.642 7f1c52b83588 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:52:18.645 7f1c52b83588 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:52:18.646 7f1c52b83588 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:52:18.646 7f1c52b83588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:52:18.646 7f1c52b83588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:52:18.646 7f1c52b83588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:52:18.646 7f1c52b83588 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:52:18.646 7f1c52b83588 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:52:18.647 7f1c52b83588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:18.647 7f1c52b83588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:52:18.647 7f1c52b83588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:18.647 7f1c52b83588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:18.647 7f1c52b83588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:52:18.647 7f1c52b83588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:21.876 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51014044, partition_space_available_1k=30076440, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=101760, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83972, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88940, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188628, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58604, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15036256, free=8922128, buffers=844612, cached=7862116, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.36, five_min_avg=2.73, fifteen_min_avg=2.91, cpu_share=2.62, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:52:21.876 7f1c509338d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:21.877 7f1c509338d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51014044 partition_space_available_1k = 30076440 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 101760 cpu_share = 0 mem_res = 0 >>contrail-schema : << mem_virt = 83972 cpu_share = 0 mem_res = 0 >>contrail-svc-monitor : << mem_virt = 88940 cpu_share = 0 mem_res = 0 >>contrail-device-manager : << mem_virt = 188628 cpu_share = 0 mem_res = 0 >>contrail-config-nodemgr : << mem_virt = 58604 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 15036256 free = 8922128 buffers = 844612 cached = 7862116 node_type = config-node >> system_cpu_usage = << one_min_avg = 2.36 five_min_avg = 2.73 fifteen_min_avg = 2.91 cpu_share = 2.62 node_type = config-node >> >> 03/31/2025 02:52:21.877 7f1c509338d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:22.647 7f1c509338d0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:52:27.652 7f1c509338d0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:52:27.653 7f1c509338d0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:52:27.653 7f1c509338d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:52:27.653 7f1c509338d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:52:27.653 7f1c509338d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:52:27.653 7f1c509338d0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:52:27.654 7f1c509338d0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:52:27.654 7f1c509338d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:27.654 7f1c509338d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:52:27.654 7f1c509338d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:27.654 7f1c509338d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:27.654 7f1c509338d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:52:27.654 7f1c509338d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:31.657 7f1c5093e828 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:52:36.662 7f1c5093e828 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:52:36.662 7f1c5093e828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:52:36.663 7f1c5093e828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:52:36.663 7f1c5093e828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:52:36.663 7f1c5093e828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:52:36.663 7f1c5093e828 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:52:36.663 7f1c5093e828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:52:36.663 7f1c5093e828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:36.663 7f1c5093e828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:52:36.663 7f1c5093e828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:36.664 7f1c5093e828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:36.664 7f1c5093e828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:52:36.664 7f1c5093e828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:40.665 7f1c509566d8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:52:45.671 7f1c509566d8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:52:45.671 7f1c509566d8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:52:45.672 7f1c509566d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:52:45.672 7f1c509566d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:52:45.672 7f1c509566d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:52:45.672 7f1c509566d8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:52:45.672 7f1c509566d8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:52:45.672 7f1c509566d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:45.672 7f1c509566d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:52:45.673 7f1c509566d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:45.673 7f1c509566d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:45.673 7f1c509566d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:52:45.673 7f1c509566d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:49.673 7f1c509566d8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:52:54.675 7f1c509566d8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:52:54.676 7f1c509566d8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:52:54.676 7f1c509566d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:52:54.676 7f1c509566d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:52:54.676 7f1c509566d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:52:54.676 7f1c509566d8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:52:54.676 7f1c509566d8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:52:54.677 7f1c509566d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:54.677 7f1c509566d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:52:54.677 7f1c509566d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:54.677 7f1c509566d8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:54.677 7f1c509566d8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:52:54.677 7f1c509566d8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:58.680 7f1c545df390 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:53:03.686 7f1c545df390 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:53:03.687 7f1c545df390 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:53:03.687 7f1c545df390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:53:03.687 7f1c545df390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:53:03.687 7f1c545df390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:53:03.687 7f1c545df390 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:53:03.688 7f1c545df390 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:53:03.688 7f1c545df390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:03.688 7f1c545df390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:53:03.688 7f1c545df390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:03.688 7f1c545df390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:03.688 7f1c545df390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:53:03.688 7f1c545df390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:07.691 7f1c545df390 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:53:12.696 7f1c545df390 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:53:12.697 7f1c545df390 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:53:12.697 7f1c545df390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:53:12.697 7f1c545df390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:53:12.697 7f1c545df390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:53:12.697 7f1c545df390 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:53:12.697 7f1c545df390 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:53:12.698 7f1c545df390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:12.698 7f1c545df390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:53:12.698 7f1c545df390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:12.698 7f1c545df390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:12.698 7f1c545df390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:53:12.698 7f1c545df390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:16.698 7f1c545df390 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:53:21.701 7f1c545df390 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:53:21.702 7f1c545df390 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:53:21.702 7f1c545df390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:53:21.702 7f1c545df390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:53:21.702 7f1c545df390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:53:21.702 7f1c545df390 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:53:21.703 7f1c545df390 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:53:21.703 7f1c545df390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:21.703 7f1c545df390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:53:21.703 7f1c545df390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:21.703 7f1c545df390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:21.703 7f1c545df390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:53:21.703 7f1c545df390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:21.877 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51017668, partition_space_available_1k=30072816, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=101764, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83972, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88944, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188636, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58628, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15026756, free=8927264, buffers=844612, cached=7866480, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.79, five_min_avg=2.48, fifteen_min_avg=2.81, cpu_share=2.5, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:53:21.878 7f1c50956e10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:21.878 7f1c50956e10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51017668 partition_space_available_1k = 30072816 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 101764 cpu_share = 0 mem_res = 0 >>contrail-schema : << mem_virt = 83972 cpu_share = 0 mem_res = 0 >>contrail-svc-monitor : << mem_virt = 88944 cpu_share = 0 mem_res = 0 >>contrail-device-manager : << mem_virt = 188636 cpu_share = 0 mem_res = 0 >>contrail-config-nodemgr : << mem_virt = 58628 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 15026756 free = 8927264 buffers = 844612 cached = 7866480 node_type = config-node >> system_cpu_usage = << one_min_avg = 1.79 five_min_avg = 2.48 fifteen_min_avg = 2.81 cpu_share = 2.5 node_type = config-node >> >> 03/31/2025 02:53:21.878 7f1c50956e10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:25.704 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:53:30.709 7f1c5007f978 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:53:30.710 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:53:30.710 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:53:30.710 7f1c5007f978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:53:30.710 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:53:30.710 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:53:30.710 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:53:30.711 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:30.711 7f1c5007f978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:53:30.711 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:30.711 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:30.711 7f1c5007f978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:53:30.711 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:34.713 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:53:39.715 7f1c5007f978 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:53:39.715 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:53:39.715 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:53:39.715 7f1c5007f978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:53:39.715 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:53:39.715 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:53:39.716 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:53:39.716 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:39.716 7f1c5007f978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:53:39.716 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:39.716 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:39.716 7f1c5007f978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:53:39.716 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:43.718 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:53:48.724 7f1c5007f978 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:53:48.724 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:53:48.724 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:53:48.725 7f1c5007f978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:53:48.725 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:53:48.725 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:53:48.725 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:53:48.725 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:48.725 7f1c5007f978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:53:48.725 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:48.725 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:48.726 7f1c5007f978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:53:48.726 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:52.729 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:53:57.733 7f1c5007f978 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:53:57.733 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:53:57.733 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:53:57.734 7f1c5007f978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:53:57.734 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:53:57.734 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:53:57.734 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:53:57.734 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:57.735 7f1c5007f978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:53:57.735 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:57.735 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:57.735 7f1c5007f978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:53:57.735 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:01.737 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:54:06.742 7f1c5007f978 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:54:06.743 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:54:06.743 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:54:06.743 7f1c5007f978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:54:06.743 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:54:06.743 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:54:06.743 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:54:06.744 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:06.744 7f1c5007f978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:54:06.744 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:06.744 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:06.744 7f1c5007f978 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:54:06.744 7f1c5007f978 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:10.743 7f1c5088ae10 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:54:12.321 7f1c5093e438 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:54:15.746 7f1c5088ae10 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:54:15.747 7f1c5088ae10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:54:15.747 7f1c5088ae10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:54:15.747 7f1c5088ae10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:54:15.747 7f1c5088ae10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:54:15.747 7f1c5088ae10 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:54:15.748 7f1c5088ae10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:54:15.748 7f1c5088ae10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:15.748 7f1c5088ae10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:54:15.748 7f1c5088ae10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:15.748 7f1c5088ae10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:15.748 7f1c5088ae10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:54:15.748 7f1c5088ae10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:19.748 7f1c509308d0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:54:20.935 7f1c50881ac8 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:54:21.893 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51021164, partition_space_available_1k=30069320, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=101844, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83976, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88944, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188644, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58680, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15097676, free=8852288, buffers=844612, cached=7870536, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.04, five_min_avg=2.41, fifteen_min_avg=2.76, cpu_share=2.62, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:54:24.750 7f1c509308d0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:54:24.751 7f1c509308d0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:54:24.751 7f1c509308d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:54:24.751 7f1c509308d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:54:24.751 7f1c509308d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:54:24.751 7f1c509308d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:54:24.751 7f1c509308d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51021164 partition_space_available_1k = 30069320 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 101844 cpu_share = 0 mem_res = 0 >>contrail-schema : << mem_virt = 83976 cpu_share = 0 mem_res = 0 >>contrail-svc-monitor : << mem_virt = 88944 cpu_share = 0 mem_res = 0 >>contrail-device-manager : << mem_virt = 188644 cpu_share = 0 mem_res = 0 >>contrail-config-nodemgr : << mem_virt = 58680 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 15097676 free = 8852288 buffers = 844612 cached = 7870536 node_type = config-node >> system_cpu_usage = << one_min_avg = 2.04 five_min_avg = 2.41 fifteen_min_avg = 2.76 cpu_share = 2.62 node_type = config-node >> >> 03/31/2025 02:54:24.751 7f1c509308d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:54:24.751 7f1c509308d0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:54:24.752 7f1c509308d0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:54:24.752 7f1c509308d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:24.752 7f1c509308d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:54:24.752 7f1c509308d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:24.752 7f1c509308d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:24.752 7f1c509308d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:54:24.752 7f1c509308d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:28.754 7f1c508b3e10 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:54:32.142 7f1c50933c18 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:54:33.757 7f1c508b3e10 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:54:33.758 7f1c508b3e10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:54:33.758 7f1c508b3e10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:54:33.758 7f1c508b3e10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:54:33.758 7f1c508b3e10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:54:33.758 7f1c508b3e10 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:54:33.758 7f1c508b3e10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:54:33.759 7f1c508b3e10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:33.759 7f1c508b3e10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:54:33.759 7f1c508b3e10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:33.759 7f1c508b3e10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:33.759 7f1c508b3e10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:54:33.759 7f1c508b3e10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:37.762 7f1c508b3e10 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:54:42.767 7f1c508b3e10 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:54:42.767 7f1c508b3e10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:54:42.768 7f1c508b3e10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:54:42.768 7f1c508b3e10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:54:42.768 7f1c508b3e10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:54:42.768 7f1c508b3e10 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:54:42.768 7f1c508b3e10 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:54:42.768 7f1c508b3e10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:42.768 7f1c508b3e10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:54:42.768 7f1c508b3e10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:42.769 7f1c508b3e10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:42.769 7f1c508b3e10 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:54:42.769 7f1c508b3e10 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:46.769 7f1c50930198 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:54:51.773 7f1c50930198 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:54:51.773 7f1c50930198 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:54:51.773 7f1c50930198 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:54:51.773 7f1c50930198 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:54:51.773 7f1c50930198 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:54:51.773 7f1c50930198 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:54:51.774 7f1c50930198 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:54:51.774 7f1c50930198 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:51.774 7f1c50930198 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:54:51.774 7f1c50930198 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:51.774 7f1c50930198 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:51.774 7f1c50930198 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:54:51.774 7f1c50930198 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:55.774 7f1c59c1aeb8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:55:00.781 7f1c59c1aeb8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:55:00.781 7f1c59c1aeb8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:55:00.781 7f1c59c1aeb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:55:00.781 7f1c59c1aeb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:55:00.781 7f1c59c1aeb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:55:00.781 7f1c59c1aeb8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:55:00.782 7f1c59c1aeb8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:55:00.782 7f1c59c1aeb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:00.782 7f1c59c1aeb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:55:00.782 7f1c59c1aeb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:00.782 7f1c59c1aeb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:00.782 7f1c59c1aeb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:55:00.782 7f1c59c1aeb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:04.785 7f1c59c1aeb8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:55:09.788 7f1c59c1aeb8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:55:09.788 7f1c59c1aeb8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:55:09.788 7f1c59c1aeb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:55:09.788 7f1c59c1aeb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:55:09.788 7f1c59c1aeb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:55:09.788 7f1c59c1aeb8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:55:09.789 7f1c59c1aeb8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:55:09.789 7f1c59c1aeb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:09.789 7f1c59c1aeb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:55:09.789 7f1c59c1aeb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:09.790 7f1c59c1aeb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:09.790 7f1c59c1aeb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:55:09.790 7f1c59c1aeb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:13.789 7f1c59c1aeb8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:55:18.793 7f1c59c1aeb8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:55:18.793 7f1c59c1aeb8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:55:18.793 7f1c59c1aeb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:55:18.793 7f1c59c1aeb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:55:18.793 7f1c59c1aeb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:55:18.793 7f1c59c1aeb8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:55:18.794 7f1c59c1aeb8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:55:18.794 7f1c59c1aeb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:18.794 7f1c59c1aeb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:55:18.794 7f1c59c1aeb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:18.794 7f1c59c1aeb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:18.794 7f1c59c1aeb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:55:18.794 7f1c59c1aeb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:21.891 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51021620, partition_space_available_1k=30068864, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=102116, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83976, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88944, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188660, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58676, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15026736, free=8921044, buffers=844612, cached=7872720, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.78, five_min_avg=2.27, fifteen_min_avg=2.69, cpu_share=2.62, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:55:21.891 7f1c508b82e8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:21.891 7f1c508b82e8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51021620 partition_space_available_1k = 30068864 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 102116 cpu_share = 0 mem_res = 0 >>contrail-schema : << mem_virt = 83976 cpu_share = 0 mem_res = 0 >>contrail-svc-monitor : << mem_virt = 88944 cpu_share = 0 mem_res = 0 >>contrail-device-manager : << mem_virt = 188660 cpu_share = 0 mem_res = 0 >>contrail-config-nodemgr : << mem_virt = 58676 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 15026736 free = 8921044 buffers = 844612 cached = 7872720 node_type = config-node >> system_cpu_usage = << one_min_avg = 1.78 five_min_avg = 2.27 fifteen_min_avg = 2.69 cpu_share = 2.62 node_type = config-node >> >> 03/31/2025 02:55:21.891 7f1c508b82e8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:22.795 7f1c508b82e8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:55:27.797 7f1c508b82e8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:55:27.797 7f1c508b82e8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:55:27.797 7f1c508b82e8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:55:27.797 7f1c508b82e8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:55:27.797 7f1c508b82e8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:55:27.797 7f1c508b82e8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:55:27.798 7f1c508b82e8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:55:27.798 7f1c508b82e8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:27.798 7f1c508b82e8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:55:27.798 7f1c508b82e8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:27.798 7f1c508b82e8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:27.798 7f1c508b82e8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:55:27.798 7f1c508b82e8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:31.802 7f1c52b83588 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:55:36.809 7f1c52b83588 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:55:36.809 7f1c52b83588 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:55:36.809 7f1c52b83588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:55:36.809 7f1c52b83588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:55:36.809 7f1c52b83588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:55:36.810 7f1c52b83588 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:55:36.810 7f1c52b83588 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:55:36.810 7f1c52b83588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:36.810 7f1c52b83588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:55:36.811 7f1c52b83588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:36.811 7f1c52b83588 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:36.811 7f1c52b83588 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:55:36.811 7f1c52b83588 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:40.811 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:55:45.818 7f1c52bbd390 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:55:45.818 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:55:45.818 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:55:45.818 7f1c52bbd390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:55:45.818 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:55:45.818 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:55:45.819 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:55:45.819 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:45.819 7f1c52bbd390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:55:45.819 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:45.819 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:45.819 7f1c52bbd390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:55:45.819 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:49.821 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:55:54.825 7f1c52bbd390 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:55:54.825 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:55:54.825 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:55:54.825 7f1c52bbd390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:55:54.825 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:55:54.825 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:55:54.826 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:55:54.826 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:54.826 7f1c52bbd390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:55:54.826 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:54.826 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:54.826 7f1c52bbd390 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:55:54.826 7f1c52bbd390 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:58.828 7f1c508570f0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:56:03.834 7f1c508570f0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:56:03.834 7f1c508570f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:56:03.834 7f1c508570f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:56:03.834 7f1c508570f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:56:03.834 7f1c508570f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:56:03.834 7f1c508570f0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:56:03.835 7f1c508570f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:56:03.835 7f1c508570f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:03.835 7f1c508570f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:56:03.835 7f1c508570f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:03.835 7f1c508570f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:03.835 7f1c508570f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:56:03.835 7f1c508570f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:07.838 7f1c508570f0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:56:12.840 7f1c508570f0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:56:12.841 7f1c508570f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:56:12.841 7f1c508570f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:56:12.841 7f1c508570f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:56:12.841 7f1c508570f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:56:12.841 7f1c508570f0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:56:12.842 7f1c508570f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:56:12.842 7f1c508570f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:12.842 7f1c508570f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:56:12.842 7f1c508570f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:12.842 7f1c508570f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:12.842 7f1c508570f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:56:12.842 7f1c508570f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:16.841 7f1c50093ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:56:21.845 7f1c50093ac8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:56:21.845 7f1c50093ac8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:56:21.845 7f1c50093ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:56:21.845 7f1c50093ac8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:56:21.845 7f1c50093ac8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:56:21.845 7f1c50093ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:56:21.846 7f1c50093ac8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:56:21.846 7f1c50093ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:21.846 7f1c50093ac8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:56:21.846 7f1c50093ac8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:21.846 7f1c50093ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:21.846 7f1c50093ac8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:56:21.846 7f1c50093ac8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:21.881 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51025984, partition_space_available_1k=30064500, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=102252, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83976, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88944, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188668, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58708, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15078836, free=8850444, buffers=844620, cached=7891212, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.79, five_min_avg=3.23, fifteen_min_avg=3.0, cpu_share=4.0, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:56:21.882 7f1c582e1780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:21.882 7f1c582e1780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51025984 partition_space_available_1k = 30064500 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 102252 cpu_share = 0 mem_res = 0 >>contrail-schema : << mem_virt = 83976 cpu_share = 0 mem_res = 0 >>contrail-svc-monitor : << mem_virt = 88944 cpu_share = 0 mem_res = 0 >>contrail-device-manager : << mem_virt = 188668 cpu_share = 0 mem_res = 0 >>contrail-config-nodemgr : << mem_virt = 58708 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 15078836 free = 8850444 buffers = 844620 cached = 7891212 node_type = config-node >> system_cpu_usage = << one_min_avg = 4.79 five_min_avg = 3.23 fifteen_min_avg = 3.0 cpu_share = 4.0 node_type = config-node >> >> 03/31/2025 02:56:21.882 7f1c582e1780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:25.845 7f1c545df240 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:56:30.850 7f1c545df240 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:56:30.850 7f1c545df240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:56:30.850 7f1c545df240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:56:30.850 7f1c545df240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:56:30.850 7f1c545df240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:56:30.850 7f1c545df240 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:56:30.851 7f1c545df240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:56:30.851 7f1c545df240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:30.851 7f1c545df240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:56:30.851 7f1c545df240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:30.851 7f1c545df240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:30.851 7f1c545df240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:56:30.851 7f1c545df240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:34.854 7f1c545df240 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:56:37.561 7f1c50856630 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:56:39.857 7f1c545df240 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:56:39.858 7f1c545df240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:56:39.858 7f1c545df240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:56:39.858 7f1c545df240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:56:39.858 7f1c545df240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:56:39.858 7f1c545df240 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:56:39.859 7f1c545df240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:56:39.859 7f1c545df240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:39.859 7f1c545df240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:56:39.859 7f1c545df240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:39.859 7f1c545df240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:39.859 7f1c545df240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:56:39.859 7f1c545df240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:43.861 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:56:48.867 7f1c52aef780 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:56:48.867 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:56:48.868 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:56:48.868 7f1c52aef780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:56:48.868 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:56:48.868 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:56:48.868 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:56:48.868 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:48.868 7f1c52aef780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:56:48.869 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:48.869 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:48.869 7f1c52aef780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:56:48.869 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:52.872 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:56:57.876 7f1c52aef780 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:56:57.876 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:56:57.876 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:56:57.876 7f1c52aef780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:56:57.876 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:56:57.877 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:56:57.877 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:56:57.877 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:57.877 7f1c52aef780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:56:57.877 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:57.877 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:57.877 7f1c52aef780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:56:57.878 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:57:01.880 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:57:06.886 7f1c52aef780 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:57:06.886 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:57:06.886 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:57:06.886 7f1c52aef780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:57:06.886 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:57:06.886 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:57:06.887 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:57:06.887 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:57:06.887 7f1c52aef780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:57:06.887 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:57:06.887 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:57:06.887 7f1c52aef780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:57:06.887 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:57:10.891 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:57:15.893 7f1c52aef780 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:57:15.893 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:57:15.893 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:57:15.894 7f1c52aef780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:57:15.894 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:57:15.894 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:57:15.894 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:57:15.894 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:57:15.895 7f1c52aef780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:57:15.895 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:57:15.895 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:57:15.895 7f1c52aef780 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:57:15.895 7f1c52aef780 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:57:19.893 7f1c4f4a9048 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:57:21.883 7f1c5312dd08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51030476, partition_space_available_1k=30060008, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=102164, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=83972, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88944, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=188676, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=58752, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15049536, free=8872076, buffers=844620, cached=7898880, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.71, five_min_avg=2.91, fifteen_min_avg=2.9, cpu_share=2.25, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/31/2025 02:57:24.898 7f1c4f4a9048 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:57:24.898 7f1c4f4a9048 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:57:24.898 7f1c4f4a9048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:57:24.898 7f1c4f4a9048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:57:24.898 7f1c4f4a9048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:57:24.898 7f1c4f4a9048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:57:24.899 7f1c4f4a9048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51030476 partition_space_available_1k = 30060008 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 102164 cpu_share = 0 mem_res = 0 >>contrail-schema : << mem_virt = 83972 cpu_share = 0 mem_res = 0 >>contrail-svc-monitor : << mem_virt = 88944 cpu_share = 0 mem_res = 0 >>contrail-device-manager : << mem_virt = 188676 cpu_share = 0 mem_res = 0 >>contrail-config-nodemgr : << mem_virt = 58752 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 15049536 free = 8872076 buffers = 844620 cached = 7898880 node_type = config-node >> system_cpu_usage = << one_min_avg = 2.71 five_min_avg = 2.91 fifteen_min_avg = 2.9 cpu_share = 2.25 node_type = config-node >> >> 03/31/2025 02:57:24.899 7f1c4f4a9048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:57:24.899 7f1c4f4a9048 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:57:24.899 7f1c4f4a9048 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:57:24.899 7f1c4f4a9048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:57:24.899 7f1c4f4a9048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:57:24.899 7f1c4f4a9048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:57:24.900 7f1c4f4a9048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:57:24.900 7f1c4f4a9048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:57:24.900 7f1c4f4a9048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:57:28.901 7f1c508b8d68 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:57:32.457 7f1c50856ac8 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/31/2025 02:57:33.903 7f1c508b8d68 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:57:33.904 7f1c508b8d68 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:57:33.904 7f1c508b8d68 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:57:33.904 7f1c508b8d68 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:57:33.904 7f1c508b8d68 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:57:33.904 7f1c508b8d68 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:57:33.904 7f1c508b8d68 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:57:33.904 7f1c508b8d68 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:57:33.905 7f1c508b8d68 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:57:33.905 7f1c508b8d68 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:57:33.905 7f1c508b8d68 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:57:33.905 7f1c508b8d68 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 1 pid = 1 http_port = 8100 start_time = 1743387251934780 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:57:33.905 7f1c508b8d68 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:57:37.908 7f1c508b8d68 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]