03/27/2025 02:27:50.566 7fbe1aebcc08 [contrail-config-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 03/27/2025 02:27:50.566 7fbe1aebcc08 [contrail-config-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 03/27/2025 02:27:50.566 7fbe1aebcc08 [contrail-config-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 03/27/2025 02:27:50.574 7fbe17eef0f0 [contrail-config-nodemgr] [ERROR]: Cannot write http_port 8100 to /tmp/contrail-config-nodemgr.0.http_port 03/27/2025 02:27:50.575 7fbe17eef0f0 [contrail-config-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8100 03/27/2025 02:27:50.575 7fbe17ee2a20 [contrail-config-nodemgr] [INFO]: Processing event[EvStart] in state[none] 03/27/2025 02:27:50.576 7fbe17ee2a20 [contrail-config-nodemgr] [DEBUG]: Add uve in the [ObjectConfigNode:NodeStatusUVE] map 03/27/2025 02:27:50.576 7fbe17ee2a20 [contrail-config-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/27/2025 02:27:50.576 7fbe17ee2a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 03/27/2025 02:27:50.576 7fbe17ee2a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:27:50.577 7fbe17ee2a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/27/2025 02:27:50.577 7fbe17ee2a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:27:50.577 7fbe17ee2a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:27:50.577 7fbe17ee2a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 03/27/2025 02:27:50.577 7fbe17ee2a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:27:50.698 7fbe1aebcc08 [contrail-config-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'contrail-api', 'group': 'contrail-api', 'pid': 92527938826172213658743430405149543539287638896777861420028921783940391633250, 'start': '1743042468000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-schema', 'group': 'contrail-schema', 'pid': 8306666438723660893597428160419144019780983688098030022177629772931638880340, 'start': '1743042468000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-svc-monitor', 'group': 'contrail-svc-monitor', 'pid': 33171598328521256683660165625024236320095425724974135458668572274685089333248, 'start': '1743042468000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-device-manager', 'group': 'contrail-device-manager', 'pid': 50346971846387209528851129190405230444682765056639493512080552757296737932935, 'start': '1743042469000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-config-nodemgr', 'group': 'contrail-config-nodemgr', 'pid': 60943549376461050803592557877894024435889761624505023204273620250438698149751, 'start': '1743042468000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 03/27/2025 02:27:50.828 7fbe1aebcc08 [contrail-config-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-config-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2966-1.', _node_type='Config', _instance_id='0', _timestamp=1743042470827923, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2966-1.', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-config-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigNode'), _seqnum=0) 03/27/2025 02:27:50.828 7fbe1aebcc08 [contrail-config-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2966-1.', _node_type='Config', _instance_id='0', _timestamp=1743042470828470, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2966-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='contrail-api', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1743042468000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-schema', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1743042468000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-svc-monitor', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1743042468000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-device-manager', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1743042469000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1743042468000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2025-03-27 00:52:08.896410", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-8321-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigNode'), _seqnum=0) 03/27/2025 02:27:50.831 7fbe21a61a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:27:50.832 7fbe21a61a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-03-27 00:52:08.896410", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-8321-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 8 num_core_per_socket = 8 num_thread_per_core = 1 >> running_package_version = nightly-nightly.el7 installed_package_version = nightly-nightly.el7 >> 03/27/2025 02:27:50.832 7fbe21a61a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:27:50.832 7fbe21a61a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:27:50.832 7fbe21a61a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/27/2025 02:27:50.832 7fbe21a61a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:27:50.832 7fbe21a61a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:27:50.832 7fbe21a61a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. deleted = False process_info = [ << process_name = contrail-api process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1743042468000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-schema process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1743042468000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-svc-monitor process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1743042468000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-device-manager process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1743042469000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1743042468000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-03-27 00:52:08.896410", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-8321-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} >> 03/27/2025 02:27:50.832 7fbe21a61a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:27:54.576 7fbe1867fc18 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:27:54.577 7fbe1867fc18 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:27:54.577 7fbe1867fc18 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:27:54.578 7fbe1867fc18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:27:54.578 7fbe1867fc18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:27:54.578 7fbe1867fc18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:27:54.578 7fbe1867fc18 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:27:54.578 7fbe1867fc18 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:27:54.578 7fbe1867fc18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:27:54.578 7fbe1867fc18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:27:54.578 7fbe1867fc18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:27:54.578 7fbe1867fc18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:27:54.578 7fbe1867fc18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:27:54.579 7fbe1867fc18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:27:58.579 7fbe16e8dc18 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:27:58.580 7fbe16e8dc18 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:27:58.580 7fbe16e8dc18 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:27:58.580 7fbe16e8dc18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:27:58.581 7fbe16e8dc18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:27:58.581 7fbe16e8dc18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:27:58.581 7fbe16e8dc18 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:27:58.581 7fbe16e8dc18 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:27:58.581 7fbe16e8dc18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:27:58.581 7fbe16e8dc18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:27:58.581 7fbe16e8dc18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:27:58.581 7fbe16e8dc18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:27:58.581 7fbe16e8dc18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:27:58.581 7fbe16e8dc18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:00.534 7fbe1aebcc08 [contrail-config-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-config-nodemgr', _source='cn-jenkins-deploy-platform-ansible-os-2966-1.', _node_type='Config', _instance_id='0', _timestamp=1743042480534726, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2966-1.', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='contrail-api', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1743042468000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-schema', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1743042468000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-svc-monitor', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1743042468000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-device-manager', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1743042469000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-config-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1743042468000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2025-03-27 00:52:08.896410", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-8321-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectConfigNode'), _seqnum=0) 03/27/2025 02:28:00.535 7fbe1aebcc08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2966-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=13721712, partition_space_available_1k=67368772, percentage_partition_space_used=17, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=1616, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=20476, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=93600, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=158276, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50904, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=5783348, free=12560444, buffers=652716, cached=13668604, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=4.72, five_min_avg=2.35, fifteen_min_avg=1.4, cpu_share=0, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/27/2025 02:28:00.535 7fbe18695438 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:00.535 7fbe18695438 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. deleted = False process_info = [ << process_name = contrail-api process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1743042468000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-schema process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1743042468000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-svc-monitor process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1743042468000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-device-manager process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1743042469000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-config-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1743042468000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-03-27 00:52:08.896410", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-8321-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} >> 03/27/2025 02:28:00.535 7fbe18695438 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:00.535 7fbe18695438 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:00.535 7fbe18695438 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 13721712 partition_space_available_1k = 67368772 percentage_partition_space_used = 17 >> } all_core_file_list = [ ] process_mem_cpu_usage = { contrail-api : << mem_virt = 1616 cpu_share = 0 mem_res = 0 >>contrail-schema : << mem_virt = 20476 cpu_share = 0 mem_res = 0 >>contrail-svc-monitor : << mem_virt = 93600 cpu_share = 0 mem_res = 0 >>contrail-device-manager : << mem_virt = 158276 cpu_share = 0 mem_res = 0 >>contrail-config-nodemgr : << mem_virt = 50904 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 5783348 free = 12560444 buffers = 652716 cached = 13668604 node_type = config-node >> system_cpu_usage = << one_min_avg = 4.72 five_min_avg = 2.35 fifteen_min_avg = 1.4 cpu_share = 0 node_type = config-node >> >> 03/27/2025 02:28:00.535 7fbe18695438 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:02.583 7fbe18695438 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:28:02.584 7fbe18695438 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:28:02.584 7fbe18695438 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:28:02.584 7fbe18695438 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:28:02.584 7fbe18695438 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:28:02.585 7fbe18695438 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:28:02.585 7fbe18695438 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:28:02.585 7fbe18695438 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:28:02.586 7fbe18695438 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:02.586 7fbe18695438 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:28:02.586 7fbe18695438 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:02.586 7fbe18695438 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:02.586 7fbe18695438 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:28:02.586 7fbe18695438 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:06.585 7fbe17ee6eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:28:06.586 7fbe17ee6eb8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:28:06.587 7fbe17ee6eb8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:28:06.587 7fbe17ee6eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:28:06.587 7fbe17ee6eb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:28:06.587 7fbe17ee6eb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:28:06.587 7fbe17ee6eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:28:06.588 7fbe17ee6eb8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:28:06.588 7fbe17ee6eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:06.588 7fbe17ee6eb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:28:06.588 7fbe17ee6eb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:06.588 7fbe17ee6eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:06.588 7fbe17ee6eb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:28:06.588 7fbe17ee6eb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:10.587 7fbe17ee6eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:28:10.588 7fbe17ee6eb8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:28:10.589 7fbe17ee6eb8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:28:10.589 7fbe17ee6eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:28:10.589 7fbe17ee6eb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:28:10.589 7fbe17ee6eb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:28:10.589 7fbe17ee6eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:28:10.589 7fbe17ee6eb8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:28:10.589 7fbe17ee6eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:10.589 7fbe17ee6eb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:28:10.589 7fbe17ee6eb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:10.589 7fbe17ee6eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:10.590 7fbe17ee6eb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:28:10.590 7fbe17ee6eb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:14.591 7fbe17ee6eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:28:14.593 7fbe17ee6eb8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:28:14.593 7fbe17ee6eb8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:28:14.593 7fbe17ee6eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:28:14.593 7fbe17ee6eb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:28:14.593 7fbe17ee6eb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:28:14.593 7fbe17ee6eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:28:14.593 7fbe17ee6eb8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:28:14.593 7fbe17ee6eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:14.594 7fbe17ee6eb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:28:14.594 7fbe17ee6eb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:14.594 7fbe17ee6eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:14.594 7fbe17ee6eb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:28:14.594 7fbe17ee6eb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:18.597 7fbe17ee6eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:28:18.598 7fbe17ee6eb8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:28:18.599 7fbe17ee6eb8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:28:18.599 7fbe17ee6eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:28:18.599 7fbe17ee6eb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:28:18.599 7fbe17ee6eb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:28:18.599 7fbe17ee6eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:28:18.599 7fbe17ee6eb8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:28:18.599 7fbe17ee6eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:18.599 7fbe17ee6eb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:28:18.599 7fbe17ee6eb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:18.599 7fbe17ee6eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:18.600 7fbe17ee6eb8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:28:18.600 7fbe17ee6eb8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:22.600 7fbe16e75048 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:28:22.601 7fbe16e75048 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:28:22.601 7fbe16e75048 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:28:22.601 7fbe16e75048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:28:22.602 7fbe16e75048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:28:22.602 7fbe16e75048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:28:22.602 7fbe16e75048 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:28:22.602 7fbe16e75048 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:28:22.602 7fbe16e75048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:22.602 7fbe16e75048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:28:22.602 7fbe16e75048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:22.602 7fbe16e75048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:22.602 7fbe16e75048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:28:22.602 7fbe16e75048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:26.603 7fbe16e75048 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:28:26.604 7fbe16e75048 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:28:26.604 7fbe16e75048 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:28:26.604 7fbe16e75048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:28:26.604 7fbe16e75048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:28:26.604 7fbe16e75048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:28:26.604 7fbe16e75048 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:28:26.605 7fbe16e75048 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:28:26.605 7fbe16e75048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:26.605 7fbe16e75048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:28:26.605 7fbe16e75048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:26.605 7fbe16e75048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:26.605 7fbe16e75048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:28:26.605 7fbe16e75048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:30.606 7fbe16e75048 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:28:30.607 7fbe16e75048 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:28:30.607 7fbe16e75048 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:28:30.607 7fbe16e75048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:28:30.607 7fbe16e75048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:28:30.608 7fbe16e75048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:28:30.608 7fbe16e75048 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:28:30.608 7fbe16e75048 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:28:30.608 7fbe16e75048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:30.608 7fbe16e75048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:28:30.608 7fbe16e75048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:30.608 7fbe16e75048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:30.608 7fbe16e75048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:28:30.608 7fbe16e75048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:34.612 7fbe16e75048 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:28:34.613 7fbe16e75048 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:28:34.613 7fbe16e75048 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:28:34.613 7fbe16e75048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:28:34.613 7fbe16e75048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:28:34.613 7fbe16e75048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:28:34.613 7fbe16e75048 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:28:34.614 7fbe16e75048 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:28:34.614 7fbe16e75048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:34.614 7fbe16e75048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:28:34.614 7fbe16e75048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:34.614 7fbe16e75048 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:34.614 7fbe16e75048 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:28:34.614 7fbe16e75048 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:38.616 7fbe16e93ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:28:38.617 7fbe16e93ac8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:28:38.617 7fbe16e93ac8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:28:38.617 7fbe16e93ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:28:38.617 7fbe16e93ac8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:28:38.617 7fbe16e93ac8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:28:38.617 7fbe16e93ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:28:38.618 7fbe16e93ac8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:28:38.618 7fbe16e93ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:38.618 7fbe16e93ac8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:28:38.618 7fbe16e93ac8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:38.618 7fbe16e93ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:38.618 7fbe16e93ac8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:28:38.618 7fbe16e93ac8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:42.618 7fbe16e93ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:28:42.620 7fbe16e93ac8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:28:42.620 7fbe16e93ac8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:28:42.620 7fbe16e93ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:28:42.620 7fbe16e93ac8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:28:42.620 7fbe16e93ac8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:28:42.620 7fbe16e93ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:28:42.621 7fbe16e93ac8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:28:42.621 7fbe16e93ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:42.621 7fbe16e93ac8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:28:42.621 7fbe16e93ac8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:42.621 7fbe16e93ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:42.621 7fbe16e93ac8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:28:42.621 7fbe16e93ac8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:46.623 7fbe16e93ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:28:46.624 7fbe16e93ac8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:28:46.624 7fbe16e93ac8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:28:46.624 7fbe16e93ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:28:46.624 7fbe16e93ac8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:28:46.624 7fbe16e93ac8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:28:46.625 7fbe16e93ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:28:46.625 7fbe16e93ac8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:28:46.625 7fbe16e93ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:46.625 7fbe16e93ac8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:28:46.625 7fbe16e93ac8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:46.625 7fbe16e93ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:46.625 7fbe16e93ac8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:28:46.625 7fbe16e93ac8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:50.629 7fbe16e93ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:28:50.630 7fbe16e93ac8 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:28:50.630 7fbe16e93ac8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:28:50.630 7fbe16e93ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:28:50.630 7fbe16e93ac8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:28:50.630 7fbe16e93ac8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:28:50.630 7fbe16e93ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:28:50.631 7fbe16e93ac8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:28:50.631 7fbe16e93ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:50.631 7fbe16e93ac8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:28:50.631 7fbe16e93ac8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:50.631 7fbe16e93ac8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:50.631 7fbe16e93ac8 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:28:50.631 7fbe16e93ac8 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:54.631 7fbe18694a20 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:28:54.632 7fbe18694a20 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:28:54.633 7fbe18694a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:28:54.633 7fbe18694a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:28:54.633 7fbe18694a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:28:54.633 7fbe18694a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:28:54.633 7fbe18694a20 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:28:54.633 7fbe18694a20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:28:54.633 7fbe18694a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:54.633 7fbe18694a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:28:54.633 7fbe18694a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:54.634 7fbe18694a20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:54.634 7fbe18694a20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:28:54.634 7fbe18694a20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:58.633 7fbe21b410f0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:28:58.634 7fbe21b410f0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:28:58.634 7fbe21b410f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:28:58.634 7fbe21b410f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:28:58.634 7fbe21b410f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:28:58.634 7fbe21b410f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:28:58.634 7fbe21b410f0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:28:58.635 7fbe21b410f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:28:58.635 7fbe21b410f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:58.635 7fbe21b410f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:28:58.635 7fbe21b410f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:58.635 7fbe21b410f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:28:58.635 7fbe21b410f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:28:58.635 7fbe21b410f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:00.602 7fbe1aebcc08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2966-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14454788, partition_space_available_1k=66635696, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=97168, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=80104, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88608, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=197524, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50956, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=6168552, free=11691884, buffers=664388, cached=14140288, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.95, five_min_avg=2.3, fifteen_min_avg=1.44, cpu_share=3.0, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/27/2025 02:29:00.603 7fbe18694f60 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:00.603 7fbe18694f60 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 14454788 partition_space_available_1k = 66635696 percentage_partition_space_used = 18 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 97168 cpu_share = 0 mem_res = 0 >>contrail-schema : << mem_virt = 80104 cpu_share = 0 mem_res = 0 >>contrail-svc-monitor : << mem_virt = 88608 cpu_share = 0 mem_res = 0 >>contrail-device-manager : << mem_virt = 197524 cpu_share = 0 mem_res = 0 >>contrail-config-nodemgr : << mem_virt = 50956 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 6168552 free = 11691884 buffers = 664388 cached = 14140288 node_type = config-node >> system_cpu_usage = << one_min_avg = 2.95 five_min_avg = 2.3 fifteen_min_avg = 1.44 cpu_share = 3.0 node_type = config-node >> >> 03/27/2025 02:29:00.603 7fbe18694f60 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:02.636 7fbe1ac7d828 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:29:02.638 7fbe1ac7d828 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:29:02.638 7fbe1ac7d828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:29:02.638 7fbe1ac7d828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:29:02.638 7fbe1ac7d828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:29:02.638 7fbe1ac7d828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:29:02.638 7fbe1ac7d828 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:29:02.639 7fbe1ac7d828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:29:02.639 7fbe1ac7d828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:02.639 7fbe1ac7d828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:29:02.639 7fbe1ac7d828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:02.639 7fbe1ac7d828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:02.639 7fbe1ac7d828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:29:02.639 7fbe1ac7d828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:06.638 7fbe16e9aa20 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:29:06.639 7fbe16e9aa20 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:29:06.639 7fbe16e9aa20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:29:06.639 7fbe16e9aa20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:29:06.639 7fbe16e9aa20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:29:06.639 7fbe16e9aa20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:29:06.640 7fbe16e9aa20 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:29:06.640 7fbe16e9aa20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:29:06.640 7fbe16e9aa20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:06.640 7fbe16e9aa20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:29:06.640 7fbe16e9aa20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:06.640 7fbe16e9aa20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:06.640 7fbe16e9aa20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:29:06.640 7fbe16e9aa20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:10.642 7fbe16e9aa20 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:29:10.643 7fbe16e9aa20 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:29:10.643 7fbe16e9aa20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:29:10.643 7fbe16e9aa20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:29:10.644 7fbe16e9aa20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:29:10.644 7fbe16e9aa20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:29:10.644 7fbe16e9aa20 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:29:10.644 7fbe16e9aa20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:29:10.644 7fbe16e9aa20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:10.644 7fbe16e9aa20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:29:10.644 7fbe16e9aa20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:10.645 7fbe16e9aa20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:10.645 7fbe16e9aa20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:29:10.645 7fbe16e9aa20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:14.647 7fbe16e9aa20 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:29:14.648 7fbe16e9aa20 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:29:14.648 7fbe16e9aa20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:29:14.648 7fbe16e9aa20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:29:14.648 7fbe16e9aa20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:29:14.648 7fbe16e9aa20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:29:14.648 7fbe16e9aa20 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:29:14.649 7fbe16e9aa20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:29:14.649 7fbe16e9aa20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:14.649 7fbe16e9aa20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:29:14.649 7fbe16e9aa20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:14.649 7fbe16e9aa20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:14.649 7fbe16e9aa20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:29:14.649 7fbe16e9aa20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:18.650 7fbe16e9aa20 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:29:18.651 7fbe16e9aa20 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:29:18.651 7fbe16e9aa20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:29:18.651 7fbe16e9aa20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:29:18.651 7fbe16e9aa20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:29:18.651 7fbe16e9aa20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:29:18.652 7fbe16e9aa20 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:29:18.652 7fbe16e9aa20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:29:18.652 7fbe16e9aa20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:18.652 7fbe16e9aa20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:29:18.652 7fbe16e9aa20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:18.652 7fbe16e9aa20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:18.652 7fbe16e9aa20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:29:18.652 7fbe16e9aa20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:22.652 7fbe17eeea20 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:29:22.653 7fbe17eeea20 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:29:22.653 7fbe17eeea20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:29:22.654 7fbe17eeea20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:29:22.654 7fbe17eeea20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:29:22.654 7fbe17eeea20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:29:22.654 7fbe17eeea20 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:29:22.654 7fbe17eeea20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:29:22.654 7fbe17eeea20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:22.654 7fbe17eeea20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:29:22.654 7fbe17eeea20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:22.654 7fbe17eeea20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:22.655 7fbe17eeea20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:29:22.655 7fbe17eeea20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:26.658 7fbe17eeea20 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:29:26.659 7fbe17eeea20 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:29:26.659 7fbe17eeea20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:29:26.659 7fbe17eeea20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:29:26.659 7fbe17eeea20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:29:26.660 7fbe17eeea20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:29:26.660 7fbe17eeea20 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:29:26.660 7fbe17eeea20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:29:26.660 7fbe17eeea20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:26.660 7fbe17eeea20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:29:26.660 7fbe17eeea20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:26.660 7fbe17eeea20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:26.660 7fbe17eeea20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:29:26.660 7fbe17eeea20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:30.664 7fbe17eeea20 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:29:30.665 7fbe17eeea20 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:29:30.665 7fbe17eeea20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:29:30.665 7fbe17eeea20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:29:30.665 7fbe17eeea20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:29:30.665 7fbe17eeea20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:29:30.666 7fbe17eeea20 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:29:30.666 7fbe17eeea20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:29:30.666 7fbe17eeea20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:30.666 7fbe17eeea20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:29:30.666 7fbe17eeea20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:30.667 7fbe17eeea20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:30.667 7fbe17eeea20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:29:30.667 7fbe17eeea20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:34.666 7fbe17eeea20 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:29:34.667 7fbe17eeea20 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:29:34.667 7fbe17eeea20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:29:34.668 7fbe17eeea20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:29:34.668 7fbe17eeea20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:29:34.668 7fbe17eeea20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:29:34.668 7fbe17eeea20 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:29:34.668 7fbe17eeea20 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:29:34.668 7fbe17eeea20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:34.669 7fbe17eeea20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:29:34.669 7fbe17eeea20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:34.669 7fbe17eeea20 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:34.669 7fbe17eeea20 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:29:34.669 7fbe17eeea20 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:38.669 7fbe1ad0e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:29:38.671 7fbe1ad0e0f0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:29:38.671 7fbe1ad0e0f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:29:38.671 7fbe1ad0e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:29:38.671 7fbe1ad0e0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:29:38.671 7fbe1ad0e0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:29:38.671 7fbe1ad0e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:29:38.671 7fbe1ad0e0f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:29:38.671 7fbe1ad0e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:38.671 7fbe1ad0e0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:29:38.672 7fbe1ad0e0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:38.672 7fbe1ad0e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:38.672 7fbe1ad0e0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:29:38.672 7fbe1ad0e0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:42.674 7fbe1ad0e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:29:42.675 7fbe1ad0e0f0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:29:42.675 7fbe1ad0e0f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:29:42.675 7fbe1ad0e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:29:42.675 7fbe1ad0e0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:29:42.675 7fbe1ad0e0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:29:42.675 7fbe1ad0e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:29:42.676 7fbe1ad0e0f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:29:42.676 7fbe1ad0e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:42.676 7fbe1ad0e0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:29:42.676 7fbe1ad0e0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:42.676 7fbe1ad0e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:42.676 7fbe1ad0e0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:29:42.676 7fbe1ad0e0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:46.676 7fbe1ad0e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:29:46.677 7fbe1ad0e0f0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:29:46.677 7fbe1ad0e0f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:29:46.677 7fbe1ad0e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:29:46.677 7fbe1ad0e0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:29:46.677 7fbe1ad0e0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:29:46.677 7fbe1ad0e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:29:46.678 7fbe1ad0e0f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:29:46.678 7fbe1ad0e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:46.678 7fbe1ad0e0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:29:46.678 7fbe1ad0e0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:46.678 7fbe1ad0e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:46.678 7fbe1ad0e0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:29:46.678 7fbe1ad0e0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:50.680 7fbe1ad0e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:29:50.681 7fbe1ad0e0f0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:29:50.682 7fbe1ad0e0f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:29:50.682 7fbe1ad0e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:29:50.682 7fbe1ad0e0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:29:50.682 7fbe1ad0e0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:29:50.682 7fbe1ad0e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:29:50.682 7fbe1ad0e0f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:29:50.682 7fbe1ad0e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:50.682 7fbe1ad0e0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:29:50.682 7fbe1ad0e0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:50.682 7fbe1ad0e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:50.683 7fbe1ad0e0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:29:50.683 7fbe1ad0e0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:54.682 7fbe16e988d0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:29:54.684 7fbe16e988d0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:29:54.684 7fbe16e988d0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:29:54.684 7fbe16e988d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:29:54.685 7fbe16e988d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:29:54.685 7fbe16e988d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:29:54.685 7fbe16e988d0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:29:54.685 7fbe16e988d0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:29:54.685 7fbe16e988d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:54.685 7fbe16e988d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:29:54.685 7fbe16e988d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:54.685 7fbe16e988d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:54.686 7fbe16e988d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:29:54.686 7fbe16e988d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:58.685 7fbe17ef7438 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:29:58.686 7fbe17ef7438 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:29:58.686 7fbe17ef7438 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:29:58.686 7fbe17ef7438 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:29:58.686 7fbe17ef7438 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:29:58.686 7fbe17ef7438 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:29:58.687 7fbe17ef7438 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:29:58.687 7fbe17ef7438 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:29:58.687 7fbe17ef7438 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:58.687 7fbe17ef7438 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:29:58.687 7fbe17ef7438 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:58.687 7fbe17ef7438 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:29:58.687 7fbe17ef7438 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:29:58.687 7fbe17ef7438 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:00.565 7fbe1aebcc08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2966-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=14865352, partition_space_available_1k=66225132, percentage_partition_space_used=18, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=97232, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=80204, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88668, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=197636, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51004, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=6394536, free=11002316, buffers=676940, cached=14591320, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=1.73, five_min_avg=2.06, fifteen_min_avg=1.41, cpu_share=2.5, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/27/2025 02:30:00.566 7fbe1c11a630 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:00.566 7fbe1c11a630 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 14865352 partition_space_available_1k = 66225132 percentage_partition_space_used = 18 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 97232 cpu_share = 0 mem_res = 0 >>contrail-schema : << mem_virt = 80204 cpu_share = 0 mem_res = 0 >>contrail-svc-monitor : << mem_virt = 88668 cpu_share = 0 mem_res = 0 >>contrail-device-manager : << mem_virt = 197636 cpu_share = 0 mem_res = 0 >>contrail-config-nodemgr : << mem_virt = 51004 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 6394536 free = 11002316 buffers = 676940 cached = 14591320 node_type = config-node >> system_cpu_usage = << one_min_avg = 1.73 five_min_avg = 2.06 fifteen_min_avg = 1.41 cpu_share = 2.5 node_type = config-node >> >> 03/27/2025 02:30:00.566 7fbe1c11a630 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:02.688 7fbe1c11a630 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:30:02.689 7fbe1c11a630 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:30:02.689 7fbe1c11a630 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:30:02.689 7fbe1c11a630 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:30:02.689 7fbe1c11a630 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:30:02.690 7fbe1c11a630 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:30:02.690 7fbe1c11a630 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:30:02.690 7fbe1c11a630 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:30:02.690 7fbe1c11a630 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:02.690 7fbe1c11a630 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:30:02.690 7fbe1c11a630 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:02.690 7fbe1c11a630 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:02.690 7fbe1c11a630 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:30:02.690 7fbe1c11a630 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:06.694 7fbe1c11a630 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:30:06.695 7fbe1c11a630 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:30:06.695 7fbe1c11a630 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:30:06.696 7fbe1c11a630 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:30:06.696 7fbe1c11a630 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:30:06.696 7fbe1c11a630 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:30:06.696 7fbe1c11a630 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:30:06.696 7fbe1c11a630 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:30:06.696 7fbe1c11a630 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:06.696 7fbe1c11a630 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:30:06.696 7fbe1c11a630 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:06.696 7fbe1c11a630 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:06.696 7fbe1c11a630 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:30:06.697 7fbe1c11a630 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:10.698 7fbe167b9240 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:30:10.699 7fbe167b9240 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:30:10.699 7fbe167b9240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:30:10.699 7fbe167b9240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:30:10.699 7fbe167b9240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:30:10.699 7fbe167b9240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:30:10.699 7fbe167b9240 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:30:10.700 7fbe167b9240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:30:10.700 7fbe167b9240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:10.700 7fbe167b9240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:30:10.700 7fbe167b9240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:10.700 7fbe167b9240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:10.700 7fbe167b9240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:30:10.700 7fbe167b9240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:14.700 7fbe167b9240 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:30:14.701 7fbe167b9240 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:30:14.701 7fbe167b9240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:30:14.701 7fbe167b9240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:30:14.701 7fbe167b9240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:30:14.701 7fbe167b9240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:30:14.702 7fbe167b9240 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:30:14.702 7fbe167b9240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:30:14.702 7fbe167b9240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:14.702 7fbe167b9240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:30:14.702 7fbe167b9240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:14.702 7fbe167b9240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:14.702 7fbe167b9240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:30:14.702 7fbe167b9240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:18.702 7fbe167b9240 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:30:18.703 7fbe167b9240 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:30:18.703 7fbe167b9240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:30:18.703 7fbe167b9240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:30:18.703 7fbe167b9240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:30:18.703 7fbe167b9240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:30:18.703 7fbe167b9240 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:30:18.704 7fbe167b9240 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:30:18.704 7fbe167b9240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:18.704 7fbe167b9240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:30:18.704 7fbe167b9240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:18.704 7fbe167b9240 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:18.704 7fbe167b9240 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:30:18.704 7fbe167b9240 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:22.703 7fbe1c11ac18 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:30:22.704 7fbe1c11ac18 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:30:22.705 7fbe1c11ac18 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:30:22.705 7fbe1c11ac18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:30:22.705 7fbe1c11ac18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:30:22.705 7fbe1c11ac18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:30:22.705 7fbe1c11ac18 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:30:22.705 7fbe1c11ac18 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:30:22.705 7fbe1c11ac18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:22.706 7fbe1c11ac18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:30:22.706 7fbe1c11ac18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:22.706 7fbe1c11ac18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:22.706 7fbe1c11ac18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:30:22.706 7fbe1c11ac18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:26.707 7fbe1c11ac18 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:30:26.709 7fbe1c11ac18 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:30:26.709 7fbe1c11ac18 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:30:26.709 7fbe1c11ac18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:30:26.709 7fbe1c11ac18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:30:26.709 7fbe1c11ac18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:30:26.709 7fbe1c11ac18 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:30:26.709 7fbe1c11ac18 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:30:26.709 7fbe1c11ac18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:26.710 7fbe1c11ac18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:30:26.710 7fbe1c11ac18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:26.710 7fbe1c11ac18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:26.710 7fbe1c11ac18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:30:26.710 7fbe1c11ac18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:30.711 7fbe1c11ac18 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:30:30.712 7fbe1c11ac18 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:30:30.712 7fbe1c11ac18 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:30:30.712 7fbe1c11ac18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:30:30.712 7fbe1c11ac18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:30:30.712 7fbe1c11ac18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:30:30.712 7fbe1c11ac18 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:30:30.713 7fbe1c11ac18 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:30:30.713 7fbe1c11ac18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:30.713 7fbe1c11ac18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:30:30.713 7fbe1c11ac18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:30.713 7fbe1c11ac18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:30.713 7fbe1c11ac18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:30:30.713 7fbe1c11ac18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:34.716 7fbe1c11ac18 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:30:34.717 7fbe1c11ac18 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:30:34.718 7fbe1c11ac18 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:30:34.718 7fbe1c11ac18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:30:34.718 7fbe1c11ac18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:30:34.718 7fbe1c11ac18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:30:34.718 7fbe1c11ac18 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:30:34.718 7fbe1c11ac18 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:30:34.718 7fbe1c11ac18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:34.719 7fbe1c11ac18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:30:34.719 7fbe1c11ac18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:34.719 7fbe1c11ac18 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:34.719 7fbe1c11ac18 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:30:34.719 7fbe1c11ac18 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:38.719 7fbe18695438 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:30:38.720 7fbe18695438 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:30:38.720 7fbe18695438 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:30:38.720 7fbe18695438 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:30:38.720 7fbe18695438 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:30:38.720 7fbe18695438 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:30:38.720 7fbe18695438 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:30:38.721 7fbe18695438 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:30:38.721 7fbe18695438 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:38.721 7fbe18695438 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:30:38.721 7fbe18695438 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:38.721 7fbe18695438 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:38.721 7fbe18695438 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:30:38.721 7fbe18695438 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:42.722 7fbe18695438 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:30:42.724 7fbe18695438 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:30:42.724 7fbe18695438 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:30:42.724 7fbe18695438 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:30:42.724 7fbe18695438 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:30:42.724 7fbe18695438 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:30:42.724 7fbe18695438 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:30:42.725 7fbe18695438 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:30:42.725 7fbe18695438 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:42.725 7fbe18695438 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:30:42.725 7fbe18695438 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:42.725 7fbe18695438 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:42.725 7fbe18695438 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:30:42.725 7fbe18695438 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:46.726 7fbe18695438 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:30:46.727 7fbe18695438 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:30:46.728 7fbe18695438 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:30:46.728 7fbe18695438 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:30:46.728 7fbe18695438 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:30:46.728 7fbe18695438 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:30:46.728 7fbe18695438 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:30:46.728 7fbe18695438 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:30:46.729 7fbe18695438 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:46.729 7fbe18695438 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:30:46.729 7fbe18695438 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:46.729 7fbe18695438 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:46.729 7fbe18695438 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:30:46.729 7fbe18695438 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:50.730 7fbe18695438 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:30:50.731 7fbe18695438 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:30:50.731 7fbe18695438 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:30:50.731 7fbe18695438 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:30:50.731 7fbe18695438 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:30:50.731 7fbe18695438 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:30:50.731 7fbe18695438 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:30:50.732 7fbe18695438 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:30:50.732 7fbe18695438 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:50.732 7fbe18695438 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:30:50.732 7fbe18695438 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:50.732 7fbe18695438 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:50.732 7fbe18695438 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:30:50.732 7fbe18695438 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:54.734 7fbe18694198 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:30:54.735 7fbe18694198 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:30:54.735 7fbe18694198 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:30:54.735 7fbe18694198 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:30:54.736 7fbe18694198 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:30:54.736 7fbe18694198 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:30:54.736 7fbe18694198 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:30:54.736 7fbe18694198 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:30:54.736 7fbe18694198 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:54.736 7fbe18694198 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:30:54.736 7fbe18694198 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:54.736 7fbe18694198 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:54.736 7fbe18694198 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:30:54.737 7fbe18694198 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:58.736 7fbe186a48d0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:30:58.737 7fbe186a48d0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:30:58.738 7fbe186a48d0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:30:58.738 7fbe186a48d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:30:58.738 7fbe186a48d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:30:58.738 7fbe186a48d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:30:58.738 7fbe186a48d0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:30:58.738 7fbe186a48d0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:30:58.738 7fbe186a48d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:58.739 7fbe186a48d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:30:58.739 7fbe186a48d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:58.739 7fbe186a48d0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:30:58.739 7fbe186a48d0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:30:58.739 7fbe186a48d0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:31:00.751 7fbe1aebcc08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2966-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15295400, partition_space_available_1k=65795084, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=97308, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=80284, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88728, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=197724, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51056, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=7748952, free=9161320, buffers=690496, cached=15064344, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.44, five_min_avg=2.21, fifteen_min_avg=1.5, cpu_share=3.0, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/27/2025 02:31:00.751 7fbe167e1828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:31:00.752 7fbe167e1828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 15295400 partition_space_available_1k = 65795084 percentage_partition_space_used = 19 >> } process_mem_cpu_usage = { contrail-api : << mem_virt = 97308 cpu_share = 0 mem_res = 0 >>contrail-schema : << mem_virt = 80284 cpu_share = 0 mem_res = 0 >>contrail-svc-monitor : << mem_virt = 88728 cpu_share = 0 mem_res = 0 >>contrail-device-manager : << mem_virt = 197724 cpu_share = 0 mem_res = 0 >>contrail-config-nodemgr : << mem_virt = 51056 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 7748952 free = 9161320 buffers = 690496 cached = 15064344 node_type = config-node >> system_cpu_usage = << one_min_avg = 2.44 five_min_avg = 2.21 fifteen_min_avg = 1.5 cpu_share = 3.0 node_type = config-node >> >> 03/27/2025 02:31:00.752 7fbe167e1828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:31:02.740 7fbe167e1828 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:31:02.741 7fbe167e1828 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:31:02.741 7fbe167e1828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:31:02.741 7fbe167e1828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:31:02.741 7fbe167e1828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:31:02.741 7fbe167e1828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:31:02.741 7fbe167e1828 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:31:02.742 7fbe167e1828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:31:02.742 7fbe167e1828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:31:02.742 7fbe167e1828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:31:02.742 7fbe167e1828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:31:02.742 7fbe167e1828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:31:02.742 7fbe167e1828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:31:02.742 7fbe167e1828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:31:06.744 7fbe167e1828 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:31:06.745 7fbe167e1828 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:31:06.745 7fbe167e1828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:31:06.745 7fbe167e1828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:31:06.746 7fbe167e1828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:31:06.746 7fbe167e1828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:31:06.746 7fbe167e1828 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:31:06.746 7fbe167e1828 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:31:06.746 7fbe167e1828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:31:06.746 7fbe167e1828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:31:06.746 7fbe167e1828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:31:06.746 7fbe167e1828 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:31:06.746 7fbe167e1828 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:31:06.746 7fbe167e1828 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:31:10.746 7fbe16e7e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:31:10.747 7fbe16e7e0f0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:31:10.747 7fbe16e7e0f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:31:10.747 7fbe16e7e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:31:10.747 7fbe16e7e0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:31:10.747 7fbe16e7e0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:31:10.747 7fbe16e7e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:31:10.748 7fbe16e7e0f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:31:10.748 7fbe16e7e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:31:10.748 7fbe16e7e0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:31:10.748 7fbe16e7e0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:31:10.748 7fbe16e7e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:31:10.748 7fbe16e7e0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:31:10.748 7fbe16e7e0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:31:14.750 7fbe16e7e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:31:14.751 7fbe16e7e0f0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:31:14.751 7fbe16e7e0f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:31:14.751 7fbe16e7e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:31:14.752 7fbe16e7e0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:31:14.752 7fbe16e7e0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:31:14.752 7fbe16e7e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:31:14.752 7fbe16e7e0f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:31:14.752 7fbe16e7e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:31:14.752 7fbe16e7e0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:31:14.752 7fbe16e7e0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:31:14.752 7fbe16e7e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:31:14.752 7fbe16e7e0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:31:14.752 7fbe16e7e0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:31:18.756 7fbe16e7e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:31:18.757 7fbe16e7e0f0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:31:18.757 7fbe16e7e0f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:31:18.757 7fbe16e7e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:31:18.757 7fbe16e7e0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:31:18.757 7fbe16e7e0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:31:18.757 7fbe16e7e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:31:18.758 7fbe16e7e0f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:31:18.758 7fbe16e7e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:31:18.758 7fbe16e7e0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:31:18.758 7fbe16e7e0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:31:18.758 7fbe16e7e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:31:18.758 7fbe16e7e0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:31:18.758 7fbe16e7e0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:31:22.758 7fbe16e7e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:31:22.759 7fbe16e7e0f0 [contrail-config-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/27/2025 02:31:22.759 7fbe16e7e0f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:31:22.759 7fbe16e7e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:31:22.759 7fbe16e7e0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:31:22.759 7fbe16e7e0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:31:22.759 7fbe16e7e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/27/2025 02:31:22.760 7fbe16e7e0f0 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/27/2025 02:31:22.760 7fbe16e7e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:31:22.760 7fbe16e7e0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/27/2025 02:31:22.760 7fbe16e7e0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:31:22.760 7fbe16e7e0f0 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:31:22.760 7fbe16e7e0f0 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1.:Config:contrail-config-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8100 start_time = 1743042470576595 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.38:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/27/2025 02:31:22.760 7fbe16e7e0f0 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/27/2025 02:31:26.762 7fbe17eefb70 [contrail-config-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/27/2025 02:31:26.763 7fbe17eefb70 [contrail-config-nodemgr] [INFO]: Session Event: TCP Connected 03/27/2025 02:31:26.763 7fbe17eefb70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/27/2025 02:31:26.764 7fbe17eefb70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:31:26.764 7fbe17eefb70 [contrail-config-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-2966-1. process_status = [ << module_id = contrail-config-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/27/2025 02:31:26.764 7fbe17eefb70 [contrail-config-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/27/2025 02:31:26.764 7fbe17eefb70 [contrail-config-nodemgr] [INFO]: Processing event[EvTcpConnected] in state[Connect] 03/27/2025 02:31:26.764 7fbe17eefb70 [contrail-config-nodemgr] [DEBUG]: Send sandesh control message. uve type count # 6 03/27/2025 02:31:26.765 7fbe17eefb70 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 03/27/2025 02:31:26.765 7fbe17eefb70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/27/2025 02:31:26.765 7fbe17eefb70 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 03/27/2025 02:31:35.632 7fbe17ee2f60 [contrail-config-nodemgr] [DEBUG]: Extracted sandesh length: 0000001016 03/27/2025 02:31:35.633 7fbe17ee2f60 [contrail-config-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 03/27/2025 02:31:35.634 7fbe18847eb8 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 03/27/2025 02:31:35.634 7fbe18847eb8 [contrail-config-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 0 03/27/2025 02:31:35.634 7fbe18847eb8 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/27/2025 02:31:35.634 7fbe18847eb8 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/27/2025 02:31:35.634 7fbe18847eb8 [contrail-config-nodemgr] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 03/27/2025 02:31:35.635 7fbe17eefb70 [contrail-config-nodemgr] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 03/27/2025 02:31:35.636 7fbe17eefb70 [contrail-config-nodemgr] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 03/27/2025 02:31:48.513 7fbe17ee2f60 [contrail-config-nodemgr] [DEBUG]: Extracted sandesh length: 0000001297 03/27/2025 02:31:48.514 7fbe17ee2f60 [contrail-config-nodemgr] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 03/27/2025 02:31:48.514 7fbe16e7ee10 [contrail-config-nodemgr] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 03/27/2025 02:31:48.514 7fbe16e7ee10 [contrail-config-nodemgr] [DEBUG]: Number of uve types in sandesh control message is 2 03/27/2025 02:31:48.514 7fbe16e7ee10 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/27/2025 02:31:48.514 7fbe16e7ee10 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/27/2025 02:32:00.511 7fbe1aebcc08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2966-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=15779496, partition_space_available_1k=65310988, percentage_partition_space_used=19, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=97364, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=80332, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=88732, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=197788, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51160, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=8768772, free=7540680, buffers=709404, cached=15646256, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.56, five_min_avg=2.52, fifteen_min_avg=1.65, cpu_share=3.75, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/27/2025 02:33:00.493 7fbe1aebcc08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2966-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=16984356, partition_space_available_1k=64106128, percentage_partition_space_used=21, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=95372, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=78888, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=87140, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=179292, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42032, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=9005520, free=20540172, buffers=734624, cached=2384796, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=3.83, five_min_avg=2.77, fifteen_min_avg=1.79, cpu_share=4.5, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/27/2025 02:33:47.827 7fbe16e75e10 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/27/2025 02:33:49.957 7fbe18877f60 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 03/27/2025 02:34:00.506 7fbe1aebcc08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2966-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=17365168, partition_space_available_1k=63725316, percentage_partition_space_used=21, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=95460, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=78944, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=87152, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=179332, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42104, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=9576976, free=19543844, buffers=743472, cached=2800820, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.9, five_min_avg=2.7, fifteen_min_avg=1.83, cpu_share=3.25, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/27/2025 02:35:00.512 7fbe1aebcc08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2966-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=17631792, partition_space_available_1k=63458692, percentage_partition_space_used=22, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=96480, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=79012, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=87192, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=179392, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42124, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=9664068, free=18774148, buffers=754908, cached=3471988, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=2.13, five_min_avg=2.5, fifteen_min_avg=1.82, cpu_share=3.38, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/27/2025 02:36:00.536 7fbe1aebcc08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2966-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=17745860, partition_space_available_1k=63344624, percentage_partition_space_used=22, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=107760, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=82448, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=90596, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=181332, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42136, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=9878060, free=18474384, buffers=758296, cached=3554372, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=5.45, five_min_avg=3.36, fifteen_min_avg=2.16, cpu_share=5.5, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/27/2025 02:37:00.547 7fbe1aebcc08 [contrail-config-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-ansible-os-2966-1.', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=17794776, partition_space_available_1k=63295708, percentage_partition_space_used=22, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-api': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=120092, cpu_share=0, mem_res=0, _EventManager__key='contrail-api'), 'contrail-schema': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=85228, cpu_share=0, mem_res=0, _EventManager__key='contrail-schema'), 'contrail-svc-monitor': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=92756, cpu_share=0, mem_res=0, _EventManager__key='contrail-svc-monitor'), 'contrail-device-manager': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=183496, cpu_share=0, mem_res=0, _EventManager__key='contrail-device-manager'), 'contrail-config-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=42144, cpu_share=0, mem_res=0, _EventManager__key='contrail-config-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=9939280, free=18364992, buffers=758340, cached=3602500, node_type='config-node'), system_cpu_usage=SysCpuInfo(one_min_avg=6.28, five_min_avg=4.01, fifteen_min_avg=2.46, cpu_share=5.88, node_type='config-node'), running_package_version=None, installed_package_version=None, _table='ObjectConfigNode') 03/27/2025 02:37:23.213 7fbe1aa2c588 [contrail-config-nodemgr] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map